Psychological Testing

Theresa J.B. Kline (Ph.D., 1990) is a Professor of Industrial-Organizational Psychology at the University of Calgary. She has an active research program in the area of psychometrics, team performance, organizational effectiveness, and work attitudes. Theresa has published two books on teams, Teams that Lead (2003) and Remaking Teams (1999), and over 50 peer-reviewed journal articles. Theresa teaches psychometrics, statistics, methods and organizational psychology at the undergraduate and graduate level. She has an active organizational consulting practice, with projects ranging from individual and organizational assessment to strategic alignment. She has made presentations and run workshops on topics that range from statistics and methods, to how to use assessment tools for personnel decision-making in an ethical manner, to executive team development practices. Theresa has supervised nine master's theses and eight doctoral dissertations. She encourages her students to take an active role in deciding what they will study and, as a result, she has learned at least as much from them they have from her; her pupils have drawn her into research areas such as organizational learning, lifelong learning, organizational citizenship, fairness, leadership, job change, workplace stress, and performance appraisal.

Preface The Assessment of Individuals: The Critical Role and Fundamentals of Measurement Measurement in the Physical Sciences Measurement in the Social Sciences Historical Highlights of Measurement Statistics Background The First Step: Identifying the Construct Links Between Constructs Construct Cleanliness Single versus Multiple Constructs Summary and Next Step Problems and Exercises Designing and Writing Items Empirical, Theoretical and Rational Approaches to Item Empirical, Theoretical and Rational Approaches to Item Literature Search Subject Matter Experts How Many Items? Attitudinal Items: Early Work in Item Generation Assessing Behaviors Pilot Testing Summary and Next Step Problems and Exercises Designing and Scoring Responses Open-Ended Responses Closed-Ended Questions Example 1: Proportional Differences for a Single Variable Example 2: Proportional Differences for Two Variables Dichotomous Responses Multiple Choice Tests Continuous Responses Ipsative versus Normative Scales Difference and Change Scores Summary and Next Step Problems and Exercises Collecting Data: Sampling and Screening Probability Sampling Non-Probability Sampling Sample Sizes Missing Data Preparing to Analyze Your Data Summary and Next Step Problems and Exercises Classical Test Theory: Assumptions, Equations, Limitations and Item Analyses Classical Test Theory Theory of True and Error Scores: Description and Assumptions Ramifications and Limitations of Classical Test Theory Assumptions Item Analysis within CTT: Approaches, Statistical Analyses, and Interpretation Descriptive Statistics Summary Problems and Exercises Modern Test Theory: Assumptions, Equations, Limitations and Item Analyses Modern Test Theory Models One-Parameter Logistic Model Two-Parameter Logistic Model Three-Parameter Logistic Model Multiple-response IRT Models Parameter Estimation Scoring Respondents Model Fit Assumptions Ramifications of the Assumptions of Modern Test Theory Practical Advantages of Modern Test Theory Limitations of Modern Test Theory Computer Programs Practical Considerations Summary Next Steps Problems and Exercises Reliability of Test Scores and Test Items Test-retest Reliability Alternative Forms Reliability Measures of Internal Consistency Setting Confidence Intervals Reliability of a Composite Difference Scores - A Reliability Concern Practical Questions Summary and Next Steps Problems and Exercises Reliability of Raters Inter-Rater Reliability Indices Reliability Generalization Modern Test Theory Approaches to Reliability Summary and Next Steps Problems and Exercises Assessing Validity Using Content and Criterion Methods Asking the Test Takers Asking the Subject Matter Experts Assessments Using Correlation and Regression: Criterion-Related Studies Classification Approaches to Test Score Validation Group Differences and Test Bias Extending the Inferences of Criterion-Related Validity Studies Summary Problems and Exercises Assessing Validity Via Item Internal Structure Principal Components Analysis Common Factor Analysis Common Factor Analysis using Analysis of Covariance Structures Some Other Issues in Factor Analysis Practical Issues Concluding Comments on Internal Structure and Validity Threats to the Validity of Scores Multitrait-Multimethod Assessment Closing Comments on Test Score Validity Summary Problems and Exercises Ethics and Professional Issues in Testing Professional Standards and Guidelines Ethical Procedures and Protocols Test Administration Integrity Testing Computerized Testing Coaching, Testwiseness, and Re-takes Testing Legislation Test Item Bias and Adverse Impact Translation Issues Electronic Presentation and Capture Summary Problems and Exercises Brief Reviews of Some Selected Tests and Concluding Comments Information about Existing Tests Some Intelligence Tests Academic Achievement Tests Structured Personality Tests Career Interest/Guidance Instruments Chapter Summary A Quick Book Review Concluding Comments Problems and Exercises Appendices References Index

Professor Kline's attempts to de-mystify complex measurement concepts are beautifully simplified and illustrated in her countless illustrations of practical and relevant problems for the mathematically-challenged student. This book is also a must-have for those who simply do not have the desire for the theoretical jargon used in similar textbooks, but are interested in the important conceptual and practical aspects of measurement as they apply in their disciplines. -- Arturo Olivarez, Jr. Kline's Psychological Testing provides a well-written treatment of the critical issues in designing and evaluating psychometric instruments. This book will be very useful to advanced undergraduate students, graduate students, and researchers. -- Richard Block "[This book] fills an important niche...It is ideally suited to professional psychologists wishing to construct or evaluate a psychological test. Dr. Kline's sensible advice will serve them well." -- Stuart J. Mckelvie * Canadian Psychology * Psychological Testing: A Practical Approach to Design and Evaluation by Theresa J. B. Kline is an easy-to-read, upper-level textbook covering topics related not only to basic psychological testing practices but also to methodological research components. The author is very effective at presenting relevant issues, facts, descriptions, examples, and statistics in a logical and concise manner. She describes and critiques several commonly used tests and the subtleties involved in test administration, as well as exploring the ethical issues surrounding the process. Readers with varying degrees of knowledge in the field will come away with a clear picture of the wide-ranging effects of psychological testing. The result is a conceptually stratified look at numerous complex measurement theories and psychometric concepts. ... Whereas some textbooks dealing with testing address statistics by giving it a chapter, Kline opts to integrate in-depth statistical analysis wherever appropriate to the topic being presented. This method is very effective, as it gives the reader the complete picture in a single snapshot. ... Psychological Testing by Theresa J. B. Kline is an accessible, easy-to-read book that effectively communicates the current concepts, trends, and controversies in the field of psychological testing. Readers are provided with an in-depth analysis of psychometrics in a format that will keep their attention and that they will be able to relate to the significance of psychological testing across numerous areas such as schools, businesses, clinical settings, military, or government. -- Todd L. Chmielewski * PsycCRITIQUES December 7, 2005 Vol. 50, No. 49, Article 12 * Psychological Testing is an accessible, easy-to-read book that effectively communicates the current concepts, trends, and controversies in the field of psychological testing. Readers are provided with an in-depth analysis of psychometrics in a format that will keep their attention and that they will be able to relate to the significance of psychological testing across numerous areas such as schools, businesses, clinical settings, military, or government. -- PsycCRITIQUES -- PsycCRITIQUES