Argument-Based Validation in Testing and Assessment

Carol A. Chapelle is Distinguished Professor of Liberal Arts and Sciences at Iowa State University, where she teaches courses in second language acquisition and assessment including a course on argument-based validity. She has over 30 years of experience working on research and development in testing and assessment for English as a second language including supervision of Ph.D. dissertation research, participation in test development projects, and advisory service for commercial, non-profit, and government projects in testing. Throughout her research and practice in language testing, she has explored the evolving methodological guidance for conducting validation research. She was led to the nascent concepts of argument-based validation in the early writing of Michael Kane while working on a project to summarize the validation research for the Test of English as a Foreign Language (TOEFL iBTTM). The team cultivated Kane's concepts into a validity argument that succeeded in providing a means for encompassing multiple types of qualitative and quantitative data within a coherent framework showing the connections across test development, test performance and the uses of the scores. The result was a book presenting the validity argument for the TOEFL iBT, Building a Validity Argument for the Test of English as a Foreign Language (Chapelle, Enright, & Jamieson, 2008), which remains a rare example of a validity argument in use for other researchers. She is the recipient of the 2012 Cambridge-International Language Testing Association Lifetime Achievement Award, the 2012 Educational Testing Service TOEFL Program Messick Memorial Lecture Award, and the 2015 Distinguished Scholarship and Service Award from the American Association for Applied Linguistics. She has served as co-editor of the Language Testing (2016-2018), co-editor of the Cambridge Series in Applied Linguistics (2007-present), and founding editor of the Encyclopedia of Applied Linguistics (Wiley-Blackwell, 2012-present).

List of Tables List of Figures Series Editor's Introduction Preface Acknowledgments About the Author Chapter 1: What Is Argument-Based Validity? Introducing Argument-Based Validity The Academic Tradition of Validity Evolving Concepts in Testing Conclusion Chapter 2: Validity Argument Design Expressing Interpretations and Uses: Three Example Tests Using Claims and Inferences to Express Interpretations and Uses Structuring Claims in a Validity Argument: From Grounds to Conclusions Identifying Evidence: Warrants, Assumptions, and Backing Identifying Weaknesses and Limitations in Arguments: Rebuttals The Language of Validity Argument Chapter 3: Uses and Consequences of Test Scores Why Start With Use and Consequences? Inferences About Use and Consequences Iowa Assessments: Warrants for Consequence Implication and Utilization TOEFL iBT: Warrants for Consequence Implication and Utilization MSCEIT: Warrants for Consequence Implication and Utilization Potential Rebuttals Chapter 4: Construct-Related Inferences: Explanation and Extrapolation Constructs in Validity Arguments Explanation Inferences for Traits Extrapolation Inferences for Performance Combining Explanation and Extrapolation for Interactionalist Constructs Threats to Construct-Related Inferences Chapter 5: Consistency-Related Inferences: Generalization and Evaluation Claims About Score Consistency in Validity Arguments Generalization Inferences for Test Scores Evaluation Inferences for Test Tasks Threats to Reliability Chapter 6: Content Domain-Related Inference: Domain Definition Test Development in Validity Arguments The Domain Definition Inference Domain Definition for the Example Tests Chapter 7: Building a Validity Argument The Logic of Validity Arguments The Sociocultural Milieu of Validation Developing a Validity Argument References Index

Chapelle's "Argument-Based Validation in Testing and Assessment" is among the best-written texts on test validity. It is an up-to-date, cogent presentation from a logical - as opposed to strictly psychometric - perspective. -- Shlomo Sawilowsky * Review * This text address complex philosophical discussions related to validity and reliability in an accessible way. It models a way of thinking about assessment decisions that many of our students need to be successful in their future work! -- Amanda C. La Guardia * Review *