The ABCs of Educational Testing

W. James Popham, professor emeritus at University of California Graduate School of Education and Information Studies, has spent the bulk of his educational career as a teacher. His first teaching assignment, for example, was in a small eastern Oregon high school where he taught English and social studies while serving as yearbook advisor, class sponsor, and unpaid tennis coach. That recompense meshed ideally with the quality of his coaching. Most of Dr. Popham's teaching career took place at UCLA where, for nearly 30 years, he taught courses in instructional methods for prospective teachers as well as courses in evaluation and measurement for graduate students. At UCLA he won several distinguished teaching awards. In January 2000, he was recognized by UCLA Today as one of UCLA's top 20 professors of the 20th century. (He notes that the 20th century was a full-length century, unlike the current abbreviated one.) In 1992, he took early retirement from UCLA upon learning that emeritus professors received free parking. Because at UCLA he was acutely aware of the perishability of professors who failed to publish, he spent his non-teaching hours affixing words to paper. The result: 30 books, 200 journal articles, 50 research reports, and 175 papers presented before research societies. Although not noted in his official vita, while at UCLA he also authored 1,426 grocery lists. His most recent books are Classroom Assessment: What Teachers Need to Know, 6th Ed. (2010) and Assessment for Educational Leaders (2006), Allyn & Bacon; The Truth About Testing (2001), Test Better, Teach Better (2003), Transformative Assessment (2008) and Instruction that Measures Up (2009) ASCD; America's "Failing" Schools (2005) and Mastering Assessment (2006), Routledge; and Unlearned Lessons (2009) Harvard Education Press. He encourages purchase of these books because he regards their semi-annual royalties as psychologically reassuring. In 1978, Dr. Popham was elected to the presidency of the American Educational Research Association (AERA). He was also the founding editor of Educational Evaluation and Policy Analysis, a quarterly journal published by AERA. A Fellow of the Association, he has attended each year's AERA meeting since his first in 1958. He is inordinately compulsive. In 1968, Dr. Popham established IOX Assessment Associates, an R&D group that formerly created statewide student achievement tests for a dozen states. He has personally passed all of those tests, largely because of his unlimited access to the tests' answer keys. In 2002 the National Council on Measurement in Education presented him with its Award for Career Contributions to Educational Measurement. In 2006 he was awarded a Certificate of Recognition by the National Association of Test Directors. In 2009, he was appointed to be a board member of the National Assessment Governing Board. Dr. Popham's complete 44-page, single-spaced vita can be requested. It is really dull reading. School Leadership Briefing Interview

Preface About the Author 1. Why Fuss With Educational Testing? 2. Three Primary Purposes of Educational Testing 3. Behind Standardized Testing's Cloudy Curtain 4. Validity: The Crux of the Caper 5. Reliability: Considering Consistency in Inconsistent Containers 6. Fairness in Testing: A Long Time Coming 7. Reporting Results: Where Rubber and Road Meet 8. Formative Assessment: Instructional Magic? 9. Students' Affect: Underappreciated, Undermeasured 10. What Next? Glossary References Index

"In the wonderful The ABCs of Educational Testing, Jim Popham says, "Too few folks who should know something about educational testing do not," and sadly, he is right. If people in his five target audiences (teachers, educational administrators, educational policymakers, parents of school-age children, and everyday citizens) read this entertaining (yes, "entertaining," a word not often associated with educational testing!) and informative book it would go a long way toward correcting this unfortunate reality. With his unique phrases and word choices (e.g., "I find myself magnetically drawn to a textbook that I wrote myself.") Jim Popham clearly describes and explains the critical aspects of educational testing that need widespread understanding. He stresses the importance of tests being used and interpreted for the purpose for which they are intended, and shows why many of the tests used to evaluate schools, teachers and students are inappropriate. He does this through 10 short chapters that illuminate his 9 assessment-related understandings and his concerns about the lack of assessment literacy. In particular, he provides clarity to often confusing concepts such as validity and reliability and fairness in testing. Popham says that "hands down" the chapter on validity is the most important, but for me the most important chapter is the one on formative assessment, which he identifies as "inexcusably underused." I consider this to be the most important because my major involvement is with K-12 teachers and administrators and until recently most teachers and administrators, through no fault of their own, have not understood the instructional uses of assessment. This is changing because of the influence of people like Jim Popham, Rick Stiggins and Dylan Wiliam. Jim provides a definition of formative assessment that stresses that it is a process (not an event) that provides evidence, which enables teachers to adjust their teaching and students to adjust their learning strategies so that greater learning takes place. Traditionally teachers have put a number on everything students do and every number has been part of student grades regardless of the purpose of the assessment, but the research shows that it is descriptive feedback and identifying student understandings and misconceptions while the learning is going on that makes the greatest contribution to improving student achievement. Jim Popham understands this and makes the case that it is "remarkably effective" and "seriously underutilized." To sum up, this book is a must read for the five audiences that Popham identifies because it is critical that everyone has the 9 assessment related understandings that he so eloquently and clearly explains and describes." -- Ken O'Connor, Consultant/Author/Former High School Teacher and Curriculum Coordinator "With The ABC's of Educational Testing, Jim Popham continues his quest to support and enlighten all who have a stake in improving our educational systems. This short but immensely powerful instrument is in the form of a 10-chapter "chat" about the often-inscrutable business of student testing in our schools. Popham's book explains the sometimes impenetrable language of educational testing in a way that is easy to comprehend and appreciate. Each chapter illuminates a specific issue that that affords the reader a clear translation of the understanding, meaning, and important role of that particular issue in student assessment. The book is precisely directed at five audiences who would have a clear impact on testing in schools: classroom teachers, educational administrators, educational policymakers, parents of school-aged children, and everyday citizens. It has the lofty yet simple aim of assuring that educational testing is understood and used as a potent tool for student learning rather than in other more pejorative and inappropriate ways. Popham's inimitable style is sure to capture the interest and engagement of any reader with an interest in student assessment. This book will be an indispensable guide for individuals and groups who want to help understand, monitor, and provide guidance for educational testing in their schools." -- Frank Philip, Ph.D., Arts Education Assessment Consultant "This is a wonderfully entertaining book about a most serious matter-what teachers and other educators need to know about testing-written by one of our nation's finest educators and most engaging writers. The ABCs of Educational Testing is a highly readable, engaging, and authoritative book that allows teachers to design, use, and interpret educational tests in an appropriate way. It ensures that educational tests will reflect the purpose they were intended to fulfill, making the data obtained useful to educators instead of alien to them. It provides the basics of test design and interpretation in a lucid and engaging manner and is highly recommended for educators, particularly teachers and school board members. It provides them with everything they need to know about test construction and interpretation-and it does so in a lively manner. The ABCs of Educational Testing will help correct the inappropriate and misleading use of educational testing, one of our nation's growth industries." -- David C. Berliner, Regents' Professor Emeritus In this wonderfully entertaining book, Jim Popham provides us with a concise and readable summary of the absolute essentials of educational assessment. With the sure touch of someone who really understands his subject, he makes complex, abstract ideas accessible to everyone. Anyone who is interested in how to improve education-teachers, administrators, policymakers, parents-needs to read this book. -- Dylan Wiliam, Emeritus Professor of Educational Assessment "Jim Popham has done it again! He has found the topic that educators must understand in this quantitative age-testing and accountability-and once more writes with clarity, grace, and a little levity about technical matters. I can't think of a better way to communicate the essential information that on its face may look daunting. Jim's book is approachable and helpful." -- Eva Baker, Founding Director (CRESST) and Distinguished Research Professor