Leading Change Through Evaluation

Kristen Rohanna is a professor in the Educational Leadership Program and Social Research Methodology Division in the School of Education and Information Studies at the University of California, Los Angeles (UCLA). Rohanna's practice and research focus on using evaluative methods to effect social change, with a particular emphasis in the area of education. As such, she works closely with K-12 teachers and administrators to support their use of evaluation and improvement science methods. Before her time at UCLA, Rohanna was a Harvard Strategic Data Fellow and the Manager for Research and Evaluation at the San Jose Unified School District. Her education experience provided the context for her New Direction for Evaluation article: Breaking the Adopt, Attack, Abandon Cycle: A Case for Improvement Science in K-12 Education, about the potential power of continuous improvement methods and the challenges school leaders face when attempting to undertake these methods. Rohanna also has a forthcoming article in the American Journal of Evaluation about building the improvement science capacity of teachers. Rohanna has extensive program evaluation experience, including leading numerous program evaluations for the California Department of Education. Currently, she is leading a statewide evaluation of the 21st Century California School Leadership Academy. The initiative provides professional learning opportunities to K-12 education leaders, including equity-centered leadership and improvement science. Rohanna received her Bachelor of Arts degree in History from the University of Pittsburgh and her Master of Arts degree in Demographic and Social Analysis from the University of California, Irvine. She received her Ph.D. in Social Research Methodology from UCLA.

Series Editor Introduction Acknowledgments Foreword by Louis Gomez Preface About the Author Part 1 * Seeking Change in a Complex World Chapter 1 * What Do We Mean by Formative Evaluation? The Evolution of Formative Evaluation Continuous Improvement and Improvement Science Other Continuous Improvement Approaches Conclusion Chapter 2 * The Truth About Complex Systems Systems and Complexity Science Learning Your Way Through Problems Multiple and Inclusive Perspectives: The Need for Embracing Actors Within the System Building Capacity for Participatory Approaches Conclusion Part 2 * The Case Study Introducing the Case Study The Challenge Case Study Conceptual Framework Chapter 3 * The Beginnings of a Network Building the Network Hub Team Building the Rest of the Team Recruiting the Network Schools Launching the Network Our Own Challenge: Decision to Separate From the Larger Network Conclusion Chapter 4 * The Re-Launch Seeking the Counsel of Others Developing the Network's Theory of Improvement Taking Action Through PDSAs The Network's Outcome, End of Year 1 Conclusion Chapter 5 * A Tale of Two Visions - Building Individual Capacity The Math Instructional Vision (Subject Knowledge) The Vision for Building Improvement Science Capacity (Profound Knowledge) The Network's Initial Improvement Science Capacity The Two Visions Intersect Conclusion Chapter 6 * The Importance of Being Structured - Building Organizational Capacity Organizational Capacity Individual Versus Organizational Capacity Network Structures Conclusion Chapter 7 * Not All Learning Is Created Equal - Fostering Transformational Organizational Learning The Learnings Generated During Year 1 The Importance of Reflective Structures for Single- and Double-Loop Learning Intentionally Designing Activities to Foster Double-Loop Learning Conclusion Chapter 8 * Lessons and Reflections From a Case Study Lessons From Research and Practice: Findings From the Case Study Lessons From Continued Practice: Additional Reflections Bringing It All Together: Learning From Astronauts, Elephants, Einstein, and Teachers Moving Forward in a Complex World Appendix: Case Study Methodology Setting Case Study Design References Index

"Leading Change Through Evaluation [provides] detailed description of how evaluation theories were implemented and tools from different fields were integrated with a specific context. It highlights the interpersonal and intrapersonal skills needed to translate and navigate the application of evaluation approaches in the real world and shows how the American Evaluation Association's (2018) proposed evaluator competencies, such as those in the methodology and context domains, are connected. Rohanna also captures how soft skills, such as reflection and humility, are pivotal to evaluation efforts.... [The] book is a valuable contribution to the evaluation field, especially to graduate students and evaluators interested in learning the concepts of improvement science, evaluation capacity-building, and other approaches as well as details of how to put these evaluation approaches into practice." -- American Journal of Evaluation This is a very well-written, well-researched, practical text for future researchers. -- Bret D. Cormier This book is an intimate, honest and real-world introduction to the use of formative evaluation practices for continuous improvement in a school system. The author's first-hand account provides actionable tools as well as demonstration of their application in practice. The book is both theoretical and applied, which makes it a useful handbook for those who are engaged in continuous improvement of complex systems. -- Dasha Shamrova This text teaches change processes through an overarching case example, which makes the concepts more accessible and understandable as students see how things are actually applied. -- John B. Stark This book provides an insightful journey from improvement science through complex system change. The case study presented allows readers to reflect on their own journeys in evaluation and change within the complex systems that we often dismiss. -- Wesley Wilson This text uses real-life examples to apply theory and emphasize the important role of continuous improvement in public and nonprofit programs. -- Ryan Yeung