Mixed Methods for Policy Research and Program Evaluation

Patricia Burch (PhD, Stanford University) is an Associate Professor at the University of Southern California in Los Angeles California. Burch's research focuses on education policy, organizational and institutional theory, qualitative and mixed methods research, and evidence based policy and practice. Burch's recent publications include Mixed Methods Research for Policy and Program Evaluation (SAGE, 2016). Hidden Markets: The New Education Privatization, (Routledge, 2009), Equal Scrutiny: Privatization and Accountability in Digital Education (Harvard Education Press, 2015). Burch's work has appeared in the Journal of Policy Analysis and Management, Educational Evaluation and Policy Analysis, Teachers College Record, Educational Researcher, and other notable journals. Burch regularly collaborates with government agencies and non-governmental organizations on program evaluation and in improving program design and policy effectiveness, with specific attention to equity and quality. Carolyn J. Heinrich (PhD, University of Chicago) is the Sid Richardson professor of public affairs at the Lyndon B. Johnson School of Public Affairs, affiliated professor of economics, and the director of the Center for Health and Social Policy at the University of Texas at Austin. Dr. Heinrich's research focuses on social welfare policy, workforce development, education, econometric methods for social program evaluation, and public management and performance management. She engages in research and evaluation work to improve education and training outcomes in programs for disadvantaged youth; labor market intermediaries and labor market outcomes for low-skilled and disadvantaged workers; healthcare systems transformation and their implications for the disadvantaged; and poverty reduction programs. She regularly consults and collaborates with government agencies and nongovernmental organizations on program evaluations and in improving program and policy design and program effectiveness. In 2004, Heinrich received the David N. Kershaw Award for distinguished contributions to the field of public policy analysis and management by a person under age 40, and in 2010, she was elected to the National Academy of Public Administration. Prior to her appointment at the University of Texas in July 2011, she was the director of the La Follette School of Public Affairs at the University of Wisconsin-Madison.

Chapter 1: Introduction: The Demand for and Value of Fully Integrated Qualitative and Quantitative Research The Role of Mixed Methods Research in Meeting the Demand for Evidenced-Based Policy and Practice What Distinguishes This Book in the Mixed Methods Literature Overview of Remaining Chapters Recommendations for How to Use This Book Chapter 2: Conceptualizing Mixed Methods Research Introduction Rationale and Conditions for Mixed Methods Research Categorizing Mixed Methods Approaches Overview of a Fully Integrated, Mixed Methods Research Effort Discussion Questions Applications to Your Own Work Chapter 3: Designing and Implementing Fully Integrated Mixed Methods Research Introduction Integration in Research Objectives and Design Integration in Instrumentation Design and Data Collection Integration in Data Analysis and Interpretation of Findings Integration in Dissemination of Research Findings Discussion Questions Applications to Your Own Work Chapter 4: Practical Tools for Integrated Mixed Methods Studies of Policy Implementation Introduction Mixed Methods Integration Strategy #1: Bridging Research Instruments for Qualitative and Quantitative Integration Mixed Methods Integration Strategy #2: Mixed Methods Coding and Analysis Memoing Mixed Methods Integration Strategy #3: Adapted Monitoring Tool and Research Reviews Mixed Methods Integration Strategy #4: Research-to-Practice Collaborative Conclusion Discussion Questions Applications to Your Own Work Chapter 5: Fully Integrated Mixed Methods Research in K-12 and Higher Education in the United States Introduction Program Evaluation of Student Achievement Guarantee in Education Racial and Ethnic Inequity in Access and Successful Student Outcomes in Higher Education Challenges and Strategies for Addressing Challenges Discussion Questions Applications to Your Own Work Chapter 6: An Application of Fully Integrated Mixed Methods in Program Development and Evaluation Introduction Program and Evaluation Motivation Pilot Program Development for Child Support Debt Reduction Demonstration Program Rollout and Early Evaluation Focus Impact Evaluation Findings and Their Interpretation and Use Conclusion Discussion Questions Applications to Your Own Work Chapter 7: Mixed Methods Research in an International Context Introduction Child Support Grant Impact Evaluation in South Africa Qualitative-Quantitative Integration Process in the CSG Impact Evaluation Benefits of a Tightly Integrated Qualitative-Quantitative Research Approach Conclusion Discussion Questions Applications to Your Own Work Chapter 8: On the Future of Fully Integrated Mixed Methods Research: Context and Common Lessons From the Cases for the Field How Context Matters Broader Lessons for the Field Concluding Thoughts

Drawing on their extensive experience in large scale project management, Patricia Burch and Carolyn Heinrich clearly articulate an agenda for program and policy evaluation that is designed to ensure actionable evidence of the effectiveness of a program and its implementation is generated for all stakeholders. The examples they provide, while not minimizing the challenges of employing a mixed methods approach, will convince readers of the practicality and value of iteratively integrating both qualitative and quantitative approaches throughout the entire evaluation process, from initial design through data integration and analysis to dissemination of research findings. We can all learn from their experiences, so generously shared. -- Patricia Bazeley In a time of scarce resources for social programs, wise policy requires making tough choices. To do this well, policymakers need reliable information on what works, why some approaches are more successful than others, and what can be done to improve effectiveness. This book is a vital primer on how to do that. It attacks the mistaken view that quantitative and qualitative research are alternatives, rather than essential complements that, when combined in a mix of research methods, can best provide guidance to policymakers and practitioners. -- Judith M. Gueron This book is just what is needed as we move into a new phase of evidence informed policy and practice. It advocates an integrated and coherent approach to policy research and program evaluation while recognizing the realities and complexities of both policy and research. It will be immensely helpful to those engaged in mixed methods research across a variety of policy fields. -- Geoff Whitty The gulf between qualitative and quantitative researchers is an old problem. With this useful and important new book, Burch and Heinrich help to bridge the divide, highlighting the need for a more complementary and integrated approach that can help us learn not just whether programs "work", but how and why they do. -- Douglas N. Harris This rich and sophisticated book provides a wealth of insights and practical guidelines that will enhance the work of both novice and experienced researchers. Burch and Heinrich show the way to making mixed methods research more relevant and more rigorous. -- Thomas Hatch This is a very useful book for researchers and evaluators contemplating or already using mixed methods in policy research and evaluation, as well as for those teaching applied evaluation courses. -- Diane Hirshberg In this book, Burch and Heinrich have successfully explained how a mixed methods approach can be applied to policy analysis and evaluation. I highly recommend it for scholars applying mixed methods to the study of public policy issues. This book should be required reading for all public policy students and researchers. -- Elizabeth A. Corley The book's biggest strength is its emphasis on the practicalities of integrating different methods throughout the life cycle of a research project. For example, all the chapters end with discussion questions, which force the reader to think through and better understand how the material presented might apply to their own work. The book is also illustrated with rich case studies (mostly from the education and social welfare arenas) to aid understanding. -- James Canton, social and behavioural research, Department for Transport