Evaluation in Today's World

Veronica G. Thomas, PhD, is a Professor in the Department of Human Development and Psychoeducational Studies at Howard University. She also serves as the Evaluation and Continuous Improvement (ECI) Director for the Georgetown-Howard Universities Center for Clinical Translational Sciences (GHUCCTS). Her research interests include culturally responsive evaluation, physical and psychological well-being of Black families, with particular emphasis on women and girls, and the academic and professional development of students of color. Over the years, Dr. Thomas has published work in numerous refereed journals including the American Journal of Evaluation, New Directions for Evaluation, Journal of Community Genetics, Journal of Black Psychology, International Journal for the Advancement of Counselling (British spelling), Family Relations, Adolescence, Educational Leadership, Journal of Adult Development, Review of Research in Education, Journal of Negro Education, Sex Roles, Journal of Social Psychology, Women and Health, and the Journal of the National Medical Association. Her work has been funded by the National Science Foundation, National Institutes of Health, U.S. Department of Education, and the Women's College Coalition. Dr. Thomas major professional associations include the American Evaluation Association (AEA), American Psychological Association (APA), and the American Educational Research Association (AERA). In 2019, she received the AEA Multiethnic Issues in Evaluation Scholarly Leader Award for scholarship that has contributed to social justice-oriented, equity-focused, and/or culturally responsive literature. Patricia B. Campbell, PhD, is the president of Campbell-Kibler Associates, Inc. She has been involved in research and evaluation with a focus on issues of race/ethnicity, gender and disability for many years. Formerly an associate professor of research, measurement and statistics at Georgia State University, Dr. Campbell is an Association for Women in Science (AWIS) Fellow and was awarded the Willystine Goodsell Award by the American Educational Research Association (AERA) and the Betty Vetter Award by the Women in Engineering ProActive Network (WEPAN). Dr. Campbell has authored more than 100 publications including coauthoring Building Evaluation Capacity: Guide I Designing A Cross Project Evaluation and Guide II Collecting and Using Data in Cross-Project Evaluations; A Framework for Evaluating Impacts of Informal Science Education Projects, Good Schools in Poor Neighborhoods: Defying Demographics; Achieving Success"and The AAUW Report: How Schools Shortchange Girls.. Dr. Campbell's websites include www .BeyondRigor.org, which provides easy to use tips to improve the quality of evaluations with diverse populations and www.FairerScience.org which provides researcher and evaluators with tips and tools to more effectively communicate their diversity related research and evaluation findings to the media and the public. In addition Dr. Campbell participates in a variety professional activities include conducting educational evaluation and research training in South Africa and Uganda and serving as an expert witness in the Citadel sex discrimination case.

Preface About the Authors Acknowledgments Chapter 1 Evaluations of Future: Inclusive, Equity-Focused, Useful, and Used Introduction An Overview of the Book An Overview of Evaluation Race, Racism, Social Justice, and a Racialized Perspective Objectivity and Bias Culture, Cultural Competence, and Cultural Responsiveness The Impact of Politics The Current Climate Chapter 2 Evaluation Ethics and Quality Standards Introduction A Brief Historical Perspective on Research Ethics Ethics in Evaluation Cultural Competence as an Ethical Imperative Ethical Sensitivity and Dilemmas Ethical Challenges and Dilemmas Across the Evaluation Process Ethical Principles and Standards for Evaluators and Evaluations Evaluation Corruptibility and Fallacies Evaluator Role, Power, Politics, and Ethics Interplay of Politics and Ethics Chapter 3 Historical Evolution of Program Evaluation Through a Social Justice Lens Introduction History of Evaluation Through a Social Justice Lens Evaluation Prior to Modern Times of the 20th Century Overview of Evaluation in the 20th Century Hidden Figures and Histories in Early-20th-Century Evaluation Evaluation in 1960-2000 Rethinking the Role of Evaluation Influential Women in Evaluation: 1970s-1990s Influential 20th-Century Evaluator: An Activity 21st-Century Evaluation: Expanding the Focus Chapter 4 Evaluation Paradigms, Theories, and Models Introduction The Value of Scientific Paradigms and Theories in Evaluation Social Science Paradigms and Theories Program Theory of Change Evaluation Theories, Models, and Approaches Distinguishing Evaluation Theories, Models, and Approaches Classifying Evaluation Approaches and Theories Evaluation Theories Within a Cultural Context Evaluation Approaches and Theories: A Summary Description of Selected Examples Chapter 5 Social Justice and Evaluation: Theories, Challenges, Frameworks, and Paradigms Introduction Social Justice Marginalized Groups Theories Providing Context for Social Justice Evaluations Race, Racism, and Evaluation Challenges to Social Justice and Evaluation Efforts to Reduce the Impact of Racism on Evaluation Evaluation Models and Social Justice Social Justice-Oriented Evaluation Frameworks and Paradigms Chapter 6 Evaluation Types With a Cultural and Racial Equity Lens Introduction Classifying Evaluations Different Types of Evaluations Developmental Evaluations: Another Alternative to Formative-Summative Putting It All Together Chapter 7 Social Programming, Social Justice, and Evaluation Introduction Understanding Social Problems and Social Programs Through a Social Justice Lens Structural Racism, Social Programming, and Evaluation Integrating Program Planning and Evaluation Planning Social Program Evaluations vs. Social Project Evaluations: Distinctions and Implications Key Program/Project Components Every Evaluator Must Understand Logic Models: Linking Program Components Logic Models and Evaluation Planning Beyond Traditional Linear Logic Models Chapter 8 Responsive Stakeholder Engagement and Democratization of the Evaluation Process Introduction Who Are Stakeholders? Valuing Stakeholders and Diverse Stakeholder Engagement Identifying and Classifying the Right Stakeholders Democratizing the Evaluation Process With Stakeholders Relationships, Values, and Stakeholder Engagement Responsive Stakeholder Engagement Continuum of Stakeholder Engagement: From Nonresponsive to Responsive Barriers to Responsive Stakeholder Engagement Benefits of Responsive Stakeholder Engagement Six-Step Process for Responsive Stakeholder Engagement Communicating With Stakeholders Chapter 9 Planning the Evaluation Introduction Dealing With Power Imbalances During Evaluation Planning Planning for Culturally Responsive and Social Justice-Oriented Evaluations Evaluation Planning Activities Identifying and Involving Stakeholders in Evaluation Planning Analysis of the Context Identifying and Clarifying Project Goals Identifying the Purpose(s) of the Evaluation Defining Success in Evaluation Planning Developing Timelines Identifying Resource Needs Assembling an Evaluation Team Evaluation Planning and Management Visualization Tools Developing a Written Evaluation Plan Overcoming Pitfalls in Evaluation Planning Chapter 10 Evaluation Questions That Matter Introduction Why Evaluation Questions That Matter? Power and Privilege Issues in Formulating Evaluation Questions Characteristics of Good Evaluation Questions: An Overview Sources of Evaluation Questions Prioritizing Evaluation Questions for Diverse Audiences Inclusion/Exclusion Criteria for Prioritizing Evaluation Questions: Two Approaches Steps to Identifying, Formulating, and Prioritizing Questions That Matter Types of Evaluation Questions Summary of Different Types of Evaluation Questions Chapter 11 Selecting Appropriate Evaluation Designs Introduction Rigor Control and Comparison Groups Longitudinal Data Evaluation Designs Rival Hypotheses and Threats to Validity The Best Design for the Question Chapter 12 Defining, Collecting, and Managing Data Introduction Qualitative and Quantitative Data Strengths and Weaknesses of Qualitative and Quantitative Data Ensuring Data Quality Protection of Human Participants Using Existing Measures or Developing New Ones Modes of Data Collection Data Management Chapter 13 The Best Analysis for the Data Introduction Deductive and Inductive Reasoning Quantitative Analysis Qualitative Analysis Chapter 14 Reporting, Disseminating, and Utilizing Evaluation Results Introduction Reporting Results Developing High-Quality, Accessible Reports and Presentations Visually Representing Data Dissemination Using Evaluation Results Chapter 15 Evaluation as a Business Introduction Perspectives on Doing Evaluation as a Business Ethics Business Knowledge and Skills Contracts Making a Business Financially Viable Selecting a Business Entity Developing a Business Plan Chapter 16 Interconnections and Practical Implications Introduction Objectivity and Bias Building Cultural Competence Personalizing a Social Justice Perspective Reflective Practice and Evaluative Thinking Applying to Practice Social Justice Evaluation Politics and Evaluation Voices From the Field: Advice for New Evaluators A Final Thought Supplemental Resources Appendix A Appendix B Glossary References

Evaluation in Today's World: Respecting Diversity, Improving Quality, and Promoting Usability by Veronica Thomas and Patricia Campbell stands as a powerful resource for evaluation instructors, practitioners, students, and scholars who desire to understand why and how to conduct socially just and culturally responsive evaluation, and who are looking for an accessible entry point into the discipline as a whole. -- American Journal of Evaluation "Very rarely are issues related to race, class and power discussed so eloquently and effectively within an evaluation text." -- John Ridings "Evaluation in Today's World is a much needed, useful and relevant text, which bridges the evaluation process and methods with our cultural, historical and political climate. It integrates diverse activities, methods and applications that will help prepare students to become successful future practitioners and evaluators." -- Karen Tinsley "Thomas and Campbell have taken topics that are often only alluded to in program evaluation texts, and brought them to the forefront, highlighting the importance of understanding the diversity of society when developing effective evaluations." -- Marcus-Antonio Galeste "Evaluation in Today's World covers the salient features of research and evaluation in clear language and includes the often overlooked impact of cultural awareness and understanding throughout the research process. In an increasingly multicultural and complex society, the authors have created a resource to help budding researchers evaluate their goals, needs, and biases as they enter this fascinating field." -- Brenda S. Gerhardt "This is the most comprehensive text on evaluation that I have seen. The section on ethics challenges students to reflect on their own beliefs and values, and how those might cause bias in evaluation." -- Sandra Handwerk "This text provides a strong voice to support the inclusion of social justice issues in evaluation planning, implementation, analysis, and reporting." -- Debra J. Dirksen