Experimental Design in the Behavioral and Social Sciences

Sandra Schneider (Ph.D., University of Wisconsin-Madison) is Professor of Psychology and former Associate Vice President for Research at the University of South Florida. She has also held positions at the National Science Foundation, serving most recently as Division Director within the Social, Behavioral, and Economic Sciences Directorate. Dr. Schneider's research focuses on cognitive and motivational processes in decision making, with an emphasis on reference dependence, goal setting, and risk taking. She has published multiple theoretical and research articles, and is co-editor of the 2003 Cambridge University Press volume, Emerging Perspectives in Decision Research.

VOLUME ONE: THE EXPERIMENTAL APPROACH TO BEHAVIOURAL AND SOCIAL SCIENCES RESEARCH PART ONE: HISTORICAL PERSPECTIVES ON EXPERIMENTS IN THE BEHAVIOURAL AND SOCIAL SCIENCES Excerpts from A System of Logic, Ratiocinative and Inductive, Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation - John Stuart Mill Rules for the Demonstration of Sociological Proof and Conclusion - Emile Durkheim Design for Social Experiments - F.S. Chapin The Role of Theory in Experimental Psychology - E.G. Boring Social Experiments - Henry Riecken and Robert Boruch The Delayed Birth of Social Experiments - Robert Brown PART TWO: THE NOMOTHETIC-IDIOGRAPHIC DEBATE ON THE NATURE OF BEHAVIOURAL AND SOCIAL SCIENCES Concept and Theory Formation in the Social Sciences - Alfred Schutz 'Nomothetic' and 'Idiographic' - James Lamiell Contrasting Windelband's Understanding with Contemporary Usage History in Search of Science - Immanuel Wallerstein Nomothetic Science and Idiographic History in 20th Century Americanist Anthropology - R. Lee Lyman and Michael O'Brien PART THREE: LABORATORY VERSUS NATURAL SETTINGS IN THE BEHAVIORAL AND SOCIAL SCIENCES The Natural Experiment, Ecology and Culture - M. Freilich Reforms as Experiments - D.T. Campbell Situated Experiments in Organizations - J. Greenberg and E.C. Tomlinson Transplanting the Lab to the Field Improving Causal Inference - T. Dunning Strengths and Limitations of Natural Experiments PART FOUR: CONSTRAINTS ON EXPERIMENTAL DESIGN IN THE BEHAVIOURAL AND SOCIAL SCIENCES Cognitive Science - Herbert Simon The Newest Science of the Artificial Social Experiments - H.W. Riecken and R.F. Boruch Economics in the Laboratory - V.L. Smith Experimental Methods in Political Science - Rose McDermott How Hard is Hard Science, How Soft is Soft Science? - L.V. Hedges The Empirical Cumulativeness of Research VOLUME TWO: HYPOTHESIS-TESTING AND INFERENCE IN THE BEHAVIOURAL AND SOCIAL SCIENCES PART ONE: ON THE HISTORY OF HYPOTHESIS-TESTING IN THE BEHAVIOURAL AND SOCIAL SCIENCES Sir Ronald Fisher and the Design of Experiments - F. Yates Statistical Methods and Scientific Induction - Ronald Fisher 'Inductive Behavior' as a Basic Concept of Philosophy of Science - J. Neyman The Fisher, Neyman-Pearson Theories of Testing Hypotheses - E.L. Lehmann One Theory or Two? PART TWO: REASONING PROCESSES IN BEHAVIOURAL AND SOCIAL SCIENCE HYPOTHESIS-TESTING Doing the Impossible - Lola Lopes A Note on Induction and the Experience of Randomness Confirmation, Disconfirmation and Information in Hypothesis-Testing - Joshua Klayman and Young-Won Ha Judging Probable Cause - Hillel Einhorn and Robin Hogarth Statistical Analysis and the Illusion of Objectivity - James Berger and Donald Berry PART THREE: NULL HYPOTHESIS SIGNIFICANCE TESTING IN BEHAVIOURAL AND SOCIAL SCIENCES RESEARCH The Earth Is Round - Jacob Cohen The 'File Drawer Problem' and Tolerance for Null Results - Robert Rosenthal The Insignificance of Null Hypothesis Significance Testing - Jeff Gill Mindless Statistics - Gerd Gigerenzer What to Believe - John Kruschke Bayesian Methods for Data Analysis PART FOUR: POWER AND EFFECT SIZE IN BEHAVIOURAL AND SOCIAL SCIENCES HYPOTHESIS-TESTING 'Statistical,' 'Practical', and 'Clinical' - Bruce Thompson How Many Kinds of Significance Do Counselors Need to Consider? A Power Primer - Jacob Cohen Effect Magnitude - Roger Kirk A Different Focus Effect Sizes - R.L. Rosnow and R. Rosenthal Why, When and How to Use Them VOLUME THREE: CONTROLS AND CONFOUNDS IN BEHAVIOURAL AND SOCIAL SCIENCES EXPERIMENTS PART ONE: SAMPLE SELECTION AND ASSIGNMENT IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Belief in the Law of Small Numbers - Amos Tversky and Daniel Kahneman Models for Sample Selection Bias - Christopher Winship and Robert Mare Assessing the Case for Social Experiments - James Heckman and Jeffrey Smith Misunderstanding Analysis of Covariance - Gregory Miller and Jean Chapman PART TWO: THE CONTROL GROUP IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Experimentation and Social Interventions - Ann Oakley A Forgotten but Important History Natural and Quasi-Experiments in Economics - Bruce Meyer The Delphi List - Arianne Verhagen et al A Criteria List for Quality Assessment of Randomized Clinical Trials for Conducting Systematic Reviews Developed by Delphi Consensus The Politics of Random Assignment - Judith Gueron Implementing Studies and Affecting Policy PART THREE: GENERALIZABILITY AND EXTERNAL VALIDITY IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS The External Validity of Experiments - Glenn Bracht and Gene Glass Designing Research for Application - Bobby Calder, Lynn Phillip and Alice Tybout External Validity Is More Than Skin Deep - Leonard Berkowitz and Edward Donnerstein Some Answers to Criticisms of Laboratory Experiments The Challenge of Representative Design in Psychology and Economics - Robin Hogarth PART FOUR: EXPECTANCY EFFECTS IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS On the Social Psychology of the Psychological Experiment - with Particular Referent to Demand Characteristics and Their Implications - Martin Orne Was There a Hawthorne Effect? - Stephen Jones Teacher Expectations and Self-Fulfilling Prophecies - Lee Jussim and Kent Harber Knowns and Unknowns, Resolved and Unresolved Controversies Neurobiological Mechanisms of the Placebo Effect - Fabrizio Benedetti et al PART FIVE: CONTEXT AND CAUSALITY IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Some Statistical Problems in Research Design - Leslie Kish Checking the Success of Manipulations in Marketing Experiments - Barbara Perdue and John Summers Unfair Comparisons - William Cooper and Alan Richardson Quality Indicators for Group Experimental and Quasi-Experimental Research in Special Education - Russell Gersten et al VOLUME FOUR: DESIGN CHOICES AND FUTURE DRECTIONS IN THE BEHAVIOURAL AND SOCIAL SCIENCES PART ONE: BETWEEN-SUBJECTS VERSUS WITHIN-SUBJECTS DESIGNS IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Experimental Methods and Outcome Evaluation - Michael Mahoney Within-Subjects Designs - Anthony Greenwald To Use or Not to Use? How to Show That 9> 221 - Michael Birnbaum Collect Judgments in a Between-Subjects Design Experimental Methods - Gary Charness, Uri Gneezy and Michael Kuhn Between-Subject and Within-Subject Design PART TWO: MULTIVARIATE DESIGNS IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Definition and Interpretation of Interaction Effects - Ralph Rosnow and Robert Rosenthal The Moderator-Mediator Variable Distinction in Social Psychological Research - Reuben Baron and David Kenny Conceptual, Strategic and Statistical Considerations Establishing a Causal Chain - Steven Spencer, Mark Zanna and Geoffrey Fong Why Experiments Are Often More Effective Than Mediational Analyses in Examining Psychological Processes On the Use of Structural Equation Models in Experimental Designs - R.P. Bagozzi and Youjae Yi PART THREE: MODELING AND SUMMARIZING EXPERIMENTAL DATA IN THE BEHAVIOURAL AND SOCIAL SCIENCES Data Analysis - C.M. Judd, G.H. McClelland and S.E. Culhane Continuing Issues in the Everyday Analysis of Psychological Data Inference and Hierarchical Modeling in the Social Sciences - David Draper Understanding Research Synthesis (Meta-Analysis) - Frederick Mosteller and Graham Colditz Meta-Analysis and the Development of Knowledge - Meow Lan Evelyn Chan and Richard Arvey PART FOUR: ETHICAL CONSIDERATIONS IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Ethics of Intervention in Human Psychological Research - with Special Reference to Stanford Prison Experiment - Philip Zimbardo Science and Ethics in Conducting, Analyzing and Reporting Psychological Research - Robert Rosenthal Reflections and Recommendations on Research Ethics in Developing Countries - S.R. Benatar Neurobiology of Intelligence - Jeremy Gray and Paul Thompson Science and Ethics PART FIVE: PROGRESS AND FUTURE DIRECTIONS IN BEHAVIOURAL AND SOCIAL SCIENCE EXPERIMENTS Innovations in Experimental Design in Attitude Surveys - Paul Sniderman and Douglas Grob Human Research and Data Collection via the Internet - Michael Birnbaum Mixed-Methods Research Designs in Counseling Psychology - William Hanson et al The Growth and Development of Experimental Research in Political Science - James Druckman et al Lab Experiments for the Study of Social-Ecological Systems - Marco Janssen et al