Running Behavioral Studies With Human Participants

Frank Ritter is a professor of Information Sciences and Technology and of Psychology at Penn State. He received his PhD from CMU and BSEE from UIUC. He teaches courses on human-computer interaction and cognitive science. His research interests include models of learning and experiments to measure learning and retention. He co-wrote Running behavioral studies with human participants: A practical guide, which this report builds upon. He also wrote Foundations for designing user centered systems: What psychology do system designers need to know with Baxter and Churchill, Building better interfaces for remote autonomous systems with Oury, and recently as a response to the pandemic, Skills to Obstruct Pandemics, with a team of ten co-authors, which is also available as a free online tutor, http://StopTheSpread.health . Jong W. Kim is a research faculty member in the Department of Psychology at the University of Central Florida. He received his Ph.D. in the Department of Industrial Engineering at the Pennsylvania State University. His academic pursuit is to improve cognitive systems supporting optimized user performance. To that end, he runs experiments with human subjects and models human cognition. His recent research, sponsored by Office of Naval Research, has investigated skill learning and forgetting, and he has developed a theory of skill retention that is being applied to a couple of intelligent tutoring systems. Current research projects focus on the influence of affect on the three stages of learning by an understanding of non-vocal expressions. Particularly, he is interested in helping autistic children learn social communication skills with human-centered computer systems. Jonathan Morgan is a research assistant and lab manager for Penn State's Applied Cognitive Science (ACS) lab, where he manages people running studies about learning, retention, and usability. Morgan has published in Computational and Mathematical Organization Theory, received two paper awards from the Behavior Representation in Modeling and Simulation (BRIMS) conference committee, and co-authored papers published in the proceedings of the annual conference of the Cognitive Science Society, the International Conference on Cognitive Modeling (ICCM), and the annual conference of the Biologically Inspired Cognitive Architectures (BICA) society. He has also contributed to the design, development, and testing of two tutors. His current research includes modeling socio-cognitive processes and examining the acquisition of procedural knowledge in complex tasks. Richard Carlson is Professor of Psychology at Penn State University, where he has been on the faculty for 27 years. He received his B.S.S. from Cornell College and his Ph.D. from the University of Illinois. He conducts experiments examining cognitive control, cognitive skill, and conscious awareness, focusing on control at the time scale of one second or less. Previous research has addressed topics such as causal thinking, the development of troubleshooting skill, task switching, the role of gesture in mental arithmetic, and the structure of conscious intentions. Current research projects focus on the role of affect in working memory and cognitive control, the effect of cognitive workload on metacognition, and on changes in metacognition with increasing skill. He has published in journals such as Journal of Experimental Psychology: Learning, Memory, and Cognition, Memory & Cognition, and Human Factors. His book, Experienced Cognition (1998), which described a theory of consciousness and cognitive skill, won a Choice Outstanding Academic Book award. Professor Carlson currently serves as Associate Head and Director of Undergraduate Studies in Penn State's Department of Psychology. He is the founding coordinator of the Department's online psychology major. In 2009, he received an Outstanding Faculty Adviser award. He serves on the editorial boards of the Journal of Experimental Psychology: Learning, Memory, and Cognition, Behavior Research Methods, and The American Journal of Psychology. He is a fellow of the APA. His website is http://psych.la.psu.edu/directory/faculty-bios/carlson.html

1. Introduction 2. Preparation For Running Experiments 3. Potential Ethical Problems 4. Risks to Validity to Avoid While Running an Experiment 5. Running a Research Session 6. Concluding a Study Appendix 1: A Checklist for Preparing Studies Appendix 2: Example Scripts for Running Studies Appendix 3: Example Consent Form Appendix 4: Example Debriefing Form Appendix 5: Example Institutional Review Board Application Appendix 6: Considerations When Running a Study Online