• Understand the differences between true and quasi-experimental designs
• Understand the limitations and advantages of quasi-experiments
• Employ a variety of principles and techniques, such as sample matching, multiple control groups, multiple pre-testing, and others to increase the internal and external validity of conclusions drawn from quasi-experiments
• Understand the logic of data analysis (steps in the procedures) that are appropriate for experimental designs

Materials

Topic 1: How do quasi-experiments differ from true experiments?

I suggest you review Chapter 4 in Bernard. Focus again on pp. 99-110 where he discusses many ways of setting up experimental designs. You may also want to revisit my document on experimental designs.

There are a number of You Tube videos about quasi-experiments. I think this one is good and it is brief, which I suspect you will appreciate by now.

How people are assigned to treatments and control. We are covering two general categories of designs with controls and interventions -- true experiments and quasi-experiments. True experiments provide the strongest evidence of a direct cause and effect relationship between one or more treatments (interventions) and outcomes. Use true experiments if you can when you need to know whether an intervention "works". However, everything Bernard says about "experiments" also apply to "quasi-experiments". The difference has to do with random assignment (not random selection, which is NOT a requirement of experiments of any type). In true-experiments, individual research units (birds, bears, atoms, people) are assigned to the treatment and control groups. In quasi-experiments, pre-existing groups of participants are randomly assigned to treatments and control groups. We use them when people "come in groups" and cannot realistically be assigned individually. Often, the group is one that is voluntarily formed by people -- membership in a sport club or in a farmer organization (see the Haiti Experiment) or some other formal group. In many cases, the groups are created by some authority -- grade school students come in groups based on home room, people in prison are assigned to the jail where they are incarcerated, people under medical treatment are in different hospitals. Quasi-experiments are by far the most common design used in evaluating interventions, such as educational programs. You will almost undoubtedly have to use a quasi-experiment at some point in your professional career and some of you will have to use a quasi-experiment for Assignment 3. You do NOT need to look at the slide show linked below before class. We will cover it in class.

Slide Show -- Quasi-Experiments

Who implemented the experiment. This ranges from YOU, the researcher, to some government agency or researcher who was working on the same project independently of you who assigns to treatment/control, some other researcher whose data you can acquire (you don't even know him/her) and who may have had a totally different question from yours (leading to metanalysis in some cases) or any organization that is involved in "putting some people into treatments and others not" even if the organization did not think of this as assignment to treatment and control groups. The basic point is that somebody did in fact assign people to treatment(s) and others not (control) -- that is the essence of the experimental design. Naturally occuring comparison groups (men vs. women, adolescents vs. emerging adults, organic vs. conventional strawberry producers, kids in foster care vs. with birth parents, and on and on) are NOT treatment and control groups. They are comparison groups and are often the focus of cross-sectional or longitudinal designs, but these natural groups are not experimental groups. In experiments -- somebody did something to someone. The Duke site video Two Kinds of Natural Experiments explains these differences well. " You may want to watch Justifying As-If Randomization as well -- I suspect you will need this.

The inability to randomly assign participants to treatment and control groups does pose a threat to the conclusions we draw based on quasi-experimental designs. This cheat sheet of mine on Strengthening Quasi-Experimental Designs provides some ideas about how to reduce the these threats.

Topic 2: Making Sense of Experimental Outcomes

This very brief discussion of the logic of data analysis for experiments should help you with Assignment 3. Focus on the logic of it, not the details. Data Analysis in Experiments

The questions on the linked document provide an opportunity for you to synthesize a lot of what you have learned so far in the semester. I most strongly encourage you to see if you can answer the questions. If not, raise the point in class. If one person has trouble, others usually do as well. Learning Guide: True and Quasi-Experiments

Recommended Readings -- These will be very useful in completing Assignment 3. Consult them in particular as you think about the strengths and weaknesses in your design.

D'Onofrio, B.M., Lahey, B.B., Turkheimer, E. & Lichtenstein, P. (2013) Critical need for family-based, quasi-experimental designs in integrating genetic and social science research. American Journal of Public Health 103(S1), S46-S55. DOI: 10.2105/AJPH.2013.301252.

Fletcher, J.M. & Conley, D. (2013) The challenge of causal inference in gene-environment interaction researchy: Leveraging research designs from the social sciences. American Journal of Public Health 103(S1), S42-S45 (supplement). DOI: 10.2105/AJPH.2013.301290.

Gersten, R., Baker, S. & Lloyd, J.W. (2000) Designing high-quality research in special education: group experimental design Journal of Special Education 34(1), 2-18.

Greeno, C.G. (2001) The skeleton: What underlies treatment research? Family Process 40(3), 361-363.

Greeno, C.G. (2001) The classical experimental design. Family Process 40(4), 495-499.

Lager, A.C.J. & Torssander, J. (2012) Causal effect of education on mortality in a quasi-experiment on 1.2 million Swedes. Proceedings of the National Academy of Sciences of the United States of America. 109(22), 8461-8466. https://doi.org/10.1073/pnas.1105839109

Rumrill, P.D., Jr. & Bellini, J.L. (1999) The logic of experimental design. J. Vocational Rehabilitation. 13, 65-70.

Shadish, W.R., Cook, T.D. & Campbell, D.T. (2002) Quasi-experimental designs that either lack a control group or lack pretest observations on the outcome. Pp. 103-134 in Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Wadsworth, Belmont, CA. e-reserve

Solomon, B.R., Draine, P., DeMoya, J. & Wickrema, R. (1998) D esign-based evaluations: Process studies, experiments and quasi-experiments. Scandanavian Journal of Social Welfare 7, 126-131.

Thayer, B.A. (2012) The role of group research designs to evaluate social work practice. Quasi-Experimental Research Designs. Oxford University Press, pp. 77-106. e-reserve

Travers, J.C., Cook, B.G., Therrien, W.J. & Coyne, M.D. (2016) Replication research and special education. Remedial and Special Education 37(4), 195-204. DOI: 10.1177/0741932516648462

White, H. (2013) An introduction to the Use of randomised control trials to evaluate development interventions. Journal of Development Effectiveness5(1), 30-49. DOI 10.1080/19439342.2013.764652

Additional Materials.

Altmejd, A., Dreber, A., Forsell, E., Huber, J., Imai, T., Johannesson, M., Kirchler, M., Nave, G. & Camerer, C. (2019) Predicting the replicability of social science lab experiments. PLoS ONE 14(12):30225826. DOI: 10.1371/journal.pone.0225826.

Anderson, C.D. & McGregor, R.M. (2017) Persuasion, partisanship and evaluations of national conditions and governmental impact: An experimental study. American Review of Canadian Studies 47(1):53-73. DOI: 10.1080/02722011.2017.1304431.

Andriano, L. & Monden, C.W.S. (2019) The causal effect of maternal education on child mortality: Evidence from a quasi-experiment in Malawi and Uganda. Demography 56(5):1765-1790. DOI:10.1007/s13524-019-00812-3

Becher, E.H., McGuire, J.K., McCann, E.M., Powell, S., Cronin, S.E. & Deenanath, V. (2018) Extension-based divorce education: A quasi-experimental design study of the parents forever program. Journal of Divorce & Remarriage 59(8), 633-652. DOI: 10.1080/10502556.2018.1466256.

Bonetti, D., Eccles, M., Johnston, M., Steen, N., Grimshaw, J., Baker, R., Walker, A. & Pitts, N. (2005) Guiding the design and selection of interventions to influence the implementation of evidence-based practice: An experimental simulation of a complex intervention trial. Social Science & Medicine 60(9), 2135-2147. doi: 10.1016/j.socscimed.2004.08.072

Boss, M. & Kleinert, J. (2015) Explaining social contagioin in sport applying Heider's balance theory: First experimental results. Psychology of Sport & Exercise 16(3):160-169. DOI: 10.1016/j.psychsport.2014.10.006.

Bravo, G. & Squazzoni, F. (2013) Exit, punishment and rewards in commons dilemmas: An experimental study. PLoS ONE 8(8): 1-6. DOI: 10.1371/journal.pone.0069871.

Byiers, B.J., Reichle, J. & Symons, F.J. (2012) Single-subject experimental design for evidence-based practice. American Journal of Speech-Language Pathology 21(4), 397-414. DOI: 10.1044/1058-0360(2012/11-0036)

Carter, H., Drury, J., Amiot, R., Rubin, G.J. & Williams, R. (2014) Effective responder communication improves efficiency and psychological outcomes in a mass decontamination field experiment: Implications for public behavior in the event of a chemical incident. PLoS ONE 9(3), 1-12 DOI: 10.1371/journal.pone.0089846..

Chirumbolo, A., Manneteti, L., Pierro, A., Areni, A. & Kruglanski, A.W. (2005) Motivated closed-mindedness and creativity in small groups. Small Group Research 36(1), 59-82. DOI: 10.1177/1046496404268535

Class, Q., D'Onofrio, B., Singh, A., Ganiban, J. et al. (2012) Current parental depression and offspring perceived self-competence: A quasi-experimental examination. Behavior Genetics 42(5), 787-797. DOI: 10.1007/s10519-012-9550-1.

Crido, H, Herreros, F., Miller, L. & Ubeda, P. (2015) Ethnicity and trust: A multifactorial experiment. Political Studies 63:131-152. DOI: 10.1111/1467-9248.12168

Czyz, S.H., Szmajke, A., Kruger, A. & Kubler, M. (2016) Participation in team sports can eliminate the effect of social loafing. Perceptual & Motor Skills 123(3):754-6=768. DOI: 10.1177/0031512516664938

deLuse, S.R., & Braver, S.L. (2015) A rigorous quasi-experimental design to evaluate the causal effect of a mandatory divorce education program. Family Court Review 53(1), 66-78. DOI: 10.1111/fcre.12131

Deliens, T., Deforche, B., Annemans, L., DeBourdeaudhuij, I. & Clarys, P. (2016) Effectiveness of pricing strategies on french friest and fruit purchases among university students: Results from an on-campus restaurant experiment. PLoS ONE 11(11):1-16. DOI: 10.1371/journal.pone.0165298.

Dixon, H., Sdcully, M., Wakefield, M., Kelly, B., Pettigrew, S., Chapman, K. & Niederdeppe, J. (2018) The impact ofunhealthy food sponsorship vs. pro-health sponsorship models on young adults' food preferences: A randomised controlled trial. BMC Public Health 18(1). DOI: 10.1186/s12889-018-6298-4.

Doughty, h. Wright, J., Verissimo, D., Lee, J.S.H., Oliver, K. & Milner-Gulland, E.J. (2020) Strategic advertising of online news articles as an intervention to influence wildlife product consumers. Conservation Science & Practice 2(10):1-14. DOI: 10.1111/csp2.272.

Eesley, C. & Wu, L. (2020) For startups, adaptability and mentor network diversity can be pivotal: Evidence from a randomized experiment on a MOOC platform. MIS Quarterly 44(2):661-697. DOI: 10.25300/MISQ/2020/15138

Gottlieb, D., Vigoda-Gadot, E. & Haim, A. (2013) Encouraging ecological behaviors among students by using the ecological footprint as an educational tool: A quasi-experimental design in a public high school in the city of Haifa. Environmental Education Research 19(6), 844-863. DOI: 10.1080/13504622.2013.768602

Greenberg, S., Brand, D., Pluta, A., Moore, D. & DeConti, K. (2018) Nudging resisters toward change: Self-persuasion interventions for reducing attitude certainty. American Journal of Health Promotion 32(4)997-1009. DOI: 10.1177/0890117117715295.

Hagglund, P. (2014) Experimental evidence from active placement efforts among unemployed in Sweden. Evaluation Review 38(3), 191-216. DOI: 10.1177/0193841X14551325

Hall, M.G., Grummon, A.H., Lazard, A.J., Maynard, O.M. & Taillie, L.S. (2020) Reactions to graphic and text health warnings for cigarettes, sugar-sweetened beverages, and alcohol: An online randomized experiment of US adults. Preventive Medicine 137:106120. DOI: 10.1016/j.ypmed.2020.106120

Hall, M.G., Marteau, T.M., Sunstein, C.R., Ribisl, K.M., Noar, S.M., Orlan, E.N. & Brewer, N. T. (2018) Public support for pictorial warning on cigarette packs: An experimental study of US smokers. Journal of Behavioral Medicine 41(3):398-405. DOI: 10.1007/s10865-018-9910-2.

Hull, D.M., Powell, M.G., Fagan, M.A.., Hobbs, C.M. & Williams, L.O. (2020) Positive youth development: A longitudinal quasi-experiment in Jamaica. Journal of applied Developmental Psychology 67:101118. DOI: 10.1016/j.appdev.2020.101118.

Hutchings, V.L., Walton Jr., H. & Benjamin, A. (2010) The impact of explicit racial cues on gender differences in support for Confederate symbols and partisanship. Journal of Politics 72(4), 1174-1188. DOI: 10.1017/S0022381610000605.

Jakobsen, M.D., Clausen, T., Andersen, T. & Louis, L. (2020) Can a participatory organizational intervenion improve social capital and organizational readiness to changes? Cluster randomized controlled trial at five Danish hospitals. Journal of Advanced Nursing 76(10):2685-2695.DOI: 10.1111/jan.14441.

Jakobsson, N. & Lindholm, H. (2014) Ethnic preferences in internet dating: A field experiment. Marriage & Family Review 50(4), 307-317. DOI: 10.1080/01494929.2013.879554

Kelly, D., Vidal, L. & Burden, B.C. (2017) A convenient truth: University employees as heterogeneous and inexpensive experimental samples. Social Science Quarterly 98(5):1339-1351. DOI: 10.1111/ssqu.12336.

Kim, C.O. (2018) Can a temporary financial assistance scheme prevent a transition into poverty among individual who experienced a life event crisis? Journal of Economic Policy Reform 21(1):80-97. DOI: /10.1080/17487870.2017.1294487

Kim, M. & Charbonneau, E. (2020) Caught between volunteerism and professionalism: Support by nonprofit leaders for the donative labor hypothesis. Review of Public personnel Administration 40(2):327-349. DOI: 10.1177/0734371X18816139

Kittel, B., Neuhofer, S. & Schwaninger, M. (2020) The impact of need on distributive decisions: Experimental evidence on anchor effects of exogenous thresholds in the laboratory. PLoS ONE 15(4):e-229853. DOI: 10.1371/journal.pone.0228753.

Kohlere, S. (2013) More fair play in an ultimatum game after resettlement in Zimbabwe: A field experiment and a structural model. PLoS ONE 8(5):1-12. DOI: 10.1371/journal.pone.0064791.

Krause, M.S. & Howard K.I. (2003) What random assignment does and does not do. Journal of Clinical Psychology 59(7), 751-766. DOI: 10.1002/jclp.10170.

Kroher, M. & Wolbring, T. (2015) Social control, social learning, and cheating: Evidence from lab and online experiments on dishonesty. Social Science Research 53:311-324. DOI: 10.1016/j.ssresearch.2015.06.003.

Kwak, D.H., Lee, J.S. & Mahan, J.E. (2013) Ad-evoked illustory judgments in fantasy sports participation: Effects of customization level and expert information. Journal of Sport Management 27(5):393-406.DOI: 10.1123/jsm.27.5.393.

Lager, A.C.J. & Torssander, J. (2012) Causal effect of education on mortality in a quasi-experiment on 1.2 million Swedes. Proceedings of the National Academy of Sciences of the United States of America 109(22), 8461-8466.

Lai, E.S.Y., Kwok, C.L., Wong, P.W.C. et al. (2016) The effectiveness and sustainability of a universal school based programme for preventing depression in Chinese adolescents: A follow-up study using quasi-experimental design. PLoS ONE 11(2), 1-20.

Liebe, U., Moumouni, I.M., Bigler, C., Ingabire, C. & Bieri, S. (2020) Using factorial survey experiments to measure attitudes, social norms, and fairness concerns in developing countries. Sociological Methods & Research 49(1):161-192. DOI: 10.1177/0049124117729707

Ma, Y., Nolan, A. & Smith, J.P. (2020) Free GP care and psychological health: Quasi-experimental evidence from Ireland. Journal of Health Economics 72:102351 DOI: 10.1016/j.jhealeco.2020.102351

Nesdale, D., Durkin, K., Maass, A. & Griffiths, J. (2005) Threat, group identification, and children's ethnic prejudice. Social Development 14(2), 189-207.

Nolan, A. (2011) An extension in eligibility for free primary care and avoidable hospitalisations: A natural experiment. Social Science & Medicine 73(7), 978-985.

Norstrom, T. (2005) Saturday opening of alcohol retail shops in Sweden: An experiment in two phases. Addiction 100(6), 767-776.

Page, K.M. & Vella-Brodrick, D.A. (2013) The Working for Wellness program: RCT of an employee well-being intervention. Journal of Happiness Studies 14(3), 1007-1031.

Phan, H.T. & Ngu, B.H. (2017) Undertaking experiments in social sciences: Sqeuential, multiple time series designs for consideration. Educational Psychology Review 29:847-867. DOI 10.1007/s10648-016-9368-0

Reeves, A. & de Vries, R. (2016) Does media coverage influence public attitudes towards welfare recipients? The impact of the 2011 English riots. British Journal of Sociology 67(2), 281-306.

Riemersma, I., van Santvoort, F., Janssens, J.M.A.M. et al. (2015) "You are okay": A support and educational program for children with mild intellectual disability and their parents with a mental illness: Study protocol of a quasi-experimental design. BMC Psychiatry 15, 1-9.

Rinehart Kochel, T. & Weisburd, D. (2019) the impact of hot spots policing on collective efficacy: Findings from a randomized field trial. Justice Quarterly 36(5):900-928. DOI: 10.1080/07418825.2018.1465579.

Rodebaugh, T.L. (2004) I might look OK, but I'm still doubtful, anxious, and avoidant: The mixed effects of enhanced video feedback on social anxiety symptoms. Behaviour Research and Therapy 42(12), 1435-1451.

Rodgers, J., Herrema, R., Freeston, M. & Honey, E. Towards a treatment for intolerance of uncertainty for autistic adults: A single case experimental design study. Journal of Autism and Developmental Disorders. 48(8), 2832-2845 DOI: 10.1007/s10803-018-3550-9.

Schaefer, D.R. (2012) Homophily through nonreciprocity: Results of an experiment. Social Forces 90(4):1271-1295. DOI: 10.1093/sf/sos065.

Scheepers, D. & Ellemers, N. (2005) When the pressure is up: the assessment of social identity threat in low and high status groups. Journal of Experimental Social Psychology 41(2), 192-200.

Schmidt, N.M., Krohn, M.D. & Osypuk T.L. (2018) Modification of housing mobility experimental effects on delinquency and educational problems: Middle Adolescence as a sensitive period. Journal of Youth & Adolescence 47(10):2009-2026. DOI: 10.1007/s10964-018-0859-9.

Sekhon, J.S. & Titiunik, R. (2012) When natural experiments are neither natural nor experimenets. American Political Science Review 106(1):35-57. DOI:10.1017/S0003055411000542

Shirom, A., Vinokur, A. & Price, R. (2008) Self-efficacy as a moderator of the effects of job-search workshops on re-employment: A field experiment. Journal of Applied Social Psychology 38(7), 1778-1804.

Skalicka, V., Belsky, J., Stenseng, F. & Wichstrom, L. (2015) Preschool-age problem behavior and teacher-child conflict in school: Direct and moderation effects by preschool organization. Child Development 86(3), 955-964.

Souchon, N., Livingstone, A.G. & Maio, G.R. (2013) The influence of referees' expertise, gender, motivation, and time constraints on decisional bias against women. Journal of Sport & Exercise Psychology 35(6):585-599. DOI: 10.1123/jsep.35.6.585.

Stage, J & Uwera, C. (2016) Social cohesion in Rwanda: Results from a public good experiment. Development Policy Review 36(5):577-586. DOI: 10.1111/dpr.12291.

Steen, M., Arendsen, J., Cremers, A., DeJong, A., DeJong, J. & DeKoning, N. (2013). Using ineractive model simulations in co-design: An experiment in urban design. CoDesign 9(1):2-16. DOI: 10.1080/15710882.2012.755549.

Terentev, E. & Maloshonok, N. (2019) The impact of response options ordering on respondents' answers to rating questions: Results of two experiments. International Journal of Social Research Methodology. 22(2):179-198l. DOI: 10.1080/13645579.2018.1510660.

Thomaes, S., Bushman, B.J., Orobio de Castro, B. et al. (2009) Reducing narcissistic aggression by buttressing self-esteem: An experimental field study. Psychological Science 20(12), 1536-1542.

Torche, F., Warren, J.R., Halopern-Manners, A. & Valenzuela, E. (2012) Panel conditioning in a longitudinal study of adolescents' substance use: Evidence from an experiment. Social Forces 90(3):891-918. DOI: 10.1093/sf/sor006

Tsvetkova, M. & Macy, M.W. (2014) The social contagion of generosity. PLoS ONE 9(2), 1-9.

Tucker, T., Fry, C.L., Lintzeris, N., Baldwin, S., Ritter, A., Donath, S. & Whelan, G. (2004). Randomized controlled trial of a brief behavioural intervention for reducing hepatitis C virus risk practices among injecting drug users. Addiction 99(9), 1157-1167.

Vanroy, C., Vanlandewijck, Y., Cras, P., Truijen, S., Vissers, D., Swinnen, A., Bosmans, M., Wouters, K. & Feys, H. (2019) Does a cycling program combined with education and followed by coaching promote physical activity in subacute stroke patients? A randomized controlled trial. Disability and Rehabilitation 41(4):413-421. DOI: 10.1080/09638288.2017.1395084

VanZomeren, M., Spears, R., Fischer, A. & Fischer, A.H. (2004) Put your money where your mouth is! Explaining collective action tendencies through group-based anger and group efficacy. Journal of Personality and Social Psychology 87(5), 649-665.

Vescio, T.K., Gervais, S.J. & Snyder, M. (2005) Power and the creation of patronizing environments: the stereotyp-based behaviors of the powerful and their effects on female performance in masculine domains. Journal of Personality and Social Psychology 88(4)-658-672.

Verhofstadt, L., Buysse, A., Ickes, W., DeClercq, A. & Peene, O.J. (2005) Conflict and support interactions in marriage: an analysis of couples' interactive behavior and on-line cognition. Personal Relationships 12(1), 23-43.

Waldzus, S., Mummendey, A. & Wenzel, M. (2005) When "different" means "worse": In-group prototypicality in changing intergroup contexts. Journal of Experimental Social Psychology 41(1), 76-83.