Objectives: After completing this module you will be able to:

• Identify the causal claims made in the published literature, including both research reports and recommendations for practitioners;
• Explain the role of internal validity, external validity and explanatory power in evaluating the strength of causal claims

Assigned Materials

Topic 1: Why do we need to worry about things like validity and rigor in research design -- enough to spend a whole 3-credit course talking about three concepts?

After all, scientists don't lie. Or do they? Read this -- you may want to check out some of the links. I did.

But we are all too smart to believe that our own personal experience is all we need to "prove" something is true -- right? Or are we that smart? Do we lie to ourselves? Read this piece in The New Yorker. http://www.newyorker.com/magazine/2017/02/27/why-facts-dont-change-our-minds

Topic 2: Why are scientists so set on DISproving their own ideas?

The Goals of Research Design Read this quickly. We will discuss it in detail in class.Also read quickly pp. 45 to 53 in Bernard.

Topic 3: How is design related to the nature of your research question -- or why can we never PROVE that smoking kills people?

Read Comparative Characteristics of Design Groups Read this with some care. We will call upon all of these ideas during exercises and assignments.

Topic 4: What about "qualitative research" (and why does Mickie dislike that term)?

Please read this document: Cypress, B.S. (2017) Rigor or reliability and validity in qualitative research: Perspectives, strategies, reconceptualization, and recommendations. Dimensions of Critical Care Nursing 36(4), 253-263. DOI: 10.1097/DCC.0000000000000253.

Summing Up

Learning Guide: The Nature & Validity of Claims Based this week on my two cheat sheets

Additional Materials -- Validity & Reliability in Research

I encourage you to share materials in this course. Collaborative learning is important in science and is a key to successful team efforts. Sharing intersting materials about research design with other is one excellent way to do generate collaborative learning. I have provided a form for you to use to share materials linked at the course home page as Sharing Materials. This only takes a few minutes to complete once you have read one a material of interest. This would be an excellent week to share material. You can earn up to 25 points for each material you share (up to 100 points total). You complete the one-page document and upload it to the discussioin baord for Shared Materials on Canvas. Please alert me when you share a material so that I can draw others' attention to the resource.

Antikainen, L. & Ellis, R. (2011). A RE-AIM evaluation of theory-based physical activity interventions. Journal of Sport & Exercise Psychology 33(2), 198-214.

Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D.S. et al. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology 23(5), 443-451.

Bleijenbergh, I.; Korzilius, H. & Verschuren, P. (2011) Methodological criteria for the internal validity and utility of practice oriented research. Quality & Quantity 45(1), 145-156.

Bouffard, M. & Reid, G. (2012) The good, the bad, and the ugly of evidence-based practice. Adapted Physical Activity Quarterly 29(1), 1-24.

Cronje, J. (2013) What is this thing called "design" in design research and instructional design. Educational Media International 50(1), 1-11 doi: 10.1080/09523987.2013.777180

Davis, S.; Gervin, D.; White, G.; Williams, A.; Taylor, A. & McGriff, E. (2013) Briding the gap between research, evaluation and evidence-based practice. Journal of Social Work Education 49(1), 16-29.

Dzewaltowski, D.A., Estabrooks, P. & Glasgow, R.E. (2004). The future of physical activity behavior change research: What is needed to improve translation of research into health promotion practice? Exercise & Sport Sciences Review 32(2), 57-63.

Epstein, I. (2011) Reconciling evidence-based practice, evidence-informed practice, and practice-based research: The role of clinical data-mining. Social Work 56(3), 284-288.

Fereday, J. & Muir-Cochrane, E. (2006) Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods 5(1), 1-11.

Flannelly, K.J., Flannelly, L.T. & Jankowski, K.R.B. (2018) Threats to the internal validity of experimental and quasi-experimental research in healthcare. J of Health Care Chaplaincy. 24(3):107-130. DOI: 10.1080/08854726.2017.1421019.

Gabriele, J.M., Dubbert, P.M. & Reeves, R.R. (2009). Efficacy of behavioural interventions in managing atypical antipsychotic weight gain. Obesity Reviews 10(4), 442-455.

Gilbert, G.H.; Gordan, V.V.; Funkhouser, E.M.; Rindal, D.B. et al. (2013) Caries treatment in a dental practice-based research network: Movement toward stated evidence-based treatment. Community Dentistry & Oral Epidemiology 41(2), 143-153.

Glasgow, R.E., Lichtenstein, E. & Marcus, A.C. (2003). Why don't we see more translation of health promotion research to practice? Rethinking the efficacy to effectiveness transition. American Journal of Public Health 93(8), 1261-1267.

Green, A.I. (2016) Keeping gay and bisexual men safe: The arena of HIV prevention science and praxis. Social Studies of Science 40(2), 210-235.

Gringeri, C., Barusch, A. & Cambron, C. (2013) Examining foundations of qualitative research: A review of social work dissertations, 2008-2010. Journal of Social Work Education 49(4), 760-773.

Hidecker, M.J.C., Jones, R.S. & Imig, D.R. (2009). Using family paradigms to improve evidence-based practice. Journal of Speech-Language Pathology 18(3), 212-221.

Keurhorst, M., Heinen, M., Colom, J. Linderoth, C. et al. (2016) Strategies in primary healthcare to implement early idenetification of risky alcohol consumption: Why do they work or not? A qualitative evaluation of the ODHIN study. BMC Family Practice 17:1-16.

King, D.K., Glasgow, R.E. &Leeman-Castillo, B. (2010). Reaiming RE-AIM: Using the model to plan, implement, and evaluate the effects of environmental change approaches to enhancing population health. American Journal of Public Health 100(11), 2076-2084.

Klesges, L.M., Estabrooks, P.A., Dzewaltowski, D.A., Bull, S.S. & Glasgow, R.E. (2005). Beginning with the application in mind: Designing and planning health behavior change interventions to enhance dissemination. Annals of Behavioral Medicine 29 (Supplement), 66S-75S.

Koro-Ljungberg, M. & Bussing, R. (2013) Methodological modifications in a longitudinal qualitative research design. Field Methods 25(4), 423-440.

Kretlow, A.G. & Blatz, S.L. (2011). The ABCs of evidence-based practice for teachers. Teaching Exceptional Children 43(5), 8-19.

Larzelere, R.E., Kuhn, B.R. & Johnson, B. (2004) The intervention selection bias: an underrecognized confound in intervention research. Psychological Bulletin 130(2), 289-303.

Liden, G. (2013) What about theory? The consequences on a widened perspective of social theory. Quality & Quantity 47(1), 213-225.

Lucas, J.W. (2003) Theory-testing, generalization, and the problem of external validity. Sociological Theory 21(3), 236-253.

McCallion, P. & Ferretti, L.A. (2010). Social work and aging: The challenges for evidence-based practice. Generations 34(1), 66-71.

Misurell, J. & Springer, C. (2013) Developing culturally responsive evidence-based practice: A game-based group therapy program for child sexual abuse (CSA). Journal of Child & Family Studies 22(1), 137-149.

Moffitt, R. (2003) Causal analysis in population research: an economist's perspective. Population & Development ReviewI 29(3), 448-458.

Morrison, K.H., Bradley, R. & Westen, D. (2003) The external validity of controlled clinical trials of psychotherapy for depression and anxiety: a naturalistic study. Psychology & Psychotherapy Research & Practice 76, 109-132.

Moscoso, S.C., Tello, F.P.H. & Lopez, J.L.L. (2006) Using generalizability theory to assess the validity of the evaluation process. Quality & Quantity 40(3), 315-329.

Nadal, K.L., Sriken, J., Davidoff, K.C., Wong, Y. & McLean, K. (2013) Microaggressions within families: Experiences of multiracial people. Family Relations 62(1), 190-201.

Opsal, T., Wolgemuth, J., Cross, J., Kaanta, T. et al. (2016) "There are no known benefits...": Considering the risk-benefit ratio of qualitative research. Qualitative Health Research 26(8), 1137-1150.

Parrish, D.E. & Rubin, A. (2012) Social workers' orientations toward the evidence-based practice process: A comparison with psychologists and licensed marriage and family therapists. Social Work 57(3), 201-210.

Roberts, M. (2010). What is "evidence-based practice" in geography education? International Research in Geographical & Environmental Education. 19(2), 91-95.

Rooney, A.A., Cooper, G.S., Jahnke, G.D., Lam, J. et al. (2016) How credible are the study results? Evaluating and applying internal validity tools to literature-based assessments of environmentla health hazards. Environment Internatonal 92:617-629 DOI: 10.1016/j.envint.2016.01.005..

Roy, A. (2012) Avoiding the involvement overdose: Drugs, race, ethnicity and participatory research practice. Critical Social Policy 32(4), 636-654.

Secret, M., Abell, M.LO. & Trey, B. (2011) The promise and challenge of practice-research collaborations: Guiding principles and strategies for initiating, designing, and implementing program evaluation research. Social Work 56(1), 9-20.

Sidhu, M.S., Gale, N.K., Gill, P.M. and Tom, J.K. (2015) A critique of the design, implementation and delivery of a culturally tailored self-management education intervention: A qualitative evaluation. BMC Health Services Research 15(1), 1-11.

Spoth, R.L., Schainker, L.M. & Hiller-Sturmhoefel, S. (2011) Translating family-focused prevention science into public health impact: Illustrations from partnership-based research. Alcohol Research & Health 34(2), 188-203.

Stadnick, N.; Drahota, A. & Brookman-Frazee, L. (2013) Parent perspectives of an evidence-based intervention for children with autism served in community mental health clinics. Journal of Child & Family Studies 22(3), 414-422.

Stuart, C., Sanders, L., Gurevich, M. & Fulton, R. (2011). Evidence-based practice in group care: The effects of policy, research and organizational practices. Child Welfare 90(1), 93-113.

Taylor, J., Taylor, A., Lewis, S., McNeill, A. et al. (2016) A qualitative evaluation of a novel intervention using insight into tobacco industry tactics to prevent the uptake of smoking in school-aged children. BMC Public Health 16:1-7.

Tracy, S.J. (2010) Qualitative quality: Eight "big tent" criteria for excellent qualitative research. Qualitative Inquiry 16(10), 837-851.

Trainor, A.A. & Graue, E. (2014) Evaluating rigor in qualitative methodology and rsearch dissemination. Remedial and Special Education 35(5), 267-274.

Whittemore, R., Chase, S.K. & Mandle, C.L. (2001) Validity in qualitative research. Qualitative Health Research 11(4), 522-537.

Zarit, S.H., Stephens, M.A.P. & Femia, E. (2003) The validities of research findings: The case of interventions with caregivers. Alzheimer's Care Quarterly 43(3), 216-228.

Zhang, W. & Watanabe-Galloway, S. (2014) Using mixed methods effectively in prevention science: Designs, procedures, and examples. Prevention Science 15(5), 654-662.