Objectives: After completing this module, you will be able to assess the implications of sampling decisions and procedures in terms of their impact on:

• The confidence that someone can have in applying research-based conclusions in their professional work and personal decision-making (internal validity)
• The degree to which the conclusions can be generalized theoretically or statistically to units (people, busiesses, groups) other than those who participated in the study (external validity)
• The degree to which the conclustions drawn add to our ability to understand, explain and take actions based on the explanatory power of a given body of knowledge.

Required Materials

Topic 1: What do you need to know about sampling to judge the internal and external validity of conclusions drawn?

I base the videos and readings for this module on the assumption that you have little or no prior knowledge about sampling for research purposes. If you DO have experience and knowledge, you will be able to move through the materials quickly. Feel free to do so. I lay out a specific step-by-step approach to covering the materials based on my assumption of little prior knowledge. I provide a selection of videos, my cheat sheets, and readings that cover what you need to learn in the module as quickly as I feel is possible. The learning guide is not specific to any one of the required materials. Use it as a guide to what you need to get from the materials as a whole. If you cannot answer a question in the learning guide, you missed something that I consider important about sampling. You can use this learning guide as a sort of checklist as you work through the materials for this week to make sure you get a good understanding of the basics of sampling. I will ask each of you to answer one or more of the questions in this learning guide during class Learning Guide: Basics of Sampling.

Basics of Sampling This scary cheat sheet prepares you for the rest of the materials (I hope).

Goals of Research Design You have seen this before. I suggest you keep it at hand as you think about the role of sampling in achieving these goals.

Topic 2: What do the terms used in sampling mean?

The three videos linked below are short, but provide what I consider good and understandable explanations of some of the key terms in sampling. They will help you answer the questions on the Learning Guide to Basics of Sampling. I suggest you watch the videos before reading the material in Bernard. However, if you have a good background in basic statistics, you should find Bernard's discussion easy to understand without watching the videos.

Research Design: Defining your Population and Sampling Strategy. This is a "must watch" video because defining the term "population" as it is used in research is critical to success in assessing the contribution of a research report. The video is about 6 minutes long.

Population vs Sample | Sampling | Finite vs Infinite Population. This video (18 minutes long) will explain key statistical terms that tell us how confident we can be in the results of a study, which is directly related to the degree to which we can extend our conclusions to people other than the usually quite small group of people who actually participate in a study.

Calculating the Sample Size with a Finite Population in Excel This is a 6 minute video that covers the thorny problem of "how big should the sample be?" This is critical to social scientific and evaluation research. Your ability to understand the material in this video will rest on your understanding of the concepts in the previous two videos.

What Is Effect Size? This is an excellent very sort video that explains what effect size is and why it is important. You are less likely to have seen effect size discussed than the other components of sampling that we study this week. Please take a look at the video. It's about 3 minutes long.

Topic 3: A Review & Summary

After you watch the three videos, go to page 160-161 in Bernard. Bernard makes five summary statements about sampling. If you can completely understand each statement and can state what all of the terms used mean, you have a good grasp of the basics of sampling. If you struggle with some of the statements, refer to the discussion of the specific topic on pages 145-160 (Chapter 6) in Bernard. This will prepare you for Assignment 1.

The materials listed below will be very helpful in Assignment 1 where you have to explain your conclusions about the quality of the sample and data analysis. Make sure you consult these materials in developing your responses for Assignment 1.

Blair, E. & Blair, J. (2015) Applied Survey Sampling, Sage, Thousand Oaks, CA. This is a lengthy reading, BUT you do not have to read the entire chapter. Read section 1.3 (Sampling Concepts) carefully. I think it is superior to my cheat sheet. I encourage you to work through Case Study 1.1 by yourself to see how well you understand the material covered in section 1.3. Read Section 1.4 (Guidelines for Good Sampling), which is excellent -- also see Exhibit 1.2 on page 23. This table is excellent and I encourage you to make sure you understand it clearly. It will be a critical guide to answering many questions in Assignment 3.

Kreamer, Helena Chmura & Blasey, Christine. (2016) How Many Subjects? Statistical Power Analysis in Research. Sage Publications, Thousand Oaks, California. Chapter 2, pp. 22-29. e-reserve This is an easy-to-read explanation of the concepts of power and precision, the key paramaters you must use to decide "how big" a sample you need. I don't expect you to actually calculate a sample size in assignments or exercises in this class. I do expect you to understand these concepts thoroughly and be able to explain how you would apply these concepts to determine sample size for the research design you create in course assigments.

The Pew Research Center for the People and the Press conducts a great deal of social research in the United States and internationally. They are real "sampling gurus" and they explain everything they do -- no mysteries in their approach. This website addresses many issues in probability sampling, including non-response, the impact of cell phones, sampling for under-represented groups and such. This is a usesful resources for Assignment 1. It has easy-to-read discussions of contemporary sampling issues, such as the impact of cell phones on telephone sampling. http://people-press.org/methodology/sampling/#3

Additional Resources about Sampling

Abrams, L.S. (2010) Sampling "hard to reach" populations in qualitative research. Qualitative Social Work 9(4), 536-550.

Ahern, K. & LeBrocque, R. (2005) Methodological issues in the effects of attrition: simple solutions for social scientists. Field Methods 17(1), 53-69.

Alessi, M.G. & Miller, C.A. (2012) Comparing a convenience sample against a random sample of duck hunters. Human Dimensions of Wildlife 17(2), 155-158.

Ancresen, E.M., Diehr, P.H. & Luke, D.A. (2004) Public health surveillance of low-frequency populations. Annual Review of Public Health, 25, 25-52.

Barratt, M.J., Ferris, J.A. & Lenton, S. (2014) Hidden populations, online purposive sampling, and external validity: Taking off the blindfold. Field Methods 27(1), 3-21.

Beckett, M. (2000) Converging health inequalities in later life -- an artifact of mortality selection? Journal of Health and Social Behavior 41, 106-119.

Benoot, C., Hannes, K. & Bilsen, J. (2016) The use of purposeful sampling in a qualitative evidence synthesis: A worked example on sexual adjustment to a cancer trajectory. BMC Medical Research Methodology 16, 1-12.

Bethlehem, J. (2016) Solving the nonresponse problem with sample matching. Social Science Computer Review 34(1), 59-77.

Bernard, R.H. (2000). Social Research Methods. Sage Publications, Thousand Oaks. Pages 143-172. E-reserve

Brick, J.M. (2011) The future of survey sampling. Public Opinion Quarterly 75(5), 872-888.

Bryant, J. (2014) Using respondent-driven sampling with "hard to reach" marginalised young people: Problems with slow recruitment and small network size. International Journal of Social Research Methodology 17(6), 599-611.

Connelly, N.A., Brown, T.L. & Decker, D.J. (2003) Factors affecting response rates to natural resource-focused mail surveys: Empirical evidence of declining rates over time. Society & Natural Resources 16(6), 541-549.

De Boni, R., Do Nascimento Silva, P.L., Bastos, F.I., Pechansky, F. et al. (2012) Reading the hard-to-reach: A probability sampling method for assessing prevalence of driving under the influence after drinking in alcohol outlets. PLoS ONE 7(4), 1-9.

Draugalis, J.R. & Plaza, C. (2009) Best practices for survey research reports revisited: Implications of target population, probability sampling, and response rate. American Journal of Pharmaceutical Education 73(8), 1-3.

Ellard-Gray, A., Jeffrey, N.K., Choubak, M. & Crann, S.E. (2015) Finding the hidden participant: Solutions for recruiting hidden, hard-to-reach, and vulnerable populations. International Journal of Qualitative Methods 14(5), 1-10.

Emery, S., Lee, J., Curry, S.J., Johnson, T. et al. (2010) Finding needles in a haystack: A methodology for identifying and sampling community-based youth smoking cessation programs. Evaluation Review 34(1), 35-51.

Etter, J & Perneger, T.V. (2000) Snowball sampling by mail: application to a survey of smokers in the general population. International Journal of Epidemiology 29, 43-48.

Evans, K.L., Greenwood, J.J.D. & Gaston, K.J. (2007) The positive correlation between avian species richness and human population density in Britain is not attributable to sampling bias. Global Ecology and Biogeography, 16(3), 300-304.

Gile, K.J. & Handcock, M.S. (2010) Respondent-driven sampling: An assessment of current methodology. Sociological Methodology 40(1):, 285-327.

Glick, P. (2008) Restating the case: The benefits of diverse samples for theory development. Psychological Inquiry 19(2), 78-83.

Griffith, D.A., Morris, E.S. & Thakar, V. (2016) Spatial autocorrelation and qualitative sampling: The case of snowball type sampling designs. Annals of the American Association of Geographers 106(4), 773-787.

Guo, Y., Li, X., Fang, X., Lin, X. et al. (2011) A comparison of four sampling methods among men having sex with men in China : Implications for HIV/STD surveillance and prevention. AIDS Care 23(11), 1400-1409.

Gupta, S., Shuaib, M., Becker, S., Rahman, M.M. & Peters, D.H. (2011) Multiple indicator cluster survey 2003 in Afghanistan: Outdated sampling frame and the effect of sampling weights on estimates of maternal and child health coverage. Journal of Health, Population & Nutrition 29(4), 388-399..

Henry, G.T. (2008) Practical sample design. PP. 33-59 in Practical Sampling, Sage, London. e-reserve

Jacobson, C.A., Brown, T.L. & Scheufele, D. (2007) Gender-biased data in survey research regarding wildlife. Society & Natural Resources 20(4), 373-377.

Kondo, M.C., Bream, K.D.W., Barg, F.K. & Branas, C.C. (2014) A random spatial sampling method in a rural developing nation. BMC Public Health 14(1), 1-15.

Korner, T. & Nimmergut, A. (2004) Using an access panel as a sampling frame for voluntary household surveys. Statistical Journal of the UN Economic Commission for Europe 21(1), 33-52.

Livingston, M., Dietze, P., Ferris, J. et al. (2013) Surveying alcohol and other drug use through telephone sampling: A comparison of landline and mobile phone samples. BMC Medical Research Methodology 13(1), 1-7.

Miller, P.G., Johnston, J. Dunn, M., Fry, C.L. & Degenhardt, L. (2010) Comparing probability and non-probability sampling methods in ecstasy research: Implications for the internet as a research tool. Substance Use & Misuse 45(3), 437-450.

Mookherji, S. & LaFond, A. (2013) Strategies to maximize generalization from multiple case studies: Lessons from the Africa Routine Immunization System Essentials (ARISE) project. Evaluation 19(3), 284-303.

Murray, G.R., Rugeley, C.R., Mitchell, D. & Mondak, J.J. (2013) Convenient yet not a convenience sample: Jury pools as experimental subject pools. Social Science Research 42(1), 246-253.

Nguyen, P. (2004) The census, sampling and African Americans. Western Journal of Black Studies 28(1), 292-302.

Noy, C. (2008) Sampling knowledge: The hermeneutics of snowball sampling in qualitative research. International Journal of Social Research Methodology 11(4), 327-344.

Ozdemir, R.S. , St. Louis, K.O. & Topbas, S. (2011) Public attitudes toward stuttering in Turkey: Probability versus convenience sampling. Journal of Fluency Disorders 36(4), 262-267.

Pike, G.R. (2007) Assessment measures: Using samples in assessment research. Assessment Update 19(2), 12-14.

Polit, D.F. & Beck, C.T. (2010). Generalization in quantitative and qualitative research: Myths and strategies. International Journal of Nursing Studies. 47(11) 1451-1458.

Reddy, S & Davalos, L.M. (2003) Geographical sampling bias and its implications for conservation priorities in Africa. Journal of Biogeography 30, 1719-1727.

Roy, K., Zvonkovic, A., Goldberg, A., Sharp, E. & LaRossa, R. (2015) Sampling richness and qualitative integrity: Challenges for research with families. Journal of Marriage and Family 77(1), 243-260.

Rumpf, H.J., Bischof, G., Hapke, U., Meyer, C. & John, J. (2000) Studies on natural recovery from alcohol dependence: sample selection bias by media solicitation? Addiction 95(5), 765- 775.

Sadler, G.R., Lee, H.C., Lim, R.S. & Fullerton, J. (2010) Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy. Nursing & Health Sciences 12(3), 369-374.

Seawright, J. & Gerring, J. (2008) Case selection techniques in case study research: A menu of qualitative and quantitative options. Political Research Quarterly 61(2), 294-308.

Simmons, A.D. & Bobo, L.D. (2015) Can non-full-probability internet surveys yield useful data? A comparison with full-probability face-to-face surveys in the domain of race and social inequality attitudes. Sociological Methodology 45(1), 357-387.

Slep, A.M.S., Heyman, R.E., Williams, M.C., VanDyke, C.E. & O'Leary, S.G. (2006) Using random telephone sampling to recruit generalizable samples for family violence studies. Journal of Family Psychology 20(4), 680-689.

Sydor, A. (2013) Conducting research into hidden or hard-to-reach populations. Nurse Researcher 20(3), 33-37.

Tuckett, A. (2004) Qualitative research sampling: The very real complexities. Nurse Researcher 12(1), 47-60.

White, V.M., Hill, D.J. & Effendi, Y. (2004) How does active parental consent influence the findings of drug-use surveys in schools? Evaluation Review 28(3), 246-252.

Yin, R.K. (2013) Validity and generalization in future case study evaluations. Evaluation 19(3), 321-332.

Wretman, J. (2010) Reflections on probability vs nonprobability sampling. In M. Carlson, H. Nyquist & M. Villani (eds.), Official Statistics -- Methodology and Applications in Honour of Daniel Thorburn, pp. 29-35. Available at http://officialstatistics.files.wordpress.com/2010/05/bok03.pdf

Zhu, J.J.H., Mo, Q., Wang, F. & Lu, H. (2010) A random digit search (RDS) method for sampling of blogs and other user-generated content. Social Science Computer Review. 29(3), 327-339.