Objectives: After completing this class, you will be able to:

• Understand and evaluate how epistemological stance affects a researcher's approach to data collection
• Be aware of the epistemological stance in this course
• Identify the sources of error in data collection, and especially systematic error that derives from poor instruments
• Analyze the relationships between validity, reliability and discriminatory power in data collection
• Use the concepts that we discuss to develop valid, reliable research instruments

Assigned Readings

Bryman Ch. 6 and 16 in Third Edition.

Maxwell, J.A. (2012) What Is Realism, and Why Should Qualitative Researchers Care? Pp. 3-13 in J.A. Maxwell, A Realist Approach for Qualitative Research. Sage, Thousand Oaks, CA. e-reserve Read quickly to get a feel for what scientific or critical realism is and its implications for research design and methods.

Adcock, R. & Collier, D. (2001) Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review 95(3), 529-546. Read this carefully. This is the approach to validity that we will take in this course. You need to understand this article thoroughly and you use it as a guide to assessing validity in your individual and group projects. You will refer to this article continually throughout the semester. I have prepared a study guide that I believe will help you understand the process of validation. Please use this -- we will discuss these questions in class.

Research Review Articles

Tebes, J.K. (2005). Community science, philosophy of science, and the practice of research. American Journal of Community Psychology 35 (3-4), 213-230.

Wikman, A. (2006). Reliability, validity and true values in surveys. Social Indicators Research 78(1), 85-110. This article deals with the problem of capturing "real" values in what they call surveys. Make sure you understand and explain that the author's concepts apply to ALL research instruments. It deals with the problem that some measurement error arises from variability in the actual thought processes of the respondent. You need to understand this and use it in assessing validity in your individual and group projects.

Class Preparation

Epistemology and Methodology Bring to class -- you do not need to read this ahead of time.

Additional Resources

Amodino, D.M. & Devine, P.G. (2006) Stereotyping and evaluation in implicit race bias: evidence for independent constructs and unique effects on behavior. Journal of Personality and Social Psychology 91(4), 652-661.

Beaulieu, J., Scutchfield, F.D. 7 Kelly, A.V. (2003) Content and criterion validity evaluation of National Public Health Performance Standards measurement instruments. Public Health Reports 118, 508-517.

Carmines, E.G. & Zeller, R.A. (1979) Reliability and validity assessment. Sage, Newbury Park. 71 pp.

Embreton, S. & Gorin, J. (2001) Improving construct validity with cognitive psychology principles. Journal of Educational Measurement 38(4), 343-368.

Friedkin, N.E. (2004) Social cohesion. Annual Review of Sociology 30(1), 409-425.

Jayasekara, H., Rajapaksa, LC. & Aaronson, N.K. (2008) Quality of life of cancer patients in South Asia: Psychometric properties of the Sinhala versino of the EORTC QLQ-C30 in cancer patients with heterogeneous diagnoeses. Quality of Life Research 17(5), 378-791.

Judge, T.A., Erez, A. & Bono, J.E. (2002) Are measures of self-esteem, neuroticism, locus of control and generalized self-efficacy indicators of a common core construct? Journal of Personality and Social Psychology 83(3), 693-710.

Krause, M.S. (2005) How the psychotherapy research community must work toward measurement validity and why. Journal of Clinical Psychology 61(3), 269-283.

Lee, L.W. (2008) Development and validation of a reading-related assessment battery in Malay for the purpose of dyslexia assessment. Annals of Dyslexia 58(1), 37-57.

Maher, B.A. & Gottesman, I.I. (2005) Deconstructing, reconstructing, preserving Paul E. Meehl's legacy of construct validity. Psychological Assessment 17(4), 415-422.

Manty, J.T. (2005) Advances in research definitions of child maltreatment. Child Abuse & Neglect 29(5), 425-259.

Midanik, L.T., Greenfield, T.K. & Bond, J. (2007). Addiction sciences and its psychometrics: The measurement of alcohol-related problems. Addiction 102 (11), 1701-1710.

Shriver, M.D., Anderson, C.M. & Proctor, B. (2001) Evaluating the validity of functional behavior assessment. Social Psychology Review 30(2), 180-192.

Smith, G.T. (2005) On construct validity: issues of method and measurement. Psychological Assessment 17(4), 396-408.

Vessey, J.A., Horowitz, J.A., Carlson, K.L. & Duffy, M. (2008) Psychometric evaluation of the child-adolescent teasing scale. Journal of School Health 78(6), 344-350.

Westen, D. & Rosenthal, R. (2003) Quantifying construct validity: two simple measures. Journal of Personality and Social Psychology 84(3), 608-618.

Yang, Y.C. & Chan, C.Y. (2008). Comprehensive evaluation criteria for English learning websites using expert validity surveys. Computers & Education 51(1), 403-422.