Chapter 5 Summary

Research Design – “a comprehensive plan for data collection in an empirical research project”

Three required processes in Research Design:

(1) the data collection process (Chapter 5)

(2) the instrument development process

(3) the sampling process

Two primary data collection methods:

• Note: Methods incorrectly linked to qualitative/quantitative – important to note DATA not METHOD qualitative or quantitative

Can be combined for Mixed-Mode when facing complex questions

Key attributes of a Research Design:

1. Internal validity (causality) – observed change in dependent variable is caused by a corresponding change in an independent variable; requires:
   1. Covariation of cause and effect
   2. Cause must proceed effect in time
   3. No plausible other explanation for change in dependent variable
2. External validity (generalization) – observed associations may be generalized from sample to broader group or other groups, contexts or time.
   • Note: Higher external validity corresponds to lower internal validity
3. Construct validity – how well a given measurement scale is measuring the hypothesis
4. Statistical conclusion validity – was the statistical procedure valid from which conclusion was derived? Applicable for Positive Method

Improving Internal and External Validity:

Best research design have high levels of both internal and external validity. To ensure high INTERNAL VALIDITY:

1. Manipulation: use of control group to compare effects of adjustment on other sample groups
2. Elimination: remove extraneous variables by holding constant across all sample groups
3. Inclusion: extraneous variables are added to research design and separately estimating effects on all sample groups
   • Note: with larger sample groups can allow for greater generalization
4. Statistical control: “extraneous variables are measured and used as covariates during statistical testing process”
5. Randomization: cancels out effects of extraneous variables through random sampling
   • Random selection – from population of sample groups
   • Random assignment – subjects randomly assigned to sample groups
   • Note: Randomization assures external validity as well (generalization)

 Popular Research Designs:

• Experimental studies: Test cause and effect in relationships
  • True experimental design: subjects randomly assigned to sample groups
  • Quasi–experimental: no random assignment
  • Strong internal validity due to control
  • Limited external generalizability as reality more complex
• Field Surveys: Records snapshots of behavior, opinions, events
  • Cross-sectional field surveys: independent and dependent variables measured at same point of time
  • Longitudinal field surveys: independent and then dependent variables are measured
  • Strong external validity/low internal validity
• Secondary Data Analysis: Using data previously collected by other sources
  • Helpful if data collection is difficult (cost, time, etc)
  • Risk if method of data collection not scientific, not robust enough for research
• Case Research: investigate a real life problem over time
  • Pro – ability to identify a wide variety of factors
  • Con – interpretation dependent on ability of researcher
  • Generalization can be improved by using Multiple Case design (ie larger sample set)
• Focus Group Research: Having subjects participate in moderated discussion on a specific topic at length
  • More for exploratory research due to lack of controls
• Action Research: Phenomenon best understood by introducing “interventions” and then observing resulting effect
  • Pro – Bridges research and practice as potentially solving the identified problem
  • Pro – Beneficial for research difficult to replicate context of phenomenon
  • Con – researcher bias and subjectiveness can influence
  • Con – limited generalization as context very controlled in research
• Ethnography: Phenomenon should be studied within it’s culture
  • Pro – Sensitive to context
  • Pro – Deep understanding and explanation
  • Pro – Minimal participant bias
  • Con – Resource intensive
  • Con – Narrow findings due to cultural context

 Selecting Research Designs:

• Depends on nature of phenomenon being studied
• Can adjust over life of a research project:
  • Early on, when defining research question, may choose case study/focus group to better explore what to study
  • As research evolves, depending if theories exist or not, then may choose a different method to tackle the topic

“Irrespective of the specific research design employed, the goal of the researcher should be to collect as much and as diverse data as possible that can help generate the best possible insights about the phenomenon of interest.” (p.41)