Summary of Chapter 9

Survey Research: 

Survey research is a research method which uses “standardized questions or interviews to collect data about people and their preferences, thoughts and behavior in a systematic manner”. This method has become very popular for quantitative research in social science.

Typically best for-

• Descriptive, exploratory, or explanatory research
• Studies that have individual people as the units of analysis

Strengths:

• Useful for measuring a wide variety of unobservable data
  • Preferences, traits, attitudes, beliefs, behaviors
• Collecting data from a population that is too large to observe directly
• Convenient and preferred by some respondents
• Interviews may be the only way to reach a certain population
  • Homeless, illegal immigrants
• Large sample surveys can detect small effects and allow comparative analysis of population subgroups

Weaknesses:

• Subject to bias responses
  • Non-response, sampling bias, social desirability bias, and recall bias

Questionnaire Surveys:

• Set of questions (items) intended to capture responses from respondent in a standardized manner
• Unstructured v. structured
  • Unstructured: Respondents provide a response in their own words
  • Structured: Respondents select an answer from a set of choices
• Self-administered mail surveys– Questionnaire is mailed to a large number of people, willing respondents complete the survey and return it via prepaid envelopes
  • Pros: unobtrusive, inexpensive
  • Cons: slow response rates, people tend to ignore survey requests
• Group-administered questionnaires– A sample of respondents are brought together and each respondent is asked to answer the survey while in that room without interacting
  • Pros: high response rate, can ask clarification questions
• Online or web survey– Administered over the internet, respondents may receive an email with a survey link where survey can be completed.
  • Pros: inexpensive, results are instantly recorded, surveys can be easily modified
  • Cons: If website is not password-protected then answers can be compromised, can’t reach people without a computer

Response formats:

• Dichotomous response-
  • Yes/no, true/false, agree/disagree
• Nominal response-
  • Presented with more than two unordered options
    • Example: What is your industry of employment: manufacturing / consumer services / retail / education / healthcare / tourism & hospitality / other.
• Ordinal response –
  • Presented with more than two ordered options
    • Example: what is your highest level of education: high school / college degree / graduate studies.
• Interval response-
  • Presented with a 5-point or 7- point Likert scale, semantic differential scale, or Guttman scale
  • Respondents enter a continuous (ratio-scaled) value with a meaningful zero point
    • Tend to be fill-in-the blanks type

Questions content and wording:

• Is the question clear and understandable?
• Is the question worded in a negative manner?
• Is the question ambiguous?
• Is the question biased?
• Is the question double-barreled (multiple answers)?
• Is the question too general?
• Is the question too detailed?
• Is the question presumptuous?
• Is the question imaginary?
  • Elicit imaginary answers (bad)
• Do respondents have enough information?

Question Sequencing:

• Questions should flow logically

Other important rules:

• Keep the survey as short as possible
• Confidentiality
• Have respondents send you a copy of results
• Thank respondents
• Pretest your questionnaire before administering it in a field setting

Interview Surveys:

Personalized form of data collection conducted by trained interviewers using the same research protocol as questionnaire surveys.

• Face-to-face interview– Interviewer works directly with the respondent to ask questions and record their responses
  • Pro: Some feel more comfortable in a one on one setting
  • Cons: Some may feel uncomfortable
• Focus group– A small group of respondents are interviewed together in a common location
  • Pros: Allow deep examination
  • Cons: Can be dominated by a dominant personality, some may be reluctant to share their opinion in a group setting.
• Telephone interviews– Interviewers conduct interviews over the phone, typically based on a random selection of people
  • Pros: High response rates
  • Cons: Not ideal for rural areas where telephone density is low

Role of interviewer:

• Prepare for the interview
• Locate and enlist the respondents
• Motivate respondents
• Clarify confusion or concerns
• Observe quality of response

Conduction the interview:

• How to probe the respondent
  • The silent probe- pause and wait
  • Overt encouragement- Occasional “uh-huh” or “okay”
  • Ask for elaboration
  • Reflection- Repeat what respondent said and then pause and wait for the respondent to elaborate

Biases in Survey Research:

• Non-response bias- If the majority of respondents fail to respond then a concern is whether respondents are not responding due to a systematic reason
  • To improve rates of responses:
    • Advance notification- Confirming participation in survey
    • Relevance of context- If it’s important to the respondent they are more likely to answer
    • Respondent-friendly– Shorter survey, clear questions, etc.
    • Endorsement-Senior- Executive attests to the importance of the study
    • Follow-up request– May encourage non-respondents to respond
    • Interviewer training- Response rates can be improved with skilled interviewers
    • Incentives- Incentivize respondents to respond
    • Non-monetary incentives
    • Confidentiality and privacy
  • Sampling bias-
    • Interviewing people over the phone only targets those with phones, online surveys target those with computers.
    • Sampling the wrong population
  • Social desirability bias- Respondents tend to avoid negative opinions or embarrassing comments about themselves or their social circles.
  • Recall bias- When responding about event that happened in the distant past, respondent might not be able to accurately respond about their motivation or behaviors.
  • Common method bias- When the phenomenon under investigation is not effectively separated from measurement artifacts.
    • Often referring to the covariance shared between independent and dependent variable