7400.685-080 - Research Methods in FCS
School of Family and Consumer Sciences
Spring Semesters - Tuesday Evenings 5:20-7:55pm in 209 Schrank Hall South
Instructor: David D. Witt, Ph.D.
Survey Research Methods

Survey research is one of the most often used methods of inquiry - perhaps the most often used. The goals of science are well represented by researchers in that the method is able to explore, describe, explain and predict a very wide array of social phenomena. 

Because the researcher has limited "control" over the variables studied (i.e., it is impossible to manipulate behavior using surveys), as is the case with experimental designs, some charge the method as having a basic weakness. I would argue that the facility of manipulation of subjects and their behaviors via experimental designs is often artificial, and thus misleading. The fact that surveys merely measure that which exists, to the extent that such measurement is valid, is the true strength of survey research. In any case, the survey (i.e., interview, questionnaire) is the method of choice for hundreds of researchers active today.

The Goals of Science and Survey Research
Exploration: when a research needs to flesh out details about a particular problem, surveys stand at the ready to achieve the goals of further exploration.  By asking the right questions of the right population, researchers can map out or profile sets of customers, people's buying habits, the effects of disasters, the impact of technology on a host of other interactions.  For this reason, marketing research stands as the first step in the placement of a product in the economy.

Description of the way things are: Sometimes the aim of survey research is to simply and accurately provide characteristics - average height, weight, age, gender, and ethnicity of a sample of respondents, or to describe the characteristics of a group, organization, city, nation, or hemisphere.  The U.S. Census (http://www.census.gov/) is such a survey - a massive undertaking. The decennial census allows for the generation of such comparison tables as this one, showing age and gender distributions for the population in Ohio.  Business interests and governmental agencies are keen to learn about the characteristics of their state and local populations as this information is the basis for growing the economy and awarding funding for various purposes.

Such quantification is imperitive in terms of government planning for future generations and age groups.  For example, wisely spent tax funds would go in to specific groups proportionately according to each subgroup's size and needs. When I was a graduate student at Penn State, working for the rural sociology professor assigned to county agents, one of my routine assignments was to prepare population distributions for my boss to take to specific counties.  These distributions would detail age group size comparisons to help the county agent recommend the disbursment of county monies.  Often the county was preparing to build a new high school at a time when the number of students was dwindling while the number of older citizens was increasing.  The suggestion to build or refurbish an adult community center instead would be met with skepticism without the hard numbers to back up the suggestion. Similarly, seeing to the needs of students or faculty at an institution of higher "learning" might be informed by surveys asking questions about personal areas of concern. 

Thus, descriptive surveys can have implications for social policy, school funding, community welfare, and so on. Survey findings can be used to support existing practices and serve as the foundations of new and innovative programs.  Descriptive surveys can also be used to lay the groundwork for further research. Think of a question about people and their behavior that defies your own understanding, such as why citizens vote against their own interests, or the processes that draw mosh pit divers to the music scene, or the thinking behind massive tattooing (partial nudity). By drawing a sample and thinking of the right questions, a researcher can begin to make connections between these folks, their behaviors, other characteristics about them in relation to the larger society.  This is the heuristic function of survey research, to generate further hypotheses and research questions.
Explanation: With careful operationalization of variables (questions in this case), thoughtful sampling strategies, and the use of deductive logic, survey research can move beyond description to explanation by linking past variables to future conditions.  Here the researcher's goal is to test hypotheses about the relationship of one variable to the other -> implying causality.

Suppose the Literature Review led to the conclusion that all past research points to a model of causality of some kind. Like this:

Looks real complicated doesn't it?
All this model reflects is the past findings on some topic (X1) that suggest effects from all the other numbered X's.
The model is telling the reader that a grouping of variables X's 5-6-7-8 "cause" or "influence" X4
X's 5 thru 8 and X4 have influence on X3.
X's 5-8, X4 and X3 have influence on X2
annnnnnnnd all those X's have influence on X1
There's also the X's 10 thru 16 that are independent of the flow of causality but have their statistical influence as well.

This is where logic comes into model building. Ready?
X's 5 thur 8 are social background variables such as parent's socioeconomic status, cultural heritage, family structure, and the era in which respondents grew up (like the 1930s, 40s, 50s and so on).
  • Social Background was demonstrated to have influence on Age at 1st Marriage (X4)
  • Both Social Background and Age at 1st Marriage influences Educational Attainment (X3)
  • S.B., Age 1st Marriage, and Educational Attainment influences one's own S.E.S. (X2)
  • And all these variables influence Marital Satisfaction and the Probability of Divorce (X1).
Social background is logically Antecedent to Age at 1st Marriage - so it goes on the left side of the model.
Educational Attainment, one's own S.E.S. and Marital Satisfaction are logically Consequential to Age at 1st Marriage.
We solve the problem of causality to some extent because early marriage can't cause poor parents, and so on.  If that is what past researchers have all been saying - and it is, because the model comes from actual literature review -  then the next step is to test the entire model, and survey research will do the trick.

The issue of causality is interesting because causation can never, ever be proven statistically. The idea that "where there's smoke - there's fire" rarely works in social research. In order for a researcher to assert causality between two variables, she would have to actually witness and record the events in a very controlled environment.

Types of Survey Design
There are three basic design types and each type is characterized by the treatment of sample selection
  • Cross-sectional surveys must have fairly rigorous sampling strategies to insure the the sample is a reflection, or is representative, of the population to which findings will be generalized.  The sample must come from the intended population and bear close characteristics to that population in order for the researcher to claim a real cross-section. A sample is drawn and the survey is administered at one point in time.
  • Convenience sample surveys are less rigorous. Often the researcher is less interested in generalizing findings than in exploration or instrument testing.  Convenience samples are just that - convenient, such as solicitation of passers-by. Such serveys are also administered at one point in time.
  • Longitudinal surveys are more complex. Here respondents are followed over a period of time and reinterviewed on a regular basis. Ideally, a sample of children can be drawn and followed through life, periodically being interviewed until they are very old.  Longitudinal samples are very expensive to conduct and are prone to attrition - people die prematurely, move away from the researcher without forwarding addresses, or otherwise vanish. Thus, a successful longitudinal study wiill either have less grand designs and limit periodic observations to two or three times, or will begin with a very large sample in hopes of having a group to survive the years to last interview.
Methods of Data Collection
Personal interviews, telephone interviews - pencil and paper questionnaires, mail questionnaires, electronically delivered questionnaires - each method has advantages and shortcomings.
Personal, or face-to-face interviews are more intimate and more prone to successful response rates. Often the researcher contacts potential respondents ahead of time to insure their cooperation. Personal interviews are expensive and time consuming, however the response rates are higher.
Telephone interviews are not as advisable since the current tolerance for invasion of privacy is quite low. The cost is low, but the refusal rate is going to be high.  Phone interviews do not have the advantage of visual cues, especially useful in probing for responses (i.e., "just point to the letter of the response that comes closest to your feeling."). Also, since surveys are often repetitive in response set or question format, reading each question over and over may lead to respondent fatigue or boredom. Phone interviews share the disadvantage with personal interviews.

Questionnaires, on the other hand, are much less expensive and require no face-to-face interaction. If designed carefully for readability and efficiency, questionnaires can yield quite servicable responses.  Questionnaire delivery can take the form of administering the instrument to a group all at once.  Questionnaires can also be delivered via the postal service, provided the researcher takes steps necessary to remind respondents in the sample to complete their survey.  Still less expensive are electronically delivered questionnaires, via the internet or other electronic means.  Sampling becomes the main problem here unless security measures such as password protection are in place.

Regardless of the method of delivery, survey research yields easily coded responses and sound datasets ready for analysis.
Questionnaire Construction
In his textbook on research methods ("Exploring Research, 5th Edition" Prentice Hall ISBN 0-13-098352-7), Salkind gives solid advice about using questionnaires. By using questionnaires, researchers are making assumptions of which they should be aware.
        The researcher is:
  1. making reasonable demands upon the respondent in terms of time, effort, and expense. If violated, the resulting data may be unreliable, incomplete, or dramatically skewed.
  2. reasonably straightforward and direct by maintaining trust, avoiding trick questions or a hidden agenda. If this assumption is violated and the respondent comes to feel manipulated, useless data may result. Further, respondent made to feel this way will be unwilling to participate in other studies.
  3. sure respondents have the ability to answer all questions. By understanding the abilities, cognitive and otherwise, of the sample will help to insure the quality of the resulting data.
Further, questionnaires should be designed to accomplish the goals of research - for this class, that means that all concepts from research questions and hypotheses are adequately operationalized into satisfactory measures (questions).

Questionnaires should be constructed to maintain the respondent's interest while engaged. By soliciting their interest, respondents will be more likely to return the completed instrument to the researcher.

Data resulting from coded questionnaires may be supplemented with data from other sources, such as records, provided the other source identifies the respondent.  For example, information about the neighborhood in which a respondent lives may be added that respondent's coded questionnaire if the researcher has asked for an address or neighborhood.  Similarly, by knowing a person's age, the researcher can apply birth cohort information to each respondent's answers. Also, questions should be about one thing at a time.  "Are you unwilling to speed on the highway, or are you unafraid of the police?" This really is asking two questions.

Questionnaire construction is an art, thus the look and functionality of the instrument are important.
The instrument should be pleasing to the eye and appear easy to complete. Dense prose or sloppy typesetting will get your instrument into the trash bin quicker than any other mistake. Neat and tidy - clear directions - obvious places to mark responses - and think about including a ready writing instrument, clear instructions on returning the instrument to the researcher, and some kind of incentive for participating.

Good Questionnaires have a rhythm about them, with easy to answer questions first, more difficult ones toward the end. In situations where sensitive questions need to be asked, be sure to warn the respondent before springing the tough questions on them. So use transition sentences and directions ("Now I want to ask you about bicycles.").  If a series of sensitive topics are necessary it would be a good idea to intersperse lighter, innocuous questions between the more difficult ones. In the beginning, items should serve to get the respondent into the momentary habit of answering questions.  As you can imagine, the best way to learn about questionnaire construction is to start constructing, and have your work reviewed by others.

Informing the respondent's a questionnaire is coming and the importance of a Cover Letter
If you are surveying a geographic area, you will be identifying specific neighborhoods and addresses in your sample. Rather than surprising a potential respondent like the team from the Publisher's Clearing House Sweepstakes, it is important to let each household know you are coming.  A post card with simple information will do the trick:

A random survey is being conducted in the city on Family Problem Solving. Your address has been selected from a larger group of 1000 households.  You should receive your questionnaire in a few days. We hope you will take time to complete and return this important survey. Your cooperation is greatly appreciated.

The actual survey should follow on the heels of the postcard exactly as stated.
Not just for postal questionnaires, cover letters lend credence and integrity to your questionnaire. For graduate students, you should use your institution's letterhead for cover letters, complete with faculty sponsor signatures and the signature of your department chair. Here's a good example.

Consent Forms
If you are collecting data, your institution's human subjects review board will require that respondent's sign a form indicating their consent to be interviewed.  This is a form of legal protection for the institution, the researcher, and the respondent.  Here's a good example.

  Return to Syllabus