7400.685-080 - Research Methods in FCS
School of Family and Consumer Sciences
Spring Semesters - Tuesday Evenings 5:20-7:55pm in 209 Schrank Hall South
Instructor: David D. Witt, Ph.D.
Major Data Collection Methods, Tools and Techniques
In this chapter, we will be looking at the spectrum of data collection methods we've already discussed, paying attention to the tools used to record observations and ways to put those observations down on paper or in electronic files. The whole point of making systematic, objective observations is to ultimately generate findings that are generalizable back to the theories from which hypotheses were derived. To do so requires near rigid adherence to the methods of objective science. Useful measures will do the trick nicely. While the observational methods have already been defined and explained to some extent, the particulars regarding the actual tools and their conversion to data remain. The big idea is to record observations into a consistent, codified data set. So we begin with the measure (which is an operationalized concept, you will remember).
In the case of quantifiable measures with values, each value will correspond to a numerical value.
Age (in years), Educational level (in completed years of formal schooling), Gender (male/female), and the first two items on the Religiosity scale from the previous chapter (Experiential Dimension#1 and Experiential Dimension #2 both measured on a 5 point scale from Strongly disagree to Strongly disagree).
The first step in creating a record of observations, after deciding how to make them, is to create a codebook on the word processor that will define the numerical value of each category of each measure, and provide the column location of each measure on a spreadsheet.
Our codebook would begin like this:
Column 1-3 Respondent Identification Number beginning with 001.
Column 4-5 Age Question 1 measured in whole years
Column 6-7 Educational level measured in completed years of formal schooling
Column 8 Gender measured as 1=male and 2=female)
Column 9 Experiential Dimension#1 measured as 1=strongly agree to 5=strongly disagree
Column 10 Experiential Dimension#2 measured as 1=strongly agree to 5=strongly
And we would continue in this fashion until every variable was located in the dataset.
From here we could actually begin to create our spreadsheet of observations:
Whether making observations with rating scales, questionnaires, interviews, by direct observation, or in experimental situations, when numerical data is required for purposes of statistical summary, the creation of codebooks and electronic data sets is the only way to go. By paying close attention to detail during these first steps in data collection, the data will be preserved for repeated analysis. More than a few of your current instructors have skipped a step or two in this process, opting to commit variable location and detail to memory. In almost every case they found that the data they spent countless hours collecting is now a big block of meaningless numbers. If care is taken to accurately record data this way, statistical analysis will be greatly simplified and greatly enhanced. We will be returning to the idea of data set construction. For now, it is important to be aware that observations are going to have to be recorded somehow. Numerical data sets are pretty handy.
In Survey Research, questions can be asked in terms of the self-report method, or in face-to-face interviews. Data recording will be the same in either case, but there are some details to consider. Self-report questionnaires should be designed to counter respondent fatique, motivate respondents to complete the form, be unambiguous and crystal clear. Because there is no interviewer to coax and encourage, self-report questionnaire design must be more goal oriented than face-to-face interviews - as brief as possible while including all the measures needed for the project.
Booklets (whole pages folded in half and inserted within each other), make for an eye-catching format.
Here's an example in booklet form survey .
Survey methods using interviewers requires an additional step which is interviewer training. Student labor, usually performed for course credit or program requirements, is the source of interviewer talent at universities. Typically, outside the university, people don't work for free, however service oriented communities such as those in Northeast Ohio have a sizeable pool of people from which to draw interviewers. Regardless of the source, interviewers and raters have to be trained before being unleashed on an unsuspecting population. Here are pages from the training manual for a team of interviewers who were assigned neighborhoods in a West Texas city from which they were to draw a sample of respondents. The ostensible topic was Quality of Family Life in Amarillo, but the actual purpose of the study was to detect levels of domestic violence. These were general rules to prepare interviewers for meeting respondents for the first time in the respondent's homes.
Also, in advance of the interviewers going out into the city, a stratified random sample of households to the block were selected using U.S. Census data. Every household on each selected residential block was sent a postcard explaining that community organizations were going to be sending interviewers out to canvass their neighborhoods. Finally, each interviewer was given a letter of introduction that stated the purpose of the study and was signed by the principle investigator (the researcher), the chair of the community organization that was sponsoring the event, and the city manager (liaison from the mayor's office). Phone numbers were listed so that any respondent who questioned the study could call and verify that the project was legitimate.
Here are the training points:
QUALITY OF FAMILY LIFE IN AMARILLO
A Training Manual for Interviewers
1. The Role of the Interviewer
Most beginning interviewers feel strain between the way they behave “naturally” and the way they are to behave in interviews. They feel false, awkward and perhaps even manipulative because they cannot make the structured role of interviewer fit with the other styles of their own behavior. The purpose of this training session is to instill in you a sense of “professionalism” and to give you some idea of what to expect when you knock on the door of a rospon-~ dent’s home and are invited inside. Listed below are some qualities that you should try to exhibit as you fill the role of interviewer:
2. Some Cautions on Conversation
Respondent’s reach for information about you.
Sometimes respondents will have never thought enough about the questions you are asking them to feel they have an opinion. Or they may be so ill at ease with their own opinions that they are reluctant to answer. As a result, repondents may ask you your opinion.
As part of your role as interviewer, you are a Neutral Observer. As such, you should explain, if the occasion arises, that you are not allowed to express your own opinions until the interview is over. Most respondents will accept this rule of the game.
Respondent's reach for relationship with the interviewer.
Sometimes respondents seem to want to talk, for sheer company and warm personal contact. Respondents may be lonely and see the interview as a chance to break their own isolation. Your role as an objective recorder of information may seem cold and heartless at first, but you must remember the goal of the project. If you can manage to keep to the subject, you will have pushed the entire project toward that goal, which is to make life better for some of the very women whom you are interviewing.
Pushing for an answer
Needless to say, we must have answers to all the questions. In the past, respondents have shown very little reluctance to answer; however, some of the questions you will be asking are extremely intimate and extracting answers will sometimes require diplomacy, compassion and tact.
Isolate the respondent as much as possible. In no case should the interview be conducted in the immediate presence of other adult family members. Ask if there is some room in the house where you could talk more privately, If this is not possible, make an appointment to return at a more convenient time, record the time on the interview schedule and promptly return. If clear and present distress is caused by any particular item, or if the respondent simply refuses to answer; mark the “no response” category on that question and continue the interview.
3. Getting the Interview
You have bolstered up your courage. You take a deep breath and knock on the door. It opens and a real, live person is looking at you with suspicion.
Once the respondents fears have been dealt with, you must be certain she fulfills the requirements of the interview.
Cutting the conversation
Traditional theory and training stress the interviewer’s role in opening up communication, but interviewers, in practice often have to cope with tactfully restricting the flow of reminiscence or irrelevance. Digressions can be shortened by interviewer cues of inattention (acted) or even dexterous interruption, but some digression may be essential for the respondent’s continued concentration, interest, and sense of the interviewer’s acceptance. Use your own judgment and try not to let it get out of hand.
The neutral probe
When the respondent’s answer is so vague or general that a Third person, unaware of the respondent’s personality would find little meaning in it; the interviewer is to “probe” (i.e., “Tell me more about that”, or “Could you give me a little more information?”) For this questionnaire, probing of the “Don’t know’ answers will be especially helpful. Probes such as, ‘Just your best estimate’, or “Are your feelings closer to ‘a’ or bt now?” The point is that interviewers should never take a “Don’t know” answer at face value.
The interview can offer something to the respondent.
This is the payoff for her time spent answering all those darned questions. So what are you giving the respondent in exchange for her answers to your questons?
Things that may be scarce in ordinary conversation:
After 5:30 I'll be at the night number. As you go out into the city to begin your interviews, keep your courage up and be sure to call any time, even if it is to get a pep talk. Remember, we are going this project for the good of all the families in the city.
In addition to Survey Research, Experimental Observations result in the same kind of codable data. Here is the the codebook for the family violence study. The main difference between coding surveys one respondent at a time and coding experimental data has to do with repeating the measures before and after a treatment. But the codebooks will be remarkably similar.
Hidden/Direct Observations and Field Notes
Observations can be made via unstructured immersion in an ecology (See a set of field notes here) or in a very structured, experimental laboratory environment. For unstructured interviews, the Anecdotal record consists of brief objective accounts (field notes) of incidences that occur over time. Specimen Recording are sequential detailed narrative accounts of the action while the Running Record is a sort of combination of the two. Unstructured observation provides a lot of color to research and can breathe a feeling of reality into otherwise sterile analyses. Method of recording ranges from pad and pencil to tape or video recording. It should be noted that recording devices may make those observed uncomfortable, so caution to the researcher regarding their open use.
Structured observations will likely be recorded using verbatim recordings, and will employ the use of checklists, rating scales, with Likert- type coding schemes. This type of research would have observers behind the mirror checking off behaviors as they occur out on the child development center recess room.
A checklist, or rating scale, is often used to insure that specific elements are consistently observed. The observer is constantly scanning the environment for all the behavioral elements at once. Therefore, the observer must be intimately familiar with all the checklist or rating scale. For example,
From: www.clt.cornell.edu/campus/teach/ faculty/Materials/Observationchecklist.pdfThe point is to provide a listing of intended observations. Similarly, a rating system in the form of a semantic differential scale allows a series of concepts with polar opposite values and a range between them to be applied to a behavioral setting. Rating scales can be applied to individual measurement, to evlauaton research, teaching effectiveness, socialization of key values and 1000's of other concepts
Teaching Observation ChecklistMechanics __Begins and ends class on time
Scholarship __Includes applications for problem solving and decision-making
__Distinguishes between fact and opinion, data and interpretation
__Emphasizes ways of solving problems rather than solutions
__are important points properly emphasized?
Opening __Focuses student attention (by demonstration, activity, questions, etc,) before
launching into lecture proper
__Presents broader framework within which day’s topic can be placed and related
__States goals or objectives for class sessions.
__Starts slowly, allowing class to “warm up”
Structure/Clarity __Indicates transitions
Pace: __seems about right
__seems too slow
__seems too fast
__all one speed; no variation
Classroom Relationships __Calls students by name
__Gives motivational cues,
__Shifts easily from presentation mode to questioning or discussion mode
__Provides opportunities for and encourages audience participation and questions
__Makes sure that comments or questions have been heard by all
__Checks to see whether answer has been understood
__ are questions from students treated seriously or as interruptions?
__what are they doing?—taking notes; looking over prepared notes?
Conclusion __Draws together contributions of various members of the group
__Summarizes and draws new conceptualizations at end
Involving Students __Prevents or terminates discussion monopolies
__Encourages and guides critical thinking
__Demonstrates a rapport with students
Quality and Content of Discussion
__Pursues student ideas when they are not clearly expressed
__Intervenes when discussion gets off the track
__Summarizes discussion periodically
and so on ...
Provided the proper work on reliability and validity goes into the construction of a semantic differential scale, the product can be an index of behavior.
By adding all the scores up among the adjective pairs, the respondent is awared a score for the overall measure. Thus, using a rating scale as an index with cut-off points is an empirical way to begin to see qualitative differences between workers, teachers, mothers, children.
These are the kind of scales that personnel officers use to make downsizing and other workforce related decisions.
Additional examples of semantic differential scales can be found in the reference section of Bierce Library.
Participant Observation - The field note discussion hold true for participant observation as a method of data gathering. The main difference between P.O. and other kinds of direct observations is simple. The participant observer actually becomes one of the group of people he or she is observing. The idea is that, at least in some sectors of human social life, an outsider will never fully penetrate the true nature of the actors under study. Only as one of them, can the observer be granted entre and be "shown" the true nature of the group. It's like going undercover to find the truth. Obviously there are dangers in such endeavor. Studying dangerous or criminal groups as a participant can easily lead to criminal behavior on the part of the observer, or may lead to harm at the hands of an angry group after exposure as an observer. If the subject matter is far afield the observer may suffer derision or contempt from colleagues and the community.
Given these shortcomings, participant observation can still be a viable tool for inquiry. Often researchers study that which is already part of their experience - evangelical religion, teaching in difficult circumstances, insider looks into the power structure of universities or the government. For FCS, studies of specific family types, families and children with severe disabilities or challenges, shopper behavior, work inside the textile mill - virtually all the disciplines within FCS can and do take advantage of participant observation's power to reveal otherwise hidden facets of life.
Some examples in print: An Annotated Bibliography of Qualitative Methodology http://filebox.vt.edu/users/nespor/qualbib.html
and these articles:
Becker, H. S., Geer, B., Hughes, E. C., & Strauss, A. L. (1961). Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press.
Anderson, E. (1990). Streetwise: Race, Class, and Change in an Urban Community. Chicago: University of Chicago Press.
Davis, F. (1980). The Cabdriver and his Fare: Facets of a Fleeting Relationship. In L. Coser (Ed.), The Pleasures of Sociology (pp. 515). New York: New American Library.
Fine, G. A., & Sandstrom, K. L. (1988). Knowing Children: Participant Observation with Minors. (Vol. 15). Thousand Oaks, CA: Sage.
Humphreys, L. (1970). Tearoom Trade: Impersonal Sex in Public Places. Chicago: Aldine.
Stack, C. B. (1974). All Our Kin: Strategies for Survival in a Black Community.
Whyte, W. F. (1955). Street Corner Society: The Social Structure of an Italian Slum. (2nd ed.). Chicago: University of Chicago Press.
Kelly D, Pearce S, Mulhall A 'being in the same boat': ethnographic insights into an adolescent cancer unit. INT J NURS STUD 41 (8): 847-857 NOV 2004
Anderson PKA bird in the house: An anthropological perspective on companion parrots. SOC ANIM 11 (4): 393-418 2003
McCaughtry N, Rovegno I. JOURNAL OF TEACHING IN PHYSICAL EDUCATION 22 (4): 355-368 JUL 2003
Development of pedagogical content knowledge: Moving from blaming students to predicting skillfulness, recognizing motor development, and understanding emotion
Dameron S. TRAVAIL HUMAIN 65 (4): 339-361 OCT-DEC 2002. The development process of cooperative relationships within design teams
Perez RL Fiesta as tradition, fiesta as change: ritual, alcohol and violence in a Mexican community. ADDICTION 95 (3): 365-373 MAR 2000