7400.685-080 - Research Methods in FCS
School of Family and Consumer Sciences
http://www3.uakron.edu/witt/rmfcs/rmfcs.htm
Spring Semesters - Tuesday Evenings 5:20-7:55pm in 209 Schrank Hall South
Instructor: David D. Witt, Ph.D.
Types of Research in FCS & Sampling Techniques

Family and Consumer Sciences covers a lot of territory. A department such as ours is really like a small university with topics of study from each of our divisions.  As previously mentioned Food science deals with the practical science of food production and preservation, Nutrition and Dietetics takes into account our internal environment (our bodies) in which the cultural as well as the biological processes of nutrition and food production ultimately take place. Clothing and Textiles is the specialization that deals with a sort of portable environment (our garmets) in terms of its effeciency and qualities as well as its production, care, fashionability, commercial appeal, and uniqueness.  Interior Designers detail a less portable but no less personal aspect of the spatial environment, concerning themselves with personal issues such as territoriality, adaptation to the space. Family and Child Developmentalists are concerned with the quality of the environments in which children and families work and grow.  Child Life Specialists are indeed specialized in their concern for the medical care of children and the impact a child's illness has on the family.

Studies of interest in each area are listed here. Some of the listings are published research, some are the major papers of graduate students just like you. Each link below is "clickable" in .pdf format for your reading and edification.
As you may gather, FCS professionals do research in a wide array of settings and for an equally wide array of applications. The Family & Consumer Sciences Research Journal http://fcs.sagepub.com is online with access to the most recent articles published. Contents from the Dec. 2004 issue were:
  •  Jane E. Workman - Features to Anticipate in Future Issues of FCSRJ: Announcing the Debut of the "Professional Issues" Section
  • Sheri Coates Broussard and M. E. Betsy Garrison - The Relationship Between Classroom Motivation and Academic Achievement in Elementary-School-Aged Children
  • Maribeth C. Clarke, Laura C. Koch, and E. Jeffrey Hill - The Work-Family Interface: Differentiating Balance and Fit
  • Jihyun Kim, Janis Stone, Patricia Crews, Mack Shelley, II, and Kathryn L. Hatch - Improving Knit Fabric UPF Using Consumer Laundry Products: A Comparison of Results Using Two Instruments -
  • Padmini Shankar, Jennie E. Long Dilworth, and Diana Cone - Dietary Intake and Health Behavior Among Black and White College Females
  •  Carole J. Makela - Family and Consumer Sciences 2003 Theses and Dissertations: Perspectives on Research
  •  Carole J. Makela - A Listing of Theses and Dissertations Completed in Family and Consumer Sciences: 2003
This kind of variety suggests the importance of types of observation, or the "kind" of research to be done.  Time for a story. On our last trip to New Orleans, we noticed a man on his knees under one of those gas street lights. he was carefully searching the ground under the light for something.  As we approached, I said, What are you looking for? He replied, I lost my keys back down this alley somewhere.  I asked, If you lost them down the alley, why are you looking for them here on the corner? He replied, "The light is so much better here."

Methodology teachers have a saying:  Methodologies used to in research are dictated by the reality of observation -  if the researcher allows them to be.  Perceiving the reality of the situation in which observations are made is the first step in allowing appropriate methodologies to bubble up to the surface of our powers of reason and logic.  Seeing the reality of the situation requires thought, experience,  and, more importantly, patience.  Put another way, researchers never decide on methodologies as a first step in the process.

This principle is true for just about everything in life, although we have to relearn it each time we try something new.
  • Don't force it - let the saw pull the wood through (experience).
  • Visualize the overall problem, then look for solutions (thought).
  • Give it time - it'll heal (patience).
If the researcher takes the time to think a project through, reads the literature of other researchers, and perhaps talks to others interested in the research topic, then methods and ideas for making observations will emerge.  What is of interest here is a listing of some possibilities with examples of each.

There are two broad categories of observations: Quantitative and Qualitative observations. Neither is preferrable to the other, and both have proven invaluable to the research interests in FCS.  Qualitative observations describe in narrative and are usually observations made in depth on few subjects - case studies, ethnographies, historical and library-based research. Qualitative analysis usually, but not always, result in narrative, logical arguments that serve as analysis, much like investigative reporting, or a judge looking over evidence in a trial.  Quantitative observations describe in numerical terms (how much of something, the thing's dimensions or features) - surveys, experiments, the quantification of qualitative data.  Quantitative observations often result in statistical analysis.

Observations can also be classified in terms of how much intrusion there is into the natural world under observation.  Unobtrusive observations are those in which observations are concealed or hidden from the environment under observation - coding behavior from behind a one-way mirror, indirect and participant observations. Obtrusive observations are those in which the observations are made out in the open - surveys, questionnaires, interviews, experiments involving human respondents.

From these two broad catagories, we can fall back on an old, useful friend - the two by two table, to help us see many of the possibilities for making observations - there can be four types:
                    Quant/obtrusive, Quant/unobtrusive, Qual/obtrusive, and Qual/unobtrusive.:


Obtrusive Observations
Unobtrusive Observations
Quantitative
Observations
Survey Research
Interviews, Self- Report Questionnaires
Experiments

Hidden Observations (one-way mirrors)
Secondary analysis of existing data sources
Archival research
Qualitative
Observations
In-depth interviews for case studies
Ethnographies involving the respondent
Direct Observations

Participant Observation
Artifact analysis (photos, tools,
the anthropological record)


Some of the best research will involve more than one type of observation, resulting in what researchers call "triangulation". If questionnaire based observations tell the researcher one thing, and indirect observations support that finding, the researcher has more faith that the results are real and true.

Survey Research is probably the most often used method of observations, and includes self-report questionnaires and interviews. With self-report questionnaires a list of questions, scales, indexes are amassed into an attrative as possible booklet.  Respondents are asked to complete the questions to the best of their ability. Here, the reading ability of the respondents has to be taken into consideration (a sample of uneducated or very young respondents wouldn't be able to complete a questionnaire).  This limitation is overcome to some extent by having questions asked in person in the form of interviews. The advantage of using an interviewer is that the respondent can be directed (sometimes referred to as probing) to answer questions that they might find confusion.  The survey research has the advantage of generating data sets with large sample sizes, and each observation is comparable to every other.  Survey has the disadvantage of having to rely on the truthfulness of respondents.  People can, and often do, lie.  There are, however, steps that should be taken to increase the level of truthfulness.  For example, to combat respondent fatigue and lack of truthfulness, the construction of the questionnaire should reflect a rhythm of light and serious questions, and the researcher should refrain from always asking questions in exactly the same format.  Variation and variety in questionning will keep the respondents' attention.

With survey research, concepts in the hypotheses are operationalized into question sets with each measure represented in question form. Later, during data analysis, the researcher can make statistical comparisons to test operational hypotheses and complete the research process with a discussion of findings. Here are three examples of successful surveys:
Experimental Observation is probably the 2nd most frequently used method for data collection. Experiments usually follow the Pre-Test - Treatment - Post-Test formula. 
  • Subjects are randomly assigned to one or the other of two groups - and Experimental Group and a Control Group. 
  • A Pre-Test observation is made to determine the baseline or starting point for the groups' measurement on the dependent variable.
  • Then the Experimental group receives some kind of treatment (a drug, educational programming, allowed to witness or experience some phenomenon), while the Control receives treatment that is determined to be belign and absent any effects.
  • Then the two groups are measured once again to guage the effect of the experimental treatment.
  • A successful experiment should show
    • little or no change in the dependent variable for the Control group and
    • significant measurable change in the dependent variable for the Experimental Group.
Experimental observation for the behavioral sciences comes right out of the physical sciences' methodological tool kit, and possesses many of their attractive characteristics, such as a high degree of control over outside influences.  Some of the problems associated with experimental observations on human subjects have to do with suitability issues - human beings are highly complex organisms that are capable, as subjects, of seeing through the experimental method and introducing error into the data (i.e., the Hawthorne Effect), and as researchers, of seeing the desired findings whether or not those findings want to see (i.e., the Rosenthal Effect).

The Hawthorne Effect is where the participants or subjects in research projects, instead of acting naturally, try to please the researcher by giving her the results she is looking for. It is named after The GE corporation in Hawthorn, Ohio. It is also known as subject or response bias

Single blind control - is where the researcher or the participant does not know the purpose of the experiment. When the researcher is 'blind' this controls for the Rosenthal Effect or researcher bias; i.e. seeing what you want to see rather than what is there.

In the case of the Hawthorne Effect, workers at a Western Electric factory were aware they were being studied for their productivity.  Workers wanted to please the researchers to receive praise for high productivity. Researchers varied the environmental conditions in the factory by increasing or decreasing lighting and other factors. Workers improved their productivity regardless of the working conditions under manipulation.

Hidden Direct Observations might come in the form of observers sitting behind one-way mirrors, or watching video images of behavior, or even eavesdropping on - the point is that the actors are unaware they are being observed.  Aside from the obvious ethical considerations of  invasion of individual privacy, direct observation has been a staple technique for as long as there has been behavioral sciences.  Data gathering using D.O. requires multiple observers for each discrete behavior observed. Observers must be trained in exactly the same way, and should be given opportunity to practice making observations before officially beginning data collection.  For these reasons, D.O. can be an expensive proposition.  Issues of inter-rater reliability and data coding require extensive planning prior to starting research with this method.

As a method of data collection, Participant Observation requires that the observer becomes part of the cultural or environmental landscape, attempting to achieve near complete immersion in the ecology under study. While objectivity is threatened (there is a good chance the participant observer will enter into relationships with subjects) and that the observer will contaminate observations by their very presence, there is an equal chance that data gathered will have a rich, first-hand feature that cannot be obtained through more invasive methodologies. For these reasons, and like all observational methods, participant observation is best coupled with other modes of data collection.  The research presented here begain as a participant observational study and was continued over several years with library, internet and secondary source analysis. See Pet Burial in the United States, published in C. Bryant (Ed.) The Handbook of Death and Dying, Vol. II, Sage Publications.

Secondary Analysis of Existing Data Sources can come in the form of examination of the historical record, including the perusal of manuscripts and existing documents and records, as in Historical research.  Secondary Analysis also includes reusing existing quantitative data multiple times. Secondary analysis can serve as an additional method of observation which can add integrity to initial quantitative or qualitative findings. Secondary analysis can also be used in original research as well.  In fact, while there is recent historical data being added to the record, we aren't generating any new ancient history, but this doesn't prevent historians from re-examining the record as their thinking evolves.

Some of the most interesting research is done in the form of case studies and ethnographies, and sometimes done by untrained "researchers".  Vance Packard (The Status Seekers) and Studs Terkle (Working) are just two non-researchers whose study of American culture received enormous success in the popular press.  Ethnography is a method of studying and learning about a person or group of people. Typically, ethnography involves the study of a small group of subjects in their own environment. Rather than looking at a small set of variables and a large number of subjects ("the big picture"), the ethnographer attempts to get a detailed understanding of the circumstances of the few subjects being studied. Ethnographic accounts, then, are both descriptive and interpretive; descriptive, because detail is so crucial, and interpretive, because the ethnographer must determine the significance of what she observes without gathering broad, statistical information. Clifford Geertz, whose thoughts about culture are excerpted in the Other Important Definitions of Culture, is famous for coining the term "thick description" in discussing the methodology of the ethnographer. (http://www.wsu.edu:8001/vcwsu/commons/topics/culture/glossary/ethnography.html)
Great resources on ethnography as a method of observation can be found at the American Folklife Center  http://www.loc.gov/folklife/other.html

Case studies are single person or single group analyses. From Prof. Garson's class notes, see http://www2.chass.ncsu.edu/garson/pa765/cases.htm
Types of case studies
. Jensen and Rodgers (2001: 237-239) set forth a typology of case studies, including these types:
  1. Snapshot case studies: Detailed, objective study of one research entity at one point in time. Hypothesis-testing by comparing patterns across sub-entities (ex., comparing departments within the case study agency).
  2. Longitudinal case studies. Quantitative and/or qualitative study of one research entity at multiple time points.
  3. Pre-post case studies. Study of one research entity at two time points separated by a critical event. A critical event is one which on the basis of a theory under study would be expected to impact case observations significantly.
  4. Patchwork case studies. A set of multiple case studies of the same research entity, using snapshot, longitudinal, and/or pre-post designs.This multi-design approach is intended to provide a more holistic view of the dynamics of the research subject.
  5. Comparative case studies. A set of multiple case studies of multiple research entities for the purpose of cross-unit comparison. Both qualitative and quantitative comparisons are generally made.
Sampling
To reiterate - the goals of scientific research - describing, explaining, and predicting phenomena and behavior - are not possible without beginning with a theoretical basis for those goals.  Equally important is the ability to generalize any findings of research back to that theoretical base.  The most critical aspect of any research project in terms of generalizability is the selection of respondents/subjects on which observations are made.  Generally speaking, sampling is the practice of selecting a portion of some identifiable population on which to make observations.  This portion (sample) must be represenative of the larger population of interest if the research is to legitimately draw conclusions from sample responses.

The idea is to be able to say with some degree of certainly that findings hold true for some defined portion of the population. The next best alternative to observing every single person in a population, which would make generalization a "slam-dunk" (and is practically impossibile), is to draw a representative sample from the population on which generalizations are intended

For example, if a researcher was attempting to generalize about the purchasing habits of older teenagers, simply identifying a group of teenagers at the mall will not provide the representativeness required to lend credibility to any results of analysis.  The research must consider as many as possible of the factors by which one sample of teenagers might differ from any other.  This is where that all important theoretical basis informs the process.
  • Are girls and boys going to have the same spending patterns?
  • Will teenagers in the industrial northeast have the same purchasing habits as those in the south or southwest.?
  • Are there critical ages to consider16-17, 18-19?
  • Do some teenagers have more disposable income than others?
  • Are factors such as social class, race or ethnicity important to the study?
By carefully defining as many demographic variables as possible into the population from which the sample will be drawn, the research will be more likely to insure representativeness.

Some things to keep in mind:
  • a population consists of all the members (elements) in a theoretically defined group (i.e., adults in the USA, senior citizens over the age of 65, women between the ages of 18-50).
  • a characteristic of a population is known as a parameter.
  • a characteristic of a sample is known as a statistic.
  • statistics are used to make inferences about the population.
  • the purpose of the study informs the degree of precision necessary in sampling. If inferences are intended for a limited population, then representativeness of the sample need only apply to that limited segment.
  • identifying the population for which inference is intended is completely within the purview of the researcher.
  • keep the sampling unit (unit of analysis) in mind. If the church, school, or family is the unit of analysis, then the sample will have to be drawn from a population of churches, schools, or families.  A common error in behavioral science is to make inferences to a population that is different from the one on which the sample is drawn (i.e., attributing findings to families that are based on a sample of mothers).
  • a helpful tool for sample is the sampling frame, which would be a complete, or near complete, listing of all the elements in a population. Telephone directories come close, except for those people who no phone or have unlisted numbers. Subscription lists to magazines are self-selecting sampling frames. Census listings to the block level are good sampling frames.
  • there is a strong relationship between the size of the sample and the precision of statistical description. The general rule is that the larger the sample the better. Larger samples make statistical significance more likely on small relationships.
  • cost of sampling is a factor.
  • consider the type of sampling needed.
  • Probability samples are better than nonprobability samples so be sure to introduce randomization into the sampling procedure. This just means insuring that every member of a defined population has an equal chance at being included in any sample drawn - an equal chance.

Sampling Exercise

1. Suppose you were issued a grant to study the job satisfaction of women in Northeast Ohio. The research question are:
"Are women satisfied in all respects with their jobs?"
"How are employed women different from women who work at home?"
Explain how you would draw a sample that would be representative of all women in Northeast Ohio.

2. You are interested in high school athletes and body image - specifically, you want to compare the body images of high school athletes to those of non-athletes. Explain your sampling strategy. Why is representativeness less important here than in #1?

Return to Syllabus