Sometimes research is a matter of disciplined listening and watching.
No real instrument will emerge in this kind of fundamental research
efforts. Most of the time - particularly when using survey or
questionnaire methods, or attempting to evaluate a program or do
content analysis on cultural artifacts or information systems,
constructing a research instrument is a major part of the research
process. The term instrument refers to a question, a series of
questions, an entire questionnaire or interview schedule, or rating
system - the set of measuring tools to be used to make observations -
and is the physical paper and ink set of items themselves. Just as an
engineer might have sets of calipers, metal rules, or metered moisture
detection equiment, the researcher in the behavioral sciences requires
specific tools for measurement as well.
Issues of Reliability and Validity are important here and fundamental
to the task of constructing a questionnaire. Servicability, or utility,
and ease of implementation and scoring, and interpretation of responses
are also important. These are matters that are decided about
during instrument construction and continue through pilot studies and
"try-outs" or test runs.
Behavioral science has been around long enough that a number of
resource books have been compiled containing thousands of tests and
measures. Usually, in addition to verbatim reproductions of the
measures themselves, such a resourced will sort measures by concept
measured, and provide citations in the literature that detail the
measure's performance record in terms of reliability and validity.
The use of most pre-existing measures are encouraged by the originator,
even to the extent that the measure is often published with
instructions for administration and coding. See
this Resource Guide for Measurement.
and take time to read and
review this reading - An Indepth look at the Development of a
Complex Concept Measure
Survey Research, using questionnaires or a schedule of
interviewer-guided questions, is the most widely used data-gathering
technique in all of social science because of convenience and economy.
While costs of conducting survey research can skyrocket, it is the most
economical form of social research for collecting data on hundreds of
respondents at a time. Thus, survey research also can be the most
widely abused and mismanaged technique in all of social science.
Several events, inventions, and technological advances have served to
increase the viability of survey research:
- Sampling techniques have become very precise
- Development of measurements such as Scales and Indexes
- Development of statistical software to run on increasingly
powerful desktop computers
- Advances in data storage
- Growth of academic consortia such as the National Opinion
Research Center, and ISPCR
- Prolifereation of funding sources.
In other words, people who are heads of corporations, or who are
running for re-election, or are otherwise in a position to suffer
greatly if the rely on faulty data, are willing to trust survey
research not to lead them down the wrong path, the 2004 Presidential
exit polling debacle not withstanding. Since the unit of analysis
for questionnaires will always be the individual respondent, the survey
method usually has the individual as its focus, although pairs or
groups of individuals that form a logical study unit (a couple, a
family, a company of workers) are often the actual unit under study.
Appropriate to Questionnaires
1. Questions about behavior
When did you last
use our product? How often to you use
marijuana? Have you ever had the opportunity to ...?
Describe your level of religious
belief. Do you think people are basically good or bad?
How well is the President doing on the following
issues, in your opinion?
What is your age at your last
birthday? What is your height in feet and
inches? Do you own a handgun?
In the next 12 months, do you expect any change in
your financial standing?
Do you think the problems in the Middle East will
continue or begin to abate?
If the Democratic party regains a majority in
congress, will this result in good things for Americans?
Are you a Liberal, a Conservative, or a
moderate? In which social class grouping would you
say you belong to?
Would you say you are more informed, or less
informed than the average person?
Match the word on the right with the definition on
Is it your consitutional right to withhold your
legal name for the police when asked?
On which continent is the country of Lithuania?
7. Demographic measures:
What is your gender? What
is your highest educational attainment? What is your age?
Areas in Writing Questions: Try to Avoid these Common Mistakes!
- Use of Jargon or specialized language - What is your
opinion of HR 134.234? Make sure
your questionnaire is simple enough for every respondent to comprehend
- Ambiguity, confusion or vagueness. Clear and concise
is the way to go.
- Don't be timid about asking for
sensitive information, if your study requires it. A question such
as Have you
ever had unprotected sex - that is, sexual contact with someone not
your spouse and without using a condom? will likely terminate
the interview with some respondents unless there is some kind of lead
in or warning ... The
next set of questions deals with some sensitive issues. Please
remember that your responses cannot be associated with you in any way,
and that you may ask to move to the next question if you prefer not to
answer. Then ask the questions and come to the point.
Place the most sensitive questions in the middle of the
questionnaire. If such information is vital to your study, think
about introducing a rhythm of safe to sensitive to safe to sensitive
questions. You want information and must maintain your resolve even
though you may be a little embarrassed for your respondents.
- Emotional language and prestige bias. What is your
opinion of political leaders who lie to their constituents to get young
people to fight and die in needless wars? -
No doubt about the personal sentiments of the author of this
question. Objectivity requires that the researcher tries to see
all sides of every issue. Your concern in doing research is
not to win converts, but to measure respondent beliefs and
attitudes. Such tactics in survey research lead to false
conclusions. The question above is really two questions. How do you believe
the President was truthful with the people regarding the current war?
you believe the current war is justified, even if some American men and
women may have to die?
- Double-Barrelled Questions - Any question that subsumes
one response, such as Do you like Peach
and Cherry flavorings? Yes or
No. Someone might like one and not the other and couldn't answer
- Leading Questions - such as, We all try to be
nice to people, but
sometimes people make us angry. When you shout at people, do you always
start out trying to be nice?
- Questions that are beyond respondent's capability to
in cases where the researcher is testing for respondent capabilities,
asking questions outside the subject's knowledge is pointless and
meaningless, and can fatique the respondent to the point of stopping
the interview. There are exceptions to this rule, if the researcher is
attempting to guage a respondent's knowledge or expertise in some
conceptual area. Here the researcher would want to allow the respondent
to continue to answer until they begin to fail.
- False premises - like this: Clearly the
conservative right is
heading the country down the right path. In your opinion, how might
we, as right-thinking Americans, head off this tide of greed?
than likely, the researcher will anger enough respondents to limit the
validity of the dataset.
- Future intentions - Be careful about asking respondents
about their behavior in the future. Do you think you
will buy a Chevy the next time you need a new car?
Instead, it is legitimate to ask about past behaviors and then to probe
regarding whether or not the respondent would behave the same today,
given some recent development.
- Double negatives - Don't never use none of these type
questions. Isn't that not confusing?
- Overlapping or unbalanced response categories - Many times
questionnaire items carry response categories that range evenly across
all possibilities (i.e., Never Rarely Often Always). The idea is to
avoid something like Never, Hardly Ever, Rarely, Once in a Blue Moon,