Constructing Public Opinion Surveys

An opinion poll, sometimes simply referred to as a "poll," is a survey of public opinion from a particular sample . Opinion polls are usually designed to represent the opinions of a population by conducting a series of questions and then extrapolating generalities in ratio or within confidence intervals.

Voter Poll

Voter polling questionnaire on display at the Smithsonian Institution

Modes of Data Collection

There are several ways of administering a survey. The choice between administration modes is influenced by: 1) cost, 2) coverage of target population, 3) flexibility of asking questions, 4) respondents' willingness to participate, and 5) response accuracy. Different methods create mode effects that change how respondents answer. The most common modes of administration are:

• Telephone
• Mail
• Online surveys
• Personal in-home surveys
• Personal mall or street intercept surveys
• Hybrids of the above.

Response Formats

Usually, a survey consists of a number of questions the respondent answers in a set format. A distinction is made between open-ended and closed-ended questions. An open-ended question asks the respondent to formulate his or her own answer, while closed-ended questions have the respondent choose an answer from a given number of options. The response options for a closed-ended question should be exhaustive and mutually exclusive. The four types of response scales for closed-ended questions are:

• Dichotomous: The respondent has two options.
• Nominal-polytomous: The respondent has more than two unordered options.
• Ordinal-polytomous: The respondent has more than two ordered options.
• (Bounded) Continuous: The respondent is presented with a continuous scale.

A respondent's answer to an open-ended question can be coded into a response scale afterwards or analyzed using more qualitative methods.

Survey Methodology

The most important aspects of a survey include:

• Identifying and selecting potential sample members.
• Contacting individuals and collecting data from those who are hard to reach (or reluctant to respond).
• Evaluating and testing questions.
• Selecting the mode for posing questions and collecting responses.
• Training and supervising interviewers.
• Checking data files for accuracy and internal consistency.
• Adjusting survey estimates to correct for identified errors.

Advantages

• They are relatively easy to administer.
• Can be developed in less time compared with other data-collection methods.
• Can be cost-effective.
• Few "experts" are required to develop a survey, which may well increase the reliability of the survey data.
• If conducted remotely, can reduce or obviate geographical dependence.
• Useful in describing the characteristics of a large population assuming the sampling is valid.
• Can be administered remotely via the Web, mail, e-mail, telephone, etc.
• Efficient at collecting information from a large number of respondents.
• Statistical techniques can be applied to the survey data to determine validity, reliability, and statistical significance, even when analyzing multiple variables.
• Many questions can be asked about a given topic, giving considerable flexibility to the analysis.
• A wide range of information can be collected (e.g., attitudes, values, beliefs, and behavior).
• Because they are standardized, they are relatively free from several types of errors.

Disadvantages

The reliability of survey data may depend on the following:

• Respondents' motivation, honesty, memory, and ability to respond:
• Structured surveys, particularly those with closed-ended questions, may have low validity when researching effective variables.
• Self-selection bias: Although the individuals chosen to participate in surveys are often randomly sampled, errors due to non-responses may exist. Some prospective respondents may simply be less likely to respond to polls generally, not interested in the subject, or may be unreachable for many reasons. For example, polls or surveys that are conducted by calling a random sample of publicly available telephone numbers will not include the responses of people with unlisted telephone numbers, cell phone numbers, who are unable to answer the phone, and who do not answer calls from unknown/unfamiliar telephone numbers.
• Question design: Survey question answer-choices could lead to vague data sets because, at times, they are relative only to a personal abstract notion concerning "strength of choice". For instance, the choice "moderately agree" may mean different things to different subjects and anyone interpreting the data for correlation. Even "yes" or "no" answers are problematic because subjects may for instance put "no" if the choice "only once" is not available.

Non-response Reduction

The following ways have been recommended for reducing non-response in telephone and face-to-face surveys:

• Advance letter: A short letter sent in advance to inform the sampled respondents about the upcoming survey.
• Training: The interviewers are thoroughly trained in how to ask respondents questions, work with computers and make schedules for callbacks to respondents who were not reached.
• Short introduction: The interviewer gives the basic information on him/herself and the survey.
• Respondent-friendly survey questionnaire: Questions must be clear, non-offensive, and easy to respond to.