Surveys or questionnaires are one of the more widely used methods of obtaining information from individuals. A well-designed survey should address issues that invite meaningful responses from the participants being asked to complete them. The questions should be unambiguous and, at the same time, not coach the participant to select a response desired by the surveyor. "Loaded" surveys lessen the utility of the data collected and may even disqualify the results.
Here are some examples of problematic survey questions.
Does your office have enough heat and light? Agree Neutral Disagree
In answering this question, the respondent may be confused. What if there is enough heat but the lighting is dim? Or what if the lighting is OK, but the office is cold? This survey question is asking two different things but there are provisions for only one answer. The other concern is whether or not this is important to know, and if known, can anything be done about it. When participants are asked to set aside time to complete the surveys, the information provided should serve a purpose. No one likes to be bothered answering questions about which nothing can be done or nothing will be done. Another way of framing these types of questions is to ask a generic question and then invite an open response that provides meaningful information such as:
Does your office environment meet your needs? Yes No Unsure
If not, what can be done to improve your working space?
What are you doing about the plethora of Harmonia axyridis found in your office and home? Nothing at All Buying Poison and Using It Vacuuming Don't Know
Huh? This silly question evidently assumes a lot, perhaps too much, but it is included to prove a point. A survey question must be understood unambiguously and also clearly, otherwise one of three possibilities will occur: (1) you will not get an accurate answer, (2) you invite the don't know or no opinion response, or (3) your survey may be tossed in the trash. The same question could be written better, such as:
What are you doing about the crowds of Asian Ladybugs that are in your office and home?
Nothing at All Insect Poison Vacuuming Other:_____________
How old are you?
What is your income?
Indicate how many organizations you are involved with?
The first question will get you lots of numbers, depending on the age of the respondent. Instead, you should provide intervals for response, depending on the uses of the data. 12-15 years, 16-18 years, 19-21 years, or whatever fits the purpose of the survey. This is even more important for the second question. By providing very broad intervals plus the option to decline answering that question, you will be more likely to get the respondent to answer this question and others. The last question is vague. Involve, affiliate, identify, belong is subject to interpretation. And, what is an organization? Does church membership count? What if I only pay dues and never show up for meetings? Hummm. Again, questions need to be designed for the purposes of the survey and to allow respondents to select responses accurately and consistently.
Given that the federal defense budget exceeds $1500 per US resident, do you think $10 per resident is enough for our national parks and recreational lands? Yes No Don't Know
Talk about a loaded question! This is an example of manipulative information, that is, trying to get the respondent to check the "no" answer. Sometimes this is done deliberately, in order to influence voters or provide publicity for a cause, whether it's worthy or not. Ask the same question without reference to the national defense budget, and different answers may result.