Everyone thinks they can design a survey. After all, surveying is just a matter of asking questions, isn't it? Yes, but the questions have to be worded carefully. They must be clear, precise, and unambiguous. A well-crafted survey asks carefully chosen, insightful questions.
Choose the Right Question Format The questions you choose will depend on what you want to find out and how complex your survey needs to be. You can use questions with rating scales, checklists, multiple choices, and weighted scoring, or questions that require a person to write in the answer. You will have to decide which type is most appropriate for your purposes.
Questions that can be answered with a yes or no may be suitable, but they are not as powerful as questions that must be answered using a scale. For example, compare these two questions you might ask about your Web site:
Is this site useful? Choose either Yes or No.
I feel that this site is useful.
Answer by selecting Strongly Agree, Agree, Undecided, Disagree, or Strongly Disagree.
The second question provides more information than the first.
Some questions lend themselves to drop-down boxes or pull-down lists. For example, if you want to ask people which state they are from, you can ask them to type in their response, but it is much easier if you provide them with a drop-down list of all the states to select from.
If you want to ask a question that probes opinions about a limited number of things, you can use a check box. For example, if you want opinions about three different products, you can ask a question such as "Which product do you like best? Choose one of the following: A) scented shaving cream B) unscented shaving cream C) shaving cream for sensitive skin."
The open-ended question will provide the most information. For example, a question such as "What suggestions do you have for improving this site?" will generate ideas that might never have occurred to you. Open-ended questions will make statistical analysis more challenging but may provide more insight into customers' needs.
Make Your Survey as Short as Possible A survey with too many questions can chase away people who might otherwise be inclined to participate. Long surveys encourage people to skip questions, read only part of the question, or opt out midsurvey. A well-worded question can often replace two or three poorly worded ones. Your audience will become restless and possibly resentful if questions appear redundant.
Pose Your Questions with Data Analysis in Mind If you need to statistically analyze the responses, you must write questions that lend themselves to numerical treatment. That's why most surveys use multiple-choice questions or a scale of, say, 1 to 10 on which you can rate your answer.
In order to get responses that can be analyzed numerically, you must pose questions that don't allow a neutral response, because a neutral response isn't effective for initiating change or gauging opinion. If your survey includes a scale, select an even number to encourage respondents to choose an answer on one side of the divide. For example, on a scale of one to four, there is no middle number, so your participants must decide which side of the middle to choose instead of giving a neutral response.
Make Sure Your Questions Are Unambiguous Make sure that your questions are written in plain, easy-to-understand English. A question such as "Does your boss engage you in interactive dialogue?" is useless because the meaning of "interactive dialogue" is subject to interpretation.
Don't use words such as "often," "usually," or "normally." These terms mean different things to different people. What is "normal" for one person may not be normal at all for another. A question such as "Do your customers normally complain very much?" is useless because it is ambiguous.
Be careful to avoid using double negatives, such as "Isn't it true that you dislike junk mail?" since the wording is confusing and may create distortions in the responses as people misread or misunderstand what the question is asking.
Avoid asking two questions in one, as this may elicit two different answers. This puts the respondent in a quandary about how to answer the questions truthfully. For example, a question such as "Do you like the colors red and blue?" presents a conflict for a person who likes one but not the other.
Test Your Questions on a Sample Audience Testing your proposed survey on a sample audience will help you identify ambiguous questions and potential pitfalls for your data collection, so that you can correct your tack while you still have time. Ask your sample audience if they thought questions seemed redundant or if the survey seemed too long or confusing.