Five Quick Steps to Improved Survey Results
By Mike Kilgore
Aug 29, 2002 - 11:10:00 AM

A few months back, a friend of mine (let�s call him �Pete,� to protect the unfortunate) got into some trouble. While traveling through California, his car ran out of gas. Pete was left standing by the roadside awaiting help for over an hour and was subsequently late for an important appointment.

�How could you run out of gas?� I asked incredulously. Pete stared down at his hands before responding. �My fuel gauge was broken,� he replied, �It showed a half a tank of gas left. If I had known how little there really was, I certainly would have filled up the tank before hitting the road.�

This experience highlights a poignant truth for business -- In order to effectively and efficiently get where you are going, you need to know the status of your business right now.

It always amazes me how many people make assumptions about their business and their clients rather than devote some effort to good research prior to beginning a major project. A minimal investment of time and money could save tens of thousands of dollars and countless wasted hours of labor.

In my business, I often counsel clients in how to conduct effective market research. A critical but often overlooked element is the importance of a good survey. Ask the wrong questions, or ask them of the wrong people, and you have worthless data.

Here are a few tips to help you create a solid survey that delivers actionable information:

Clarify Question Text

Examine the questions carefully before distributing the survey. Ask someone less familiar with the project, such as a colleague, to review the questions for clarity. Good questions should be easy to understand.

What is being asked in the question �Did you travel to Europe or Africa last summer?� Is it asking you to respond �Yes� if you traveled to either Europe or Africa? Is it asking you to select the destination you visited? This seemingly simple question can be very confusing. �Did you travel to Europe last summer?� is a much clearer question.

Be Sure That the Correct Response is Available

I cannot tell you how often a survey question is presented to me for which there is no correct response. Think about the question, �If you plan to buy a new vehicle this year, what model will you purchase?� How do I respond if I do not intend to purchase a new vehicle this year? Should I select the choice I would purchase if I were to do so? Should I leave the question blank?

Consider rewording this question so it reads �If you were to purchase a vehicle this year, what vehicle would you likely buy?� This question clarifies that I should respond regardless of my intention to buy a vehicle this year or not. In a multiple response question, you might also consider including the choice �I do not intend to purchase a vehicle this year.� These are just two ways to simplify the survey for the respondent and ensure more accurate data.

Focus on a Single Point in Each Question

It may seem obvious that a question should contain a single thought, but many surveys contain questions in which this is not true. A typical �dual� question might ask something such as �Do you own a car or would you rather bicycle to work?� Isn�t it possible that I own a car and would rather bicycle to work? This question should be divided into two distinct questions.

Another example of a dual question is �What type of beverage do you prefer to drink with your meals?� I like to drink coffee or juice for breakfast, but rarely do so for dinner. How would I respond to this question? Would the data derived from my response be meaningful? This question, too, should be split into two or the meal in which the surveying company is interested should be specified.

Carefully Consider How the Data from the Survey will be Used

When I discuss with clients how they intend to use the data generated by their surveys, the dialogue usually results in the addition, subtraction, or revision of questions within the survey.

Asking the right questions of the right respondents is critical. Many of my clients intend to ask questions such as �What brand of widget do you prefer?� without clarifying that the respondent is part of their target audience. Adding a single qualifying question such as �Do you make purchasing decisions for widgets within your organization?� or �Do you use widgets?� allows us to filter out non-targeted respondents and ensures that the data derived are meaningful.

Using skips and branching within your survey also allows you to ask certain questions of only the appropriate respondents. If your survey would benefit from the inclusion of automated skips and branching, you may wish to consider using an electronic method for collecting survey data rather than relying on paper questionnaires.

Ask the Right Number of Questions

How many questions is the right number? That obviously depends on your audience. Paid survey respondents and focus group members can likely be asked a lot more questions than non-reimbursed respondents who are selected as they pass through your store or along the street.

If your survey asks too many questions, respondents might suffer from fatigue, resulting in inaccurate data. They may also become frustrated with the survey, which could result in a bad impression of your product or service. On the other hand, a survey needs to ask all of the questions which you require for meaningful data.

I recommend that you begin by including all of the questions in your survey that you would like to ask, and then trimming the survey down until you reach a reasonable length. While paid respondents will have a long attention span, respondents asked to take your survey in an intercept situation should probably not be asked more than 15 to 20 questions, depending on the complexity of the survey.

Creating a survey that provides your organization with actionable insights is a critical part of your market research efforts. Take time to review the survey carefully to ensure that questions are clearly worded and focused on a single point. Be sure that respondents have the ability to select the appropriate response. Consider how the data will be used and keep the survey length appropriate to the situation.

A good survey empowers you to make critical business decisions and maximize your efforts.

Mike Kilgore is the Vice President in charge of Operations for Feedback Systems, Inc. He can be reached at [email protected].


© Copyright 2001