Over the years, I’ve seen my fair share of bad surveys. Those tend to be easier to identify than good ones. You’ve probably taken some yourself – they often have poorly worded questions and tend to drag on forever.
Good surveys, on the other hand, are forgettable. The best experience you can aim for is for the respondent to finish and think, “Well, that was tolerable.”
Want to learn how to create good surveys? There are some basic guidelines to make sure people walk away without feeling frustrated from an overly complicated survey design. Here are a few tips you should follow.
1) Make the survey as short as possible.
Focus on what’s really important. What data do you need to make your argument, launch your campaign, or change a product? Extraneous, “good to know” questions bog down surveys and deaden their focus. The most dangerous words you can hear when developing questions is “Wouldn’t it be interesting if …” Remember that your survey respondents don’t really care about what is interesting to you or your company – they care about how quickly they can finish the survey.
If you’re experiencing pushback on survey length, it’s always a good idea to note that drop-offs happen more with longer surveys. What would your boss prefer: A nice and thorough survey with only 15 completed responses, or a shorter, tighter survey with 200 responses? I’m guessing the latter.
I know the reality is that sometimes your survey has to be on the longer side. If that’s the case, guide your respondents through the sections. Let them know what you’re going to ask them. Give them cues when they’re almost done (“In this last section, we’re going to ask you …”). If your survey tool allows it, show them a progress bar so they know how much of the survey is left. And always thank them for taking the time to give feedback.
2) Don’t ask “yes” or “no” questions.
Respondents have a tendency to answer yes when asked a “yes” or “no” question. It’s a psychological bias (unless you ask if they’ve done something wrong). Plus, professional survey takers (they exist!) will immediately know they should answer “yes” to move further in the survey, putting them closer to getting an incentive or prize for finishing it.
Instead of asking directly, try to get at the answer in a more roundabout way. Give them a picklist and ask if they use or know any of the items listed. Only let respondents move on if they happen to select the item(s) that you are interested in. Then, the aim of your study is hidden to professional survey takers and you can be sure the people who answer your question are more likely to know what they’re talking about.
Don’t do this:
Do this:
3) Randomize your answer options.
There’s also a first choice bias in surveys, where people automatically click the first answer listed. It’s prevalent in “select all that apply” type questions. Randomizing your options helps to combat a survey taker’s tendency to check the first option they’re given.
Most survey tools will allow you to anchor options such as “Don’t know” or “None of the above” at the bottom of the list and exclude them from randomization. (Note: Please don’t randomize numerical or alphabetical lists such as age ranges or country lists. That will make the survey much harder to take.)
Not so great:
Perfect:
4) Try to keep your question text neutral.
You’ll influence your respondents if you ask a leading question. That could suit your needs, but be aware that if you publish your results and people see the leading question text, they may end up questioning the authority of your data. Here’s an extreme example to show you what I mean: “Don’t you think product X is amazing in the following ways? Yes, it is amazing because of x. It’s amazing because of y. It’s amazing because of z.”
Instead, ask: “How would you rate product X on a scale of 1 to 5?” And if you want to know what specifically they like, you can follow up with people who answer 4 or 5 on why they love it. You can do that with the people who answer the lower ratings, too. This gives you way more actionable data on what people love about your product and what you need to work on.
Leading question that is clearly pushing an agenda:
Neutral questions that will get you honest feedback:
5) Use matrix questions judiciously.
Your survey should not have more than a few matrices. These are precious, complicated questions. Use them for questions that really matter.
Also, each matrix should not have more than 5-7 row or header options. I am a seasoned survey taker and when I see a 15 x 8 matrix I run away crying. Or, more accurately, I close the survey window. Don’t let that happen to you.
This is manageable:
This will cause people to revert to grade school, back when they made patterns on their Scantron sheet:
6) Make sure your question text and answer options allow for every type of survey respondent.
You may live and breathe your product or industry, but you should not assume your survey respondent knows what you’re talking about. It never hurts to give examples and explain concepts or jargon to educate respondents who are less knowledgeable than you. Likewise, make sure you give answer options that give your respondent an out if they don’t know an answer.
A simple example of this is asking if someone’s team got bigger or smaller. While it may seem innocuous on the surface, you’re actually excluding some possible answers. Maybe the respondent’s team stayed the same or the respondent is a new hire and does not know how big the group was a year ago. Make sure you have options those people can select.
Not quite right:
Awesome:
7) Bonus: Include a “Red Herring” question to weed out inattentive or fake respondents.
As a quality check, you can ask a simple demographic question at the beginning and end of a survey. This is especially important when you’re designing long surveys. You may be surprised (and dismayed) to see how many respondents forget what country they’re from or how many employees their company has.
You shouldn’t always toss out respondents based on just the red herring mismatch, but you can use it in conjunction with other checks to validate the quality of a respondent’s answers. For example, only remove a response if the survey taker failed the red herring and answered “Don’t know” for 5 out of 10 questions in your survey. My favorite quality check is to ask for the size of a company and then the size of a specific team, like a marketing team. If a respondent indicates that their marketing team is bigger than their whole company, that is a pretty strong indication their answers are going to be wacky.
At the end of the day, you’re asking busy people to take time out of their day and give you data you want. To ensure that they have the best experience and you get the best quality data, try to incorporate the tips above.
from HubSpot Marketing Blog http://ift.tt/1e3d3ML
from Tumblr http://ift.tt/1S4Aeof
No comments:
Post a Comment