From research to content tools: Online surveys

Test hypotheses and get the numbers with quantitative research

By Team CSI
|
May 23 2017
|
Categories: Articles

Online surveys are a great method for measuring attitudes or behaviours around content. They’re ideal for putting numbers behind the content insights you gathered during audience interviews.

Different question formats will help you uncover your audience’s priorities among topics, channels, goals, and preferred formats. You can also use surveys to find out which terms your audience prefers and understands, and their opinions of your content.

Get the information you need

The goal of quantitative research is to find statistically valid content insights. With thoughtful survey preparation and distribution, you can maximize your number of participants and the likelihood that they’ll provide useful data.

Maximize your survey participants and the likelihood that they’ll provide useful data with thoughtful preparation. Click To Tweet

Maximize your participants

Decide early on who you’re targeting and how you’ll reach potential participants. You should already know your audience segments from preparing for audience interviews. Decide whether you also want to target a segment of the general population to represent your potential audience.

Distribution methods

To reach your current customers, distribute your survey through:

Your email list
Your website
Social media channels

To reach potential customers in your target market, consider:

Paid social media ads
Relevant industry, professional, or social networks or clubs
Paid survey recruitment

Sample sizes

The more people respond to your survey, the more confident you can be that your results are accurate. We recommend aiming for ~400 respondents (95% confidence level), but being satisfied with at ~250 (90% confidence level).

Remember that if you’re going to segment your responses, you’ll have lower confidence levels for those smaller groups. Learn more about confidence levels and margins of error.

Incentives

To help reach this number, it’s often worthwhile to offer an incentive. We recommend a small token, like a $5 gift card, or the chance to be entered into a draw for a larger prize.

Ask clearly and concisely

Before you create your questions, make sure you know the purpose of your survey. This will form an overall framework for your questions. We recommend using the following guidelines to help ensure the maximum number of participants complete your survey, and that they provide useable data.

Be clear

Label scales with words rather than numbers.
Use a unipolar scale rather than a bipolar scale. Items should range from “extremely” to “not at all,” rather than, for example, “extremely useful” to “extremely unnecessary.”
Clarify everything that could be interpreted in more than one way.

Be brief

Tell participants how long the survey should take in the intro text.
Keep questions and surveys as short as possible. We recommend 5 – 10 minutes per survey.
Use skip logic to make sure participants only see questions that apply to them.

Avoid bias

Randomize options in multiple choice or ranking questions.
Remove unnecessary information or language that might indicate your hypotheses.
Avoid agreement scales. People tend to agree with whatever the statement is.

Structure strategically

Use intro and exit text. Set expectations and express your appreciation before and after.
Divide questions into categories. Use page titles and instructions.
Organize questions with an audience focus.
1. Screening questions: “Where do you live?”
2. Easy questions: “How often do you use Twitter?”
3. Difficult questions: “Rank these topics in order of importance.”
4. Personal questions: “What’s your job title?”

Choose the most useful question type

Some types of questions lend themselves to certain formats. Use the best format for each type of question to glean the most insights about your content.

Question format	Good for	Example
Ranking scale (also called the Likert scale)	Measuring satisfaction, effectiveness, likelihood, priority, or frequency.	How important is it to know what events are going on in your area? Extremely important Very important Somewhat important Not very important Not important at all
Multiple choice	Questions with a limited number of options Demographic questions.	How would you prefer to find about about events we’re holding in your area? Email Text Facebook Twitter I don’t want to know about your events
Open ended	Situations where attitudes and opinions are unknown. Gathering one or two word responses to take the temperature of a topic or situation.	What information do you think is the most important to know about [topic]?

Analyze and communicate your insights

Most survey tools make it easy to sort and filter responses based on various demographics. Start by segmenting by obvious categories like age, location, or whether participants are current clients/donors.

You can move on to more nuanced segments. For example, what percentage of 25-30 year old participants who read our newsletter are regularly on Facebook? Or which topics do 45 – 50 year old non-customers find most important?

Must survey tools can also present these responses in different ways. To get the best look at what you’ve discovered we recommend using the following visual representations:

Vertical bar chart – to compare the responses of 2 – 7 different groups.
Horizontal bar chart – to compare the responses of 8 or more different groups.
Pie chart – to illustrate a sample breakdown.

Remember to give context when communicating your findings to your team and beyond. Pull out themes and overall findings, then show the differences by segment. Do the work of insight gathering so others don’t have to.

Create your content tools

Results from your online survey should be used in every type of content tool. Once you’ve gathered content insights, you can use them to create:

Personas and scenarios
Audience messaging cards
Content mapped to customer journeys
Content mix

Stay tuned to learn how to turn these insights into content tools!