Editorial
The Gist
- Understand significance. Even small samples can yield significant insights if selected correctly and representative of the larger population.
- Reduce biases. Implementing diverse sampling methods is crucial to avoid biases and ensure the representativeness of survey results.
- Cost-effective insights. Smaller, well-calculated samples can provide reliable insights without the need for excessive data collection.
Wondering how many people in the United States have a smartphone? If you poll 20 people on the street, your survey won’t be representative of the entire American population. But if you just want to know how many of the people who live on your street have a smartphone, polling 20 neighbors is probably enough to get an accurate picture.
In essence, your sample size for surveys will vary according to the size of the group you’re polling. How do you calculate how many respondents you need answers from to obtain accurate results? Understanding this is crucial to ensure the reliability of your customer experience survey data.
Why Does Customer Experience Survey Sample Size Matter?
When it comes to surveys, bigger doesn’t always mean better. Clients often assume they need to examine every customer comment or entice most of their customers to take their survey to gather statistically significant results.
But here’s the truth: You don’t need to hear from every customer experience survey respondent to make confident inferences about the entire population.
Thanks to a statistical method called sampling, you can eliminate the guesswork and calculate exactly how many survey responses you need to get replicable, reliable results.
Maybe you have thousands of customer comments you’d like to analyze. We just need to analyze about 370 of them (per population) to make an accurate analysis. Maybe you’ve sent your survey to thousands of your customers. We just need a fraction of them to reply.
Related Article: How to Create Effective Customer Surveys and Obtain Actionable Insights
Sample Size for Surveys: The Big Picture
Let’s start with some definitions:
- A population is the whole group you want to make inferences about.
- A sample is a small portion of the population that you analyze.
By examining a small fraction of your data and determining the customer experience survey sample size, you can infer what an entire population feels and thinks, regardless of its size.
Don’t worry if this sounds too technical. It’s a simple concept you probably use every time you cook.
Imagine you’re making a pot of stew. Maybe you’re making just a small batch for yourself, or maybe you’re making a huge batch to feed a crowd. Either way, you only need to take a small taste of the stew to see if you need to add salt or pepper. Of course, this only works if you’ve given your stew a good stir to ensure it’s well mixed — and we’ll dive into that part of sampling soon.
Calculating the sample size for your survey is pretty straightforward: it just means deciding exactly how big your “taste” needs to be based on the size of your population and desired margin of error.
Sample Size for Surveys: The Details
If you already know all the details of sampling or if you need to calculate a sample size quickly, there are many sample size calculators available on the internet. Based on your population size, confidence level, and desired margin of error, these calculators will determine your ideal sample size.
But if you need a primer on what all those terms mean and how they work together, read on.
So, how does the sampling process work? Let’s assume you have a spreadsheet with 5,000 online customer reviews. You’re interested in using text analysis to look for trends in what your customers said.
First, look at your data to see whether your population is homogenous or diverse. For example, dine-in restaurant customers may have very different feedback than to-go order customers or catering customers.
Learning Opportunities
Conference
May
22
Salesforce's Connections Chicago 2024
Register
Conference
Jun
3
Customer Contact Week Las Vegas 2024
Register
You need to examine a sample of a few hundred comments from each customer group to get an accurate representation of each population’s opinions.
Related Article: Building Winning Customer Satisfaction (CSAT) Surveys
A Warning About Survey Sample Size Biases
For sampling to work, two things are critical:
- Your sample must be large enough.
- It must be representative of the entire population.
Bias creeps in when your sample isn’t representative of the entire population. Here’s a simple example: Imagine your population is 500 iPhone users and 500 Android users, but your sample includes nine iPhone users and 269 Android users. Your sample is biased, and any conclusions you draw will be skewed.
This is the simplest form of sample bias, but it’s not the only one. Here are a few common types of sample bias:
- Nonresponses: Nonresponse bias happens when not enough participants reply to your survey. If you don’t get enough survey responses, you can’t extrapolate from your results to the full population.
- Bias through convenience: Beware when you implement voluntary surveys. They can be convenient, but because of their convenience, they only include information from people who signed up to take the survey in the first place. These types of surveys miss all the people who are part of the target population but didn’t register to take the survey.
- Selection bias: This type of bias happens when researchers exclude parts of the population they plan to study. If you conduct a survey through the mail, you won’t reach people who lack a mailing address.
Related Article: Making Customer Surveys Count in the B2B, Industrial Worlds
How to Calculate Sample Size for Your Surveys
So how do you avoid introducing bias into your sample to ensure it represents the overall population? Here’s how the sampling process works:
Step 1: What’s Your Population?
Think carefully about how you will bound your population. If you’re analyzing a dataset of online restaurant reviews, will you include reviews from food delivery couriers? Former restaurant employees? Verified customers only, or anyone who claims to be a customer?
Step 2: What Sampling Method Do You Want to Use?
There are several common sampling methods:
- Simple random sampling, like drawing lots or using a random number generator
- Stratified sampling: first classify the population into subgroups and then randomly sample each subgroup. This method is useful for ensuring that all subgroups are represented.
- Systematic sampling: select every Nth individual from a list of your population, using a random starting point.
Step 3: How to Calculate Your Survey Sample Size
Last, to calculate the right sample size for your survey, you need to know these variables:
- Population size
- Margin of error
- Confidence level
- Standard deviation
Population Size: Based on how you defined your population in Step 1 above, your population size should be clear.
Margin of error: The lower your margin of error, the more closely your results will match the entire population. Polling studies often use a margin of error of 5%. (And remember, that’s plus or minus 5% — a spread of 10%). If the stakes are higher (like you’re building a NASA rocket), your margin of error should be lower.
Confidence level: Often, statisticians refer to a confidence level. What’s that? Let’s assume a confidence level of 95%. If you were to repeat your survey or analysis over and over, your findings would match the actual population 19 times out of 20, or 95%. When you combine your confidence level (for example, 95%) with the margin of error (for example, 5%), you can say that you are 95% certain that your survey sample’s results are within 5% of the true measurement of your overall population.
Once you’ve decided on the margin of error and confidence level you need, you can easily calculate the sample size for your survey’s results to be statistically valid.
Surprisingly, the survey sample size you need is probably much smaller than you would think, especially for large numbers. Assuming a confidence level of 95% and a margin of error of 5%:
- You’ll need a sample of 80 if your population is 100.
- You’ll need a sample of 278 if your population is 1,000.
- You’ll need a sample of 385 if your population is 1,000,000.
Final Thoughts on Customer Experience Survey Sampling
So, if you think it would be prohibitively expensive to tag every single customer comment in your dataset, think again. You probably need to tag just a few hundred of them to get accurate insights.
When it’s done right, sampling takes the guesswork out of achieving statistically significant customer experience survey findings — and that means less work for you.
Learn how you can join our contributor community.
