For anyone carrying out their own market research online there are a number of pitfalls to watch out for. Chris Brookes and Gill Wales of the Independent Consultants Group offer some in-depth guidance
When you’re considering doing a survey yourself, don’t forget that all surveys aren’t the same. It is fine to conduct certain types of online survey yourself, for example:
- customer feedback on a recent purchase experience
- incidence checks (to find out who is buying a particular product or doing a particular thing)
- PR stories
- employee surveys on non-sensitive subjects such as their views of the staff restaurant
- who's reading you and why
Why aren’t all surveys suitable for DIY research?
It’s because, for any given survey, there are different levels of technical complexity, sensitivity and business risk.
If you need to carry out technically complex projects such as market segmentation, pricing policy, or attitudes and motivation, then you will need specialist assistance.
Different survey topics have different levels of sensitivity. Sometimes it is because the subject matter is highly personal, such as personal hygiene, personal finance, or religion, or what someone thinks about their job or their boss. Sometimes it is because of perceived objectivity: will the respondent feel they can trust you with the information?
Often, an independent third party is important in order to reassure respondents of confidentiality. Examples of this are employee surveys and surveys that people think may be a selling exercise rather than genuine research. In addition, some projects are politically sensitive internally to the organisation conducting them. It’s worth taking professional advice if the survey subject might be sensitive for any of these reasons.
Research can generally be categorised as strategic (because it relates to the organisation’s long-term strategy) or tactical (because it relates to smaller, single issues).
Strategic research projects could have a major impact on your organisation’s future, so there is greater risk attached to them. Generating unreliable or biased results could have serious consequences. For strategic research surveys it’s important that professional researchers are always involved.
As with all other research methods, online research can often be of greater value if combined with other approaches, such as focus groups or face-to-face interviews. These (qualitative) techniques can enable deep exploration of issues that are specific or unique to the organisation.
How to write a good questionnaire
The purpose of a questionnaire is to collect information that you can compile into statistics. This means that the questions have to be clearly understood without any risk of people interpreting the same question in different ways.
If the information you collect is to have any value, it is essential that questions are not leading and that people feel able to give clear and truthful answers.
A poorly constructed questionnaire will generate vague responses that either don’t answer your objectives or, worse still, give you unreliable data.
Some complicated types of research need interviewers to ask the questions and/or complex analysis systems to process the data. But even the most straightforward questionnaire design benefits from the knowledge of some basic rules.
The importance of sampling
The type and number of people you include in your survey (your sample) is even more important than writing a good questionnaire. You can write the best questionnaire ever created, but if you end up with responses from an unrepresentative sample of people, or you invite responses from people to whom the survey topic isn’t relevant, then you’re no closer to getting the information you need. You may have collected data that is misleading or just plain wrong. You also need to take care how you use and store the data, bearing in mind the Data Protection Act.
For example, you might be asking about a recent purchase experience, but if the people you invite to take part in the survey include ex-customers, prospective customers, or people who haven’t received their order yet, that part of your sample will find it impossible to respond and you may create a poor impression.
So, before you write a questionnaire, think about what type of person you need answers from. Then work out how you are going to reach them. Check e-mail lists for relevance, currency and completeness. Don’t forget that the contacts on e-mail lists may also be used by other departments. Your potential research respondents could even be in dispute with another department, their account could be in arrears, or they may have recently notified your business about a bereavement.
Avoid embarrassing your organisation and causing distress to the respondent by checking the quality of your list. You’ll also need to co-ordinate your surveys to ensure that you’re not contacting people too often, inappropriately or inconsistently.
If you are posting the survey on a website, does that website have the type of traffic you need? Also, it is usually important to know the survey response rate — that is, what percentage of the people who saw the survey actually participated.
The validity of the research results depends not only on the number of people who take part, but also how well those people represent the audience you are researching. Does your survey sample share the same characteristics (e.g. spread of age and occupations, mix of genders, products purchased) as the population it is intended to represent? For example, if you e-mailed your questionnaire to 100 men and 100 women and only get responses from women, then the results will not be representative.
Making sense of the findings
It is important to accurately analyse and communicate your findings. You will probably have both open-ended and closed questions to analyse, will need to know how to read data tables, and should consider the use of cross tabulations.
Think about the person or people who will be using and acting on the findings; what information do you need to give them? Above all, you will have to organise and interpret the data so that conclusions are reliable and the common pitfalls of working with survey data are avoided. Choosing appropriate types of charts, and a consideration of the nature and size of the sample you achieved, are important elements here.
The journalists’ Who, What, When, How and Why questions are a useful guide when you are preparing a report:
- Why was the research done?
- What does your audience really want to know?
- Who did you survey and who needs to know the results?
- When do they need the results?
- How best to report the results? A PowerPoint presentation, Word document, verbal briefing?
- So what? Ask yourself if the survey results tell you enough to make firm recommendations for action, or do they raise new questions, needing more research?
Six top tips for DIY research
- Start with your end-goal in mind — as you are writing the questionnaire, consider how you will present the findings.
- Keep your survey as short and simple as the topic allows.
- Doing a survey isn’t just about writing questions: your sample and how you communicate your findings are equally important.
- Involve (at least) one other pair of eyes at each stage, to help review, pilot, or check understanding.
- Liaise with people who can help you, either internally or externally.
- Don’t be afraid to ask for help.