Tips For Developing Questionnaires & Surveys
/Surveys are the preferred elicitation technique when faced with a large number of stakeholders or when stakeholders are geographically dispersed and you need to gather the same information from them. Surveys can also be used to gather requirements anonymously. A practical application is as follows: using it to gather information on stakeholders’ reactions to existing systems or proposed requirements.
A survey may comprise open-ended questions that allow users to provide answers in their own way or closed questions where users provide answers from a range of options. If you ever need to develop a questionnaire or survey for eliciting requirements, here are some tips you should be aware of:
1. Start by identifying the objective of the survey. This will help you streamline the questions you need to ask to fulfil the objective. Remember that shorter surveys get higher response rates so don’t try to achieve too many objectives with one survey.
2. Determine the group of stakeholders to be sampled. For very small groups of less than 150, sample everyone, if possible. For larger groups comprising thousands, select the sample of participants that are representative of the user population.
3. In some cases, it may make sense to divide users into different categories. An example is designing a survey to focus on the different modules of an ERP application and having the users of each module respond to specific questions.
4. Determine what tools will be needed for the survey. There are many technologies dedicated to conducting surveys. One example is Survey Monkey. There are different types of surveys you may consider. Popular examples include web-based surveys, email surveys, telephone surveys and paper-based surveys.
5. Develop a rough draft of the questions and examine if the responses will fulfil the objectives of the survey. This stage is very important since the quality and type of questions you use will determine how your results will be analysed. Include instructions for completing the survey and details of whom to contact if there are questions. There are different types of survey questions. You may decide to use one or more of these variations:
- Multiple Choice
- Binary
- Ranking
- Closed
- Checklist response
- Production
- Number scaled
- Open-ended numeric
- Open-ended text
See Types of Questions To Use In Questionnaires/Surveys for more on the format of these variations of survey questions.
In framing questions, take note of the following:
- Consider rephrasing how questions are composed where necessary to be able to verify the accuracy of responses.
- Ensure that your questions are concise and easy to understand. Avoid the use of jargon and ambiguous words.
- Avoid the use of leading or double-barrelled questions. Composing questions for a survey requires you to follow the same rules you would if you were posing the questions verbally. Visualize the responses you would get and rephrase your questions as required.
- Move from general to specific questions.
- Arrange questions in a logical manner. Questions that are similar should be grouped within the same section. This categorisation can also be done based on type of question. For example, multiple-choice questions can be grouped together within the same section.
6. Test and revise the survey based on the results. This may involve conducting a pilot exercise so that it can be adequately validated. Observe how long it takes respondents to complete the survey so that you can get an accurate timing for your survey.
7. Make sure you send a cover letter or notification letter out with the survey that clearly indicates:
- The purpose of the survey
- The sponsor
- How the results will be used
- Any incentives for participating in the survey
- Whether or not the responses will be confidential
- The deadline for submitting a response &
- The estimated time to complete the questionnaire.
8. Distribute the survey and follow up with a reminder before the completion date.
9. At the end of the survey, tabulate and analyse the results. Are you on the right track? Are there any missing requirements?
What other tips do you think should be on this list?
User story maps are an interesting and collaborative way of eliciting user requirements. One of the reasons why I find it so powerful is because it provides a unique approach for aligning discussions relating to the user, their goals, the process that supports the accomplishment of their predefined goals; and the requirements that need to be addressed to solve business problems.