Survey Design – KISS (Keep It Short and Sweet)

We all hate homework. And filling out a survey is a little like homework. To get good feedback from a survey, it needs to be as painless as possible.

Most important is to keep the number of questions short. If the evaluation is too long, participants will simply not fill it out. While some conferences bribe participants with a T-shirt or similar prize for returning a survey, that does not mean that the quality is better. Keep it simple – the questions should fit on one side of one page, with plenty of white space for comments. An ideal number is 5 questions.

Survey Design

Whether rating a conference, a newsletter or a training session, it is tempting to make all questions about numbers. While numbers help in reporting statistics, they are often not very specific, or helpful when applying specific feedback to the next event or newsletter.

• Allow the responders to provide a context – are they evaluating one day of a conference, or the full multi-day conference? Is the trainee in a session experienced or completely new to the topic?
• Include one rating question about the value of the entire event. Opposing theories advocate an even number scale versus an odd number scale. For example, ratings from 1-6 force someone to lean on the high or low side, since there is no mid-point. A rating of 1-5 allows someone to be neutral, and give a 3 – right in the middle. Having a larger range – such as 1-10 – may not be much help, since the difference between a 7 and 8 is inconsequential.
• Do not include multiple rating questions for the same session. Over 50% of participants will circle the same rating for each question, so you are not getting differentiated feedback about different aspects (speaker knowledge/presentation skills/content/hand-outs). It’s often even a stretch when asking about content vs. facilities of a conference!
• Use open-ended questions that require a text response for more detailed and creative suggestions. You will be surprised how many thorough responses you receive when the majority of the questions are in this essay question style – some respondents even use the back page!

Survey Responses

Now it’s time to consolidate, analyze and act on the survey responses.

• Consolidating feedback electronically is the most helpful for calculating average scores. This may mean entering paper surveys into a spreadsheet, word processing document, or online survey tool – many of which provide out-of-the-box reports.
• Once consolidated, distribute the results to those being evaluated by the survey – be it conference organizers, trainers, speakers, newsletter authors.
• Keep in mind that some participants are "hard graders." Aim for above the mid-point of your scale; don’t expect a perfect score. It may also be helpful to break down the survey feedback by the level of experience of the responder.

Finally, act on the responses! If someone signed a survey and voiced a specific concern or question, make sure you provide an individual response. Announce anonymous summary results, when feasible, for example in a newsletter. Incorporate the feedback in future sessions, events and messages.

The more your audience sees that you are listening, the more likely it is that you will receive honest feedback in the future – including, of course, the compliments!