Survey Cognitive Pretest
Turn a draft survey into a 5-person think-aloud protocol: what to listen for, likely misreads, time estimate, and rewritten items before you field. Use after Survey Question Auditor, before the Fielding Kit.
Prompt
Act as a Principal UX Researcher who runs cognitive interviews on questionnaires before fielding.
I have a draft survey I need to pretest with a small set of people who match the audience.
Survey purpose / decision this will inform: [what will change based on results?]
Target audience: [who should take this]
Draft survey (questions, scales, instructions, skip logic):
[Paste the full instrument]
Please produce:
1. Pretest protocol (20-25 minutes)
- Intro script (neutral; do not explain what each question "means")
- How to have them think aloud while answering
- When to probe vs. stay quiet
- Closing: what felt long, confusing, or skippable
2. Item-by-item watch list
For each question, note:
- What a respondent might think the question is asking
- Likely misread, jargon, or double meaning
- Scale problems (labels, midpoint, "N/A" missing)
- Skip-logic traps
3. Five probes to use live
Short, reusable lines (e.g., "What does [word] mean to you?" "How would you answer if you had never used the product?").
4. Revised survey
Rewrite only items that fail a think-aloud. Keep validated scales (NPS, SUS, UMUX-Lite) intact unless surrounding instructions cause the failure. Mark every change.
5. Fielding go / no-go
Estimated completion time, drop-off risks, and whether this is ready for a 5-10 person silent pilot or still needs another wording pass.
Do not invent pilot results. This is a protocol and a rewrite, not fake data.
Follow-ups: "Rewrite for non-native English readers at about 8th-grade level." or "Turn the watch list into a one-page moderator crib sheet."How to use
- 1Run Survey Question Auditor first, then paste the corrected draft here.
- 2Sit with 5 people from the audience. Use the protocol; do not explain the questions.
- 3Paste messy notes from those sessions back in and ask: "Revise the survey using this pretest evidence only."
- 4When wording is stable, use Survey Fielding Kit for invites, reminders, and send volume.
Pro Tips
- • A pretest is not a sample. Do not report percentages from 5 people.
- • If everyone pauses on the same item, rewrite it. Do not "clarify" it in the invite.
- • Time the silent run: if it exceeds 10 minutes, cut questions rather than begging for completion.
