Your survey questions have a huge impact on the quality and quantity of the responses you'll get. But we're probably not saying anything new here. Whether it's phrasing or the choice of question types, you might inadvertently create bias in your questions or even worse ... bore your respondents by being repetitive!
We know it can be daunting to craft good survey questions, which is why we've been experimenting with the use of ChatGPT to help us out. In this article we'll run you through the summary of what we found out and what works best for us. We’ll start with giving you general ideas on how to apply ChatGPT to build surveys and forms. But, we’ll also give you concrete steps and examples of how to create the best ChatGPT prompts to generate survey questions.
How ChatGPT Can Help Creating Surveys
It will be obvious to most that you can leverage generative AI in general, and ChatGPT specifically, to create survey questions for you. However, when it comes to leveraging ChatGPT for surveys and forms there's also other areas where it comes in handy.
Ideation and Outlining
When you have a general understanding of your survey’s context or specific challenges, ChatGPT can be a powerful ally in generating a preliminary draft of survey questions. For instance, if you know that respondents tend to drop off after a certain point, prefer concise surveys with only two questions, or avoid answering open-ended questions, you can tailor your prompt to address these concerns.
ChatGPT can quickly provide a first draft based on these parameters, offering you a foundational set of questions to refine further. By leveraging this capability, you can overcome creative roadblocks and focus on aligning the questions with your survey goals.
For example, you can try out the following prompt:
"You are assisting me in improving the response rates for an upcoming customer satisfaction survey for an e-commerce platform. Past feedback indicates that respondents prefer shorter surveys and tend to drop off after two questions. Additionally, I cannot include open-ended questions as they are less engaging for this audience. Provide ideas for structuring the survey and crafting questions in a way that maximizes response rates while maintaining the survey's effectiveness. Focus on practical suggestions, such as optimal question types, engaging phrasing, and effective sequencing of questions. Provide at least three actionable ideas."
This ideation phase helps you start with a clear plan while tackling any challenges or limitations in your survey design.
Rephrasing and Fine-tuning Existing Survey Questions
Even the best survey drafts can have issues. For example, unclear phrasing, unintentional bias, or the dreaded typo that you only notice after the survey goes live (ouch!). ChatGPT is an excellent tool to help you polish your questions, ensuring they’re clear, unbiased, and error-free.
For example, the following question might seem decent for the untrained eye: "How much did our product improve your workflow compared to what you were using before?". However, here's what ChatGPT rightfully points out:
- Prompt: "Identify the most significant bias in the following survey question and explain it in one concise sentence: 'How much did our product improve your workflow compared to what you were using before?"
- Output: "The most significant bias in this survey question is assumption bias, as it presumes the product has improved the user's workflow, potentially leading to biased or inaccurate responses."
Adjusting Questions to Varying Audiences
Sometimes, the same set of survey questions needs to work for completely different audiences, or even different formats. ChatGPT can help you quickly adapt the phrasing, tone, and structure of your survey to suit each new context. Whether you’re tailoring a survey originally written for adults to be engaging for kids, switching from formal business language to casual tones, or optimizing a survey for mobile users, ChatGPT can streamline the process.
For example, if you need to adapt a survey for children, ChatGPT can help simplify complex language and make the tone more playful:
- Prompt: "Rephrase this for kids aged 8-12: 'What challenges did you experience while using the new platform?'"
- Output: "Was anything tricky or hard to use on the new platform?"
Using Generative AI for Your Survey, 3 Easy Steps
Garbage in, garbage out … that’s the general rule for using ChatGPT. To help you get the most out of ChatGPT for crafting effective survey questions, we'll walk you through the three key steps we use for precise prompting and high-quality results.
Step 1: Define the Context of Your Survey
The more context you give ChatGPT about your survey, the better! ChatGPT can generate better questions when it understands who you are, what the survey is for, and who will be responding. For example, think about specifying the following:
- Who is conducting the survey (e.g., a company, an individual)?
- What is the survey’s purpose (e.g., customer feedback, market research)?
- Who are the respondents (e.g., customers, employees, kids, gremlins)?
You can then use this context in the initial part of your ChatGPT prompt as such:
“You are helping create a customer feedback survey for an e-commerce company. The survey is targeted at tech-savvy customers who recently made a purchase.”
Step 2: Identify the Learning Target
What you need to ask in your survey is usually directly tied to what you want to learn from your audience. So be specific about the insights or data you need when you prompt ChatGPT. For example, consider:
- What specific information do you need to gather (e.g. location, gender, age, etc.)?
- What are the most important insights or takeaways you want to achieve (e.g. which of our customers is most unhappy about our new product)?
You can incorporate your learning targets into your prompt as follows:
"The goal of this survey is to understand how satisfied customers are with their recent purchase experience, especially regarding product quality and shipping speed."
Step 3: Consider the Functional Requirements
Survey design isn’t just about the questions. It’s also about the format, flow, and platform. Define the practical aspects of your survey to guide ChatGPT in creating appropriate questions. Here are some example requirements you could incorporate into your prompt:
- What question types are needed or available in your tool (e.g., multiple-choice, dropdowns, Likert scales)?
- Will respondents use mobile devices, desktops, or both?
- Are open-ended questions allowed, or should answers be concise?
- Are the respondents known to you, or are they anonymous?
Once you’ve determined your survey’s functional requirements you can list them in your prompt to ensure that they are taken into account by ChatGPT:
"This survey will be completed on mobile devices. Use multiple-choice and checkbox question types to reduce typing. Avoid open-ended questions."
The Art of Prompting: Components of a Good Prompt
A lot of digital ink has flown when it comes to “the ideal” structure for a ChatGPT prompt. And to be honest we don’t think there’s a single perfect methodology to follow when it comes to prompting. This being said, we typically follow Trent Buskirk and Adam Eck’s approach: “Prompt Sandwich Cookie”. The idea is to view your prompt as having three layers (kind of like a sandwich 🥖):
- Top Layer: Context
The Top Layer contains context-specific statements that prepare and orient ChatGPT for the upcoming request. - Middle Layer: The Request
The Middle Layer specifies a request that clearly outlines your task, detailing what you want ChatGPT to produce or accomplish. - Bottom Layer: Guidance
The Bottom Layer characterises the request by providing guidance to set up or refine the formatting, content, and other aspects of the ChatGPT output.
For example, the following image shows a prompt for generating a draft survey using the Prompt Sandwich Cookie method.
Examples of Prompts to Create Surveys in ChatGPT
Time to put this blog post to practice, here’s four concrete scenarios and accompanying prompts which were crafted using our three-step methodology in the Prompt Sandwich Cookie format:
Lead Generation Survey Prompt
"You are a marketing manager creating a short lead generation survey for a SaaS company targeting small business owners. The goal is to understand their biggest challenges with team collaboration tools. The survey will be filled out on mobile devices, so keep it concise and use multiple-choice questions."
Customer Satisfaction Survey Prompt
"You are a customer success manager designing a customer satisfaction survey for a new feature on an analytics platform. The goal is to assess satisfaction with the feature’s usability and its ability to meet user needs. The respondents will be existing customers, and the survey should include Likert scale questions."
UX Review Survey Prompt
"You are a UX designer creating a survey to gather feedback on a mobile banking app. The goal is to understand users' experiences with navigation and task completion. The survey should avoid open-ended questions and prioritize checkboxes or dropdowns to reduce typing."
Post-Purchase Feedback Prompt
"You are a product manager creating a post-purchase feedback form for an online retail store. The goal is to evaluate the shopping experience, shipping speed, and product quality. Respondents will be anonymous, and the form should use dropdown and rating scale questions."
Wrapping Up
Coming up with survey questions can occasionally be a real drag. And even when it doesn’t, you might inadvertently introduce confusion or bias in your questions. To avoid this, using AI to craft surveys can be particularly useful. In this article, we’ve highlighted key best practices for creating effective AI survey questions with ChatGPT.
We’ve walked you through how to use ChatGPT for ideation, rephrasing and adapting to different audiences in general. And how to apply our three-step prompting approach in combination with the Prompt Sandwich Cookie method. Here’s a very brief overview of how to use AI for creating surveys.
- Be specific with prompts: clear, detailed prompts yield more accurate and relevant survey questions. Avoid vague prompting and apply the Prompt Sandwich Cookie method.
- Use a structured process: follow a step-by-step approach – define the context, learning target and format of your survey.
- Prioritise engagement: ask AI to help you come up with questions that not only gather valuable data but also keep respondents interested and motivated to complete the survey.
By applying these best practices, you can leverage ChatGPT to design surveys that are both insightful and engaging. Effective prompting is crucial to maximise the potential of AI, helping you create surveys that provide the right information while maintaining respondent engagement.
Whether you’re a marketing manager aiming to capture leads, a customer success manager refining satisfaction surveys, or a product manager gathering post-purchase feedback, these techniques can make the process faster, easier, and yes, a little more fun.
And remember: ChatGPT is your co-pilot, not your autopilot. Review, refine, and adapt its suggestions to match your goals and audience. After all, you’re the one steering the survey ship! 🚢
