In a perfect world you could send your online survey to your audience, and get statistically relevant survey data back within a couple of days. Unfortunately, figuring out how to get survey responses is a challenge we all know too well. The issue is so common that there are entire sub reddits and communities dedicated to filling out other people's surveys and finding survey respondents.
So unless you have a research budget for incentives, you're left to find survey respondents on your own. Luckily, you can partially solve your pain by leveraging artificial intelligence. Tools like ChatGPT allow you to create synthetic users. These virtual representations of real users can complement the data you collect from real humans. In other words, you can get free survey responses, making AI a valuable asset in yet another aspect of your life: conducting research.
In this article you'll learn when to use ChatGPT to generate survey results and which prompts to use. You'll also learn about the science of synthetic users, in case you're skeptical about the approach (as you should be!). Whether you’re testing a new questionnaire or need a starting point before reaching real participants, AI can help bridge the gap—quickly and cost-effectively.
The Role of AI in Survey Tools and in Generating Responses
There's many benefits of integrating AI in your survey building process. We'll only cover one benefit in depth: getting more responses for your survey. Nonetheless, here's a general overview of other juicy AI-powered survey benefits:
Brainstorming Survey Designs
If you only use ChatGPT in one part of the survey building process, it should be for ideation and survey design. Even with basic prompts, you'll be able to rapidly iterate on different questions, question types and flows for your survey. In fact, we've written an article entirely dedicated to getting the most out of AI survey design.
Improving Survey Question Phrasing
You might already have a survey prepared but you're just unsure about the phrasing of certain questions. Don't worry, ChatGPT has you covered. For example, here's a great prompt to analyze a specific question for potential biases: "Analyze the most significant bias in the following survey question, explaining how it may influence responses and suggesting ways to rephrase it for greater neutrality."
Quickly running your questions through ChatGPT before publishing is an easy way to avoid obvious biases in your questionnaires.
Automatic Data Analyses
You'll get the most out of AI's advice when analyzing open-ended questions. AI, and LLMs specifically, are ideally suited to detect patterns, analyze sentiment and classify unstructured responses in questionnaires. In contrast, AI's added value in analyzing closed-ended questions is rather limited. You'll typically only need (basic) statistical analyzes to summarise responses and extract insights.
Generating Survey Responses with ChatGPT: Step-by-Step Guide
It's important to note that we're writing this article on the day that OpenAI launched Operator (24/01/2025). We expect that Operator will enable new methods to have AI fill forms. We'll update this section whenever Operator is available within the EU.
Step 1: Create a Synthetic User to Fill the Form
To increase the accuracy of the generated answers you'll want to first provide ChatGPT with as much data on your target audience as possible. This can range from simply describing the desired respondent persona, to providing results from previous surveys or even full interviews. Here's an example prompt you can use as a template. Notice that the prompt not only specifies superficial elements (i.e. the persona's profession) but also their pain points, interests, etc.
"Create a detailed synthetic persona of a Customer Success Manager (CSM) at a fast-growing DTC e-commerce startup, including their background, role, key objectives, and daily responsibilities. Describe their biggest challenges, such as customer retention, post-purchase drop-off, and scaling personalized engagement, as well as their interactions with marketing, fulfillment, and product teams. Outline their mindset, decision-making process, and the tools they use, ensuring consistency in how they prioritize tasks and handle customer interactions. This persona should be realistic, well-rounded, and detailed enough to reference in further discussions."
If you want more detailed methods to generate synthetic users, we wrote another article with concrete tips and tricks about that.
Step 2.A: Generate Individual Responses
If you're testing your survey, or if you're looking to supplement early-stage research then it might be sufficient to answer individual responses. In this case you can use a prompt like this one:
"From now on, answer all subsequent questions as if you were a [INSERT USER TYPE], adopting their mindset, challenges, and decision-making process. Your responses should reflect their demographics, motivations, pain points, and behavioral traits, providing thoughtful, context-driven answers. Stay consistent with their persona, ensuring responses align with their typical concerns and preferences. If a question seems irrelevant, interpret it as they would and respond accordingly. Maintain a natural and realistic tone, as if this user were genuinely answering."
Of course, an added benefit of this approach is that you can ask follow-up questions to the persona. In turn this might make you rethink certain questions or add additional ones for when you send the form to real people.
Step 2.B: Generate Bulk Responses
It's impractical to create responses one by one if you need a larger sample set of data. The prompt below lets you gather multiple responses in a CSV file. You can even start playing with the synthetic personas created in the first step to have a wider range of personas contributing to your dataset.
"Generate 20 distinct responses to the following survey (which will be pasted below) as if they were answered by different instances of a [INSERT PERSONA TYPE]. Each response should remain within the bounds of this persona type but vary in individual experiences, priorities, and opinions to reflect natural diversity. Ensure that responses are nuanced and realistic, avoiding repetition while staying true to the persona’s overall characteristics. Output the responses in CSV format, where each row represents a different instance of the persona and each column corresponds to a survey question."
The Science Behind AI-Generated Survey Responses
Is using artificial intelligence to generate survey answers simply cheating, or are they valid for research? In writing this article we wondered the same thing, hence why we went digging to see whether any smart people already did some research on the topic. Turns out they did, and the results are nuanced (as you would expect from smart people!).
A German study examined the potential of GPT-3.5 to predict public opinion in Germany, particularly in relation to voting behavior. They fed ChatGPT personas of German voters and benchmarked the AI generated voting results with the results of the actual 2017 election. The researchers discovered that GPT struggled to accurately represent voter opinions. It exhibited a preference toward Green and left-wing parties, overestimating support for these groups.
So have you been reading this far into our article to find out that the technique we discuss is completely invalid? Not really, another study from Google DeepMind and Stanford University indicated that AI-generated answers to a test survey matched responses from real persons with 85% accuracy. The key difference with the German study being that they first fed ChatGPT detailed data about the target audience (i.e. two-hour long interviews).
Future of Survey Research in an AI-Native World
As with all knowledge-related fields, AI is set to play in increasingly important role in future survey research. We expect LLMs to be able to increasingly approximate real human personas in accuracy, requiring less fine-tuning data from you. We'll be able to leverage AI for a wide range of insight-related tasks: from predicting customer satisfaction trends to identifying potential pain points before launching new products or services.
We would be tempted to say that AI will never entirely replace surveying real humans. But if we've learned one thing in the last couple of years, it's to never say never!
Frequently Asked Questions
Can ChatGPT fill out surveys accurately?
Yes, but it will largely depend on your prompt and the provided data. Organizations should supplement AI-generated answers with human feedback.
How can I use ChatGPT to get survey responses for free?
You can simulate responses by specifying personas and generating bulk answers in CSV format. You can also directly use OpenAI's Operator if you're a pro user.
How do I train ChatGPT on my own survey data?
Provide ChatGPT with past survey responses or customer feedback to generate more relevant answers. The AI can mimic trends but won’t fully replace real users.
What are the limitations of using AI to answer surveys?
AI-generated responses may include bias, overgeneralization, or lack of unpredictability. It works best for pre-testing surveys and filling early research gaps.
Is it ethical to use AI-generated responses in survey research?
Using AI in surveys is ethical if responses are clearly labeled as synthetic. Organizations should avoid misrepresenting AI-generated answers as real feedback.
Can AI help improve my survey question quality?
Yes! AI can analyze survey wording, detect bias, and suggest clearer phrasing. This helps ensure survey questions are neutral and easy to understand.
