· Andrew Le Breuilly · 5 min read · Rethinking Survey Data — Part 6
AI Ready Survey Data Starts with Structure Not Prompts
AI cannot compensate for missing survey context. Reliable AI-assisted analysis starts with structured responses, metadata and governed transformations.
There is a tempting idea that we can put an AI assistant in front of any dataset and simply start asking questions.
Sometimes the demonstration even works. Ask for a summary, receive a confident answer and it can feel as though the difficult data work has disappeared.
Survey data gives us a useful reality check.
If the system sees a variable called Q12 with a value of 2, it cannot know whether that means “Quite satisfied”, “Female”, “Twice a week” or a particular brand. A clever prompt cannot recover context that never reached the model.
If we want reliable AI-assisted analysis, the work begins before we choose the model or write the prompt. It begins with the data.
A response does not explain itself
Human analysts rarely interpret survey values in isolation. We use the questionnaire, codebook and our understanding of how the study was designed.
An AI system needs access to the same context:
- Full question wording
- The answer options and code values
- Instructions shown to the respondent
- Whether the question allowed one answer or several
- Routing and analytical bases
- Whether a variable was collected or derived
- Definitions for nets, scores and groupings
- Relationships between questions and repeated units
Without that information, the system is being asked to make an educated guess. The answer may sound fluent while being analytically wrong.
That is precisely the kind of error that is hardest to spot.
Clean is not the same as meaningful
We often talk about preparing clean data for AI. Usually that means valid values, consistent formats and fewer missing fields.
All of that matters, but survey data also needs semantic context. The system must understand what each value represents and how it relates to the research design.
A well-structured survey model connects responses to respondents, questions, options and sources. It exposes definitions for derived variables and can record changes between waves.
This does not guarantee that an AI-generated answer will be correct. Nothing does. It gives the system a much stronger basis for interpretation and makes the route to the answer easier to inspect.
Not every transformation needs AI
This is another area where I think we need to be disciplined.
If a processing task has a defined rule, use a deterministic transformation. Recoding a value, applying a calculation or reshaping a question should be predictable, testable and repeatable.
AI earns its place when the task requires interpretation. Useful examples include:
- Classifying sentiment in open responses
- Detecting the language used in a verbatim
- Translating open text or labels
- Coding brand mentions and themes
- Creating structured classifications from natural language
Even then, AI should sit inside the governed data pipeline. We should know which source was used, what instruction was applied, which model produced the result and how the output will be checked.
The principle is simple: deterministic logic where the rules are known, AI where judgement adds value.
A conversational question can hide a lot of analysis
Imagine asking an AI assistant:
Has consideration increased among younger customers?
It sounds like a straightforward question. To answer it properly, the system may need to determine:
- Which variable represents consideration
- Which answer options count as consideration
- How “younger customers” are defined
- Which waves should be compared
- Whether weights should be applied
- Whether the bases are large enough to report
- Whether the difference is statistically significant
That is not just a language problem. It is a semantic and analytical problem.
The model needs access to definitions, relationships and calculation rules. Otherwise it may make reasonable-sounding choices that are inconsistent with the study.
Governance cannot be added afterwards
Survey data often contains personal information or commercially sensitive material. AI introduces additional questions about where the data is processed, which providers are used and how generated outputs are reviewed.
The UK’s Information Commissioner’s Office stresses that organisations remain responsible for governance, accountability and transparency when AI systems process personal data. Its guidance on AI and data protection is a useful starting point.
That is another reason to bring AI into the existing data workflow rather than sending extracts into disconnected tools. The organisation can choose approved services, control what data is provided and retain a clearer record of the operation.
Different projects will need different controls. Some may use a customer-selected model provider. Others may keep the entire process deterministic. The technology should follow the governance requirement.
The opportunity goes beyond automated commentary
Much of today’s discussion focuses on using AI to summarise results or write chart commentary. That is useful, but it is only the surface of the opportunity.
With the right data foundation, an assistant could help users find relevant questions, understand changes between waves, investigate anomalies, create governed transformations and explore results conversationally.
The limiting factor will not only be the intelligence of the model. It will be whether we have preserved enough context for the model to understand the survey.
AI-ready survey data starts long before the prompt. It starts with treating the survey as a properly engineered and governed data asset.
The previous article explains how reusable pipelines make this governance and context easier to retain.
The final post in the series follows the complete journey from a survey project to a reusable data asset.
