Bring your own bucket, bring your own model

Ask a research team where their survey data goes once it enters a new platform, and you will often get a pause.

It is uploaded. It is processed. Something comes out the other end. But which AI model reads the open-ended responses? Could that model be learning from them? And when the project is finished, how do you get everything back out?

These are fair questions. Survey data can contain personal information, commercially sensitive findings and years of accumulated insight. Research agencies also hold that data on behalf of clients who ask exactly these questions in their procurement checklists.

We think you should be able to answer them without a pause. That is why two ideas sit at the heart of how Purify Intelligence is built: bring your own bucket and bring your own model.

A cog, not a walled garden

Most data platforms want to become the centre of your world. Your data goes in, and the easiest way to use it is to stay inside their tools, their dashboards and their storage.

We have taken a different view. Purify Intelligence is designed to be a cog in a larger machine. Your organisation already has a way of working: a cloud account, a warehouse, BI tools, statistical software and people who know how to use them. The job of a survey processing layer is to fit into that machine and make it run more smoothly, not to replace it.

For that to work, two things need to stay under your control:

1. Where the finished data is delivered once it has been processed
2. Which AI models are allowed to work with it, and on what terms

Bring your own bucket

With bring your own bucket, Purify Intelligence delivers processed data directly into your own Amazon S3 bucket. S3 is where we started, and support for more storage locations is on our roadmap.

Access is restricted and customer controlled. You create the bucket, you decide the permissions and you can revoke them at any time. Once the finished data lands, that copy sits in your cloud account, under your security policies, your retention rules and your audit trail. Your projects are also stored on our platform, so pipelines can be rerun as new waves arrive, and you can delete them whenever you choose.

This has some practical benefits:

• It slots into existing pipelines. If your warehouse, data lake or BI tool already reads from S3, the processed survey data simply appears where your other data already flows. There is no extra export step and no new place for analysts to look.
• It avoids lock-in. Your processed data is held in open formats such as Parquet in storage you own. If you ever stop using Purify Intelligence, you already hold a full copy of your processed data.
• It simplifies governance. The copy your teams work from is covered by the same controls your security team already applies to everything else.
• It keeps clients reassured. Agencies can tell clients exactly where their data is held and which account it is delivered to.

Survey data should end up as a reusable organisational asset, as we argued in our Rethinking Survey Data series. An asset is much more useful when it sits alongside the rest of your data, in a place you control.

Bring your own model

AI is now a routine part of survey processing. It can classify open-ended responses, identify languages, translate labels, clean verbatims and create new variables from existing answers. Purify Intelligence includes ready-made recipes for exactly these tasks.

But every time an AI model reads a response, it is fair to ask where that response has gone.

With bring your own model, you connect your own OpenAI or AWS Bedrock account. AI-assisted steps in the pipeline then run through that account, under the agreement your organisation has with that provider. That means:

• You choose the model. Use the one your organisation has approved, and change it when your policy changes.
• You see the usage and the cost. Spend appears in your own provider account, rather than being hidden inside someone else’s bill.
• You decide what the model remembers. The data handling terms are the ones you have agreed with your provider, not terms chosen on your behalf.

Forgetful by default

That last point deserves more attention, because it is where much of the anxiety around AI and research data comes from.

Our default is that AI should be forgetful. Purify Intelligence uses the model to complete a specific task, such as classifying a batch of responses, and does not use your data to train models. The response goes in, the structured result comes out and the result is stored as a visible, auditable step in your pipeline.

However, forgetful is a default, not a rule we impose. Some organisations want a model that learns from their data: a classifier tuned to their own brand names, their category language or the way their respondents describe a product. If that is what you want, it is your decision to make, in your own provider account, with your own data and on terms you have agreed. Your model, your data, your call.

What matters is that the choice is yours, and that you know which choice has been made.

Why this combination matters

Either idea is useful on its own. Together, they mean you can trace the whole journey of your survey data:

• It is processed through a governed pipeline where every transformation is recorded
• It stays in your project on our platform until you choose to delete it
• Any AI-assisted step runs through a model you selected, under terms you agreed
• The finished result is delivered to storage you own and control

There is no point in the process where your data disappears into a black box. That is the assurance research teams, their clients and their security teams increasingly need before AI can be used with confidence.

Control without extra work

None of this should mean more effort. Connecting a bucket or an AI provider is a one-off setup step. After that, the pipeline does the work: data is processed, AI recipes run through your chosen model and the finished output lands where your team expects to find it.

That is the balance we are aiming for with Purify Intelligence. You keep control of the decisions that matter, and we take care of the repetitive work in between.

If you would like to see how bring your own bucket and bring your own model would fit into your existing stack, get in touch.