AI agent platforms: 7 criteria to choose one

All agent demos look alike: a question, a fluent answer, an action carried out in seconds. What sets one platform apart from another does not show in a demo. It shows the day an agent reads a document it should not have read, sends an email without approval, or when your DPO asks where the data went. This guide gathers the seven questions we would ask any vendor, ourselves included, and the evidence to ask for on each.

Assistant, agent builder, platform: what are we talking about?

Three families of tools are sold today under the same word, “agent”. Telling them apart avoids comparing things that do not play the same role.

• A conversational assistant answers one person’s questions, from what they give it or a few connected sources. It helps individually, but does not run processes for the team.
• An agent builder lets you assemble agents: instructions, tools, sometimes a sequence of steps. It is often designed for technical teams, and leaves governance, traceability and access rights to you.
• An agent platform brings together company data, agents, approvals and traceability in one frame, shared by several teams. That is what you need when agents act on behalf of the company, on data that is not accessible to everyone.

The criteria below mostly address that third case. If you are looking for an assistant for a few people, several of them still apply, especially the first two.

The 7 criteria

1. Where does your data live?

The question does not stop at where the platform is hosted. Your documents, their backups, the index, the logs and the model calls can live in different places, with different subprocessors. Ask where each of these lives, whether your data is used to train models, how long it is kept, and what happens when you remove a document from a source.

2. Who sees what?

An agent connected to your drive or your CRM must not see more than the person using it. The platform has to carry over the access rights of each source, and apply them when the agent searches, not only when it answers. Ask how a change of rights in the source is reflected, and whether data access is logged.

3. Who approves what?

Reading and summarising is one thing; emailing a customer, editing a record or triggering a payment is another. A serious platform lets you define which actions require human approval, by whom, and within which limits. An action subject to approval has to stop and wait, not run and send a notification afterwards.

4. What is traced?

When an agent gets something wrong, you need to be able to reconstruct what it did: the request, the sources read, the steps, the tools called, the approvals and the result. Without that trace, a mistake remains a mystery and trust does not come back. Ask to see the trace of a real execution, and how it can be exported to your own audit tools.

5. Which models?

Models change fast, and so does your policy on them. Check that you can choose the model providers you approve, switch without rebuilding everything, and that personal data is handled before it is sent to a model. Be wary of a platform that depends on a single provider without saying so.

6. Which tools?

An agent is only useful if it reaches the tools where the work happens: messaging, CRM, support, storage, business software. List your tools and ask, for each, whether the platform can read, write or both, and on whose behalf the action is taken. An integration that acts with a technical account holding broad rights bypasses everything you set up in criterion 2.

7. How do you leave?

Reversibility is negotiated on the way in, not on the way out. Ask what you can export (agents, configurations, traces), in which format, and how quickly your data is deleted at the end of the contract. A platform that answers this question clearly has usually thought about the other six.

The grid: question to ask, evidence to request

A spoken answer is not enough. For each criterion, here is the evidence that lets you check what you are told.

Common pitfalls

• Judging on a demo prepared with clean data. Ask for a trial on your documents, with their duplicates, scans and contradictory versions.
• Confusing “our servers are in France” with “your data stays in France”. Model calls, backups and support can go elsewhere.
• Forgetting access rights. An agent connected with an admin account sees everything, and will tell whoever asks.
• Connecting write actions before approvals are defined. Start with reading, then open writing one action at a time.
• Measuring success by the number of agents created. What counts is the work actually done, and the time your teams no longer spend redoing it.
• Evaluating alone. Business, IT, security and legal do not look at the same criteria; bring them together from the first demo.

How we answer at Scalt

Since we ask others these questions, here are our answers, as they appear on our website and in our documents.

• Data: the platform, your documents and their backups are hosted in France. Your data is not used to train models. A document removed from a source is purged from the index within 24 hours, and retention periods are set source by source.
• Access: each agent only reads what it is allowed to read, according to your sources’ rights. Data access is limited and logged.
• Approvals and traces: you define the approval steps; an action subject to approval stops and waits for your consent. Every execution is traced in Cases.
• Models: you choose the model providers you approve. Personal data is anonymised before any call to a model.
• Documents: the DPA, the list of subprocessors, the record of processing and the security policy are available under NDA for your teams.

For available connectors and exit terms, ask us during a demo, with your list of tools: these are points to check with any vendor, ourselves included.

Frequently asked questions