Trust Is a Metric: What an AI Summit Taught Me About Reliable Agents
Why trust, not capability, is the bottleneck
Agents can already do a lot. What keeps them out of production is whether we can trust them. Last week I attended World Summit AI in Amsterdam. Across the sessions I followed (workshops on agent harnesses, talks on agents that improve over time, and panels on hybrid architectures, privacy and healthcare safety), one conclusion kept coming back, and it is my main takeaway: trust has to be measured, not declared.
This is an opinion piece: my personal conclusions from the event, written from my notes and photos of the slides. The views are mine alone, not those of the speakers, the organisers or any company mentioned. It is my reading of the sessions, not a transcript, so treat the arguments as mine and the session titles as the starting point.
Context needs a quality score
A workshop on the agent harness made its argument in the title: minimum complete context is only possible with measurable reliability. The harness is everything around the model that decides what the agent sees, and that is where reliability is won or lost.
The two words in «minimum complete» pull in opposite directions. Minimum means only what the task needs, which keeps cost and noise down. Complete means nothing the task depends on is missing, so the agent never has to guess. In my view, without a measurement you can claim both and verify neither.
That is where, as I see it, a vendor such as Iris.ai fits in this picture: supplying context quality metrics, so each context can be scored before it reaches the agent and tracked across runs. The exact metrics are the vendor’s to define. The signals I would want to see are:
- Relevance: is what we retrieved actually about the task?
- Completeness: is anything the task needs absent?
- Noise: how much of the context is redundant or distracting?
- Traceability: can every claim be tied back to a source?
Agents that improve over time
An agent that improves is a loop around the agent, not a one-off build. Two examples on stage showed the same pattern. The first loop: run the agent, collect feedback, trace what happened, then suggest a change to one of three things: the model weights, the harness code or the context. The second, a data product that gets better with use: a review agent studies past interactions to find context gaps and propose fixes, every change is tested with evaluations, and the logic behind published work is reused as context for future questions.
The discipline underneath is the same in both: improve with evidence, not instinct. Ship what passes the evals and catch what breaks. I would argue context is usually the fastest and cheapest of the three levers to change.
How much of the loop runs without people is a choice with three levels:
- Manual: humans analyse the traces and make the changes.
- Human in the loop: the AI proposes the change and a person reviews it.
- Fully automated: the loop closes on its own, which demands the strongest evaluation behind it.
A deep-agent harness gives that loop something to act on. Its capabilities group into four areas:
- Execution environment: filesystem and sandbox, where the agent acts safely.
- Context management: skills, memory and context offloading.
- Steering: human-in-the-loop controls to guide and approve.
- Delegation: planning and subagents to split complex work.
The rule of thumb I took away: raise the level of automation only as far as your measurements can support it.
Hybrid architectures for critical infrastructure
A panel on hybrid AI architectures for critical infrastructure raised a question every harness has to answer: where does each piece of the system run? Energy, industry, transport and finance cannot depend on a single environment. A hybrid architecture combines cloud with edge or on-premises compute, large models with small specialised ones, and different kinds of hardware, then places each workload where it fits best. What follows is my framing of the topic, not a summary of what the panel said.
The strategy comes down to four moves:
- Place by workload: latency-critical, sensitive decisions stay close to the asset. Heavy training and large-scale analysis run in the cloud.
- Right-size the model: small models for routine tasks, large ones only for complex reasoning.
- Keep control of data: regulated and proprietary data stay inside defined boundaries.
- Stay modular: open interfaces let you swap a model, a chip or a vendor without a redesign.
The advantages follow directly: resilience when a link or component fails, low latency for real-time control, easier security and data sovereignty, lower cost and energy use, and room to adopt new hardware later. For critical infrastructure, I think the question is not cloud or edge. It is how to combine them so the system is fast, secure and verifiably reliable, which brings us back to measurement.
Consent, governance and privacy in ambient AI
A panel asked whether we can still consent in an age of ambient intelligence. When sensors, assistants and agents collect data continuously and almost invisibly, a one-time click on terms and conditions stops meaning much. I believe trust has to be built into the system itself, in three layers:
- Security: least-privilege access for agents, sandboxed execution, defences against prompt injection and data poisoning, and a trace of every action so it can be audited.
- Governance: a named owner for each decision and dataset, risk assessments before and after deployment, compliance with rules such as GDPR and the EU AI Act, and human oversight in proportion to the risk.
- Privacy: collect the minimum, keep it only as long as needed, give people clear and revocable controls, and process locally where you can.
Notice how these connect to the earlier sections. Traces, evaluations and context metrics are not only engineering tools: they are the evidence a regulator, a customer or a patient will ask for.
Healthcare: how safe is safe enough?
Healthcare puts the question at its sharpest. What follows is my own framing: safety is built in layers. Reliable data comes first, because a model is only as dependable as what feeds it. Continuous validation after deployment comes next, since performance drifts. Last, a professional makes the final decision while the AI recommends.
Even softer qualities can be measured. The EmpathyAgent study scores agents on eight dimensions from 1 to 10, including understanding of the person, adaptability and legality, and reports that automatic scoring agrees with human scoring about 75% of the time. Two lessons carry over: define the dimensions explicitly, and keep people in the loop to validate the automatic judge. The study concerns embodied agents in simulation, so applying it to clinical or customer settings is an analogy, not a result.
What to do on Monday
- Score your context. Define relevance, completeness and noise checks, and track them per run.
- Trace everything. If you cannot see what the agent did, you cannot improve it or defend it.
- Evaluate every change. Treat prompts, harness code and context like code: nothing ships without evals.
- Choose your automation level on purpose. Start with human in the loop and move up as the metrics earn it.
- Place workloads deliberately. Decide what runs at the edge, what runs in the cloud and which data never leaves your boundary.
- Design privacy in, not on. Minimise data, make consent revocable and keep an audit trail.
The thread I saw through every session was the same. Capability gets an agent into the room. Measurement is what lets it stay. That is my opinion, and I would like to hear where you disagree.
Sources
Session and panel titles are taken from slides I photographed at World Summit AI, Amsterdam.
- EmpathyAgent: Can Embodied Agents Conduct Empathetic Actions? (arXiv)
- Session and panel titles are taken from slides I photographed at World Summit AI, Amsterdam.
- https://worldsummit.ai/
