Trust, governance, and accountability in AI work
An agent sends a plausible but unauthorised commitment to a customer. The team can see what happened, but cannot say who granted the authority or who must repair the customer outcome. This is not merely an accuracy problem.
Justified trust
Operational trust means knowing what the system may do, tracing output to evidence, detecting boundary violations, recovering from them, and naming the owner accountable for the outcome.
Six operational foundations of trust
User enthusiasm or a high model-confidence score is not sufficient evidence of trustworthiness.
Boundary
Permitted data, tools, users, and actions are explicit.
Provenance
Sources and transformations behind an output are traceable.
Evaluation
Quality and risk are tested on examples that represent real use.
Ownership
Roles accountable for policy, system, and customer outcome are named.
Incident evidence
Who acted, when, with which data and authority can be inspected.
Recovery
Override, rollback, customer repair, and recurrence prevention are defined.
Governance case
An agent's unauthorised customer promise
A sales-support agent interprets a pricing table and sends a customer a two-year fixed-price commitment. The message is polished and attractive, but the agent has no authority to make a contractual promise.
Which controls are missing?
Preventive controls need send authority and price boundaries; detective controls need a commitment alert and trace; recovery needs message recall, a named customer owner, and a post-incident rule update.
Embed governance in the work
Prevent
Make unacceptable action difficult through authority and data boundaries.
Detect
Expose drift, unexpected impact, and boundary violations.
Respond
Stop, reverse, notify affected people, and repair the outcome.
Learn
Turn incident evidence into evaluation and design changes.
Misconception
“As models become more accurate, trust and governance are mostly solved.”
A correct-looking output may still be unauthorised, stale, or applied in the wrong context. Trust requires authority, traceability, and recovery alongside accuracy.
Test your understanding
When an AI workflow fails, who notices, who stops it, who repairs the outcome, and what evidence remains?
If one of those four questions has no answer, there is a governance gap.
Keep this
Trust is not a feeling or confidence score. It is the evidence-based capacity to prevent, detect, and recover within known boundaries.