Trust, governance, and accountability in AI work
An agent sends a plausible but unauthorised commitment to a customer. The team can see what happened, but cannot say who granted the authority or who must repair the customer outcome. This is not merely an accuracy problem.
Justified trust
Operational trust means knowing what the system may do, tracing output to evidence, detecting boundary violations, recovering from them, and naming the owner accountable for the outcome.
Operational trust stack
A confident-looking output is not necessarily trustworthy; trust is built through boundaries, evidence, evaluation, observability, ownership, and recovery.
Confidence ≠ Trust
Six operational foundations of trust
User enthusiasm or a high model-confidence score is not sufficient evidence of trustworthiness.
Boundary
Permitted data, tools, users, and actions are explicit.
Provenance
Sources and transformations behind an output are traceable.
Evaluation
Quality and risk are tested on examples that represent real use.
Ownership
Roles accountable for policy, system, and customer outcome are named.
Incident evidence
Who acted, when, with which data and authority can be inspected.
Recovery
Override, rollback, customer repair, and recurrence prevention are defined.
Authority and trust case
Correct information, unauthorized commitment
A sales agent sends a factually correct two-year fixed-price commitment to a customer. The agent is not authorized to make contractual commitments.
Embed governance in the work
Prevent
Make unacceptable action difficult through authority and data boundaries.
Detect
Expose drift, unexpected impact, and boundary violations.
Respond
Stop, reverse, notify affected people, and repair the outcome.
Learn
Turn incident evidence into evaluation and design changes.
Misconception
“As models become more accurate, trust and governance are mostly solved.”
A correct-looking output may still be unauthorised, stale, or applied in the wrong context. Trust requires authority, traceability, and recovery alongside accuracy.
Take it to work
Beyond accuracy, which trust layer is easiest to break in one AI output?
Choose boundaries, evidence, evaluation, observability, ownership, or recovery, and name the signal that would monitor it.
Private reflection · not saved
Keep this
Trust is not a feeling or confidence score. It is the evidence-based capacity to prevent, detect, and recover within known boundaries.