AI Risk Assessment and Model Cards
Assessments that end in a decision — proceed, proceed with conditions, or stop — and model documentation that states where a system fails as clearly as where it works.
A risk register with forty entries, each rated amber, is not a risk assessment. It is a record that someone looked. An assessment that is worth the time ends in named decisions, with conditions attached and an owner for each, and it names the things the organisation will not do.
AI risk assessment identifies what could go wrong with a specific AI system, how likely and how consequential each failure is, who bears the harm, and what will be done about it. Model cards are the accompanying documentation: intended use, performance by subgroup, known limitations, and the conditions under which the system should not be relied on.
Assessing the harms that matter, to whom
Organisational risk assessment asks what could harm the organisation. AI risk assessment has to also ask what could harm the person on the other end, because that is what regulation, and eventually reputation, turns on:
| Dimension | The question | Commonly missed |
|---|---|---|
| Who is affected | Customers, employees, applicants, the public | Non-users affected by a decision they never saw |
| Severity | How bad is the worst realistic outcome for them? | Cumulative harm from many small errors |
| Distribution | Does it fall unevenly across groups? | Aggregate performance hiding a failing subgroup |
| Recourse | Can an affected person contest the outcome? | No route to challenge, or one nobody can find |
| Reversibility | Can the harm be undone? | Decisions that propagate into other systems |
| Detection | Would we notice it happening? | Silent failure with no feedback loop |
The last row governs everything above it. A severe risk you would detect within a day is a different proposition from a moderate one that could run unnoticed for a year, and most AI failure is the second kind.
Model cards that are worth reading
Intended use, and explicit out-of-scope use
What the system is for and what it must not be used for. The second half is what prevents the most common real-world failure: a model built for one purpose quietly repurposed for another by a team that never saw the evaluation.
Performance by subgroup, not in aggregate
A single accuracy figure conceals the segments where the system fails. Performance is reported across the groups that matter commercially and legally, with sample sizes shown so a reader can judge the confidence.
Known limitations, stated plainly
Where it degrades, which inputs it handles badly, what it was never tested on. A card that lists no limitations tells a reader that nobody looked, and it is the single clearest signal of immature documentation.
Data provenance and its constraints
What the system learned from, over what period, with what known gaps and licensing constraints. Much model risk originates in the data, and this is where a reader can see it.
Human oversight, described as it actually works
Who reviews what, on what evidence, with what authority to override. 'A human is in the loop' is not a description; if the reviewer sees a score with no context and a queue of four hundred, that is what the card should say.
Write for the reader you do not control
A model card should be usable by an auditor, an enterprise customer's risk team, a regulator, and the engineer who inherits the system in two years. Cards written as internal notes fail all four. We write them to be handed over, which also has the useful effect of making evasive claims uncomfortable to write.
From assessment to decision
- Scope the system precisely. What it does, what it decides, what happens downstream of its output.
- Identify harms with the affected functions in the room — not only engineering, because engineering will not think of every group affected.
- Assess severity, distribution, recourse and detectability rather than a single likelihood-times-impact score, which averages away the interesting cases.
- Decide. Proceed, proceed with conditions, or do not proceed — each with a named owner and, where conditional, the conditions written down.
- Document as a model card that the system's future readers can use.
- Set the review trigger. What change or event requires reassessment, so the document does not silently expire.
How the engagement runs
Assessments end in a named decision with an owner, not in a rating.
Scoping and evidence
Systems in scope defined; evaluation results, training data documentation and oversight arrangements gathered.
Assessment workshops
Harms identified with engineering and the affected business functions; severity, distribution, recourse and detectability assessed.
Measurement gaps
Where claims cannot be evidenced, the testing needed is specified and, where feasible, run.
Decisions and model cards
Decisions recorded with owners and conditions; model cards written for external readers.
Handover
Template, method and review triggers handed over so your team can assess the next system.
What you receive
Decisions with owners, and documentation an outside reader can rely on.
Risk assessments
Per system, covering affected parties, severity, distribution, recourse, reversibility and detectability.
Decision record
Proceed, proceed with conditions or stop, with a named owner and the conditions written down.
Model cards
Intended and out-of-scope use, subgroup performance, limitations, data provenance and real oversight arrangements.
Measurement gap list
Claims that cannot currently be evidenced, and the testing required to close each.
Review triggers
The changes and events that require reassessment, so documentation does not silently expire.
Method and templates
So your team assesses the next system without us.
Is this the right engagement?
Worth being direct. AI Risk Assessment and Model Cards is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- A system is about to be deployed and nobody has assessed what it could do wrong.
- Documentation is needed for an auditor, a customer or a regulator.
- An existing risk register lists AI systems with no decisions attached.
- Model cards are required by a framework you are aligning to.
- A system affects individuals and there is no articulated route to recourse.
Choose something else if
- The requirement is bias measurement specifically. See bias and fairness audit.
- The requirement is adversarial testing. See AI red teaming.
- You want documentation produced without engagement from the people who own the systems.
- No decision will follow the assessment, in which case it is paperwork.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
What should an AI risk assessment actually produce?
A decision, with a named owner. Proceed, proceed with stated conditions, or do not proceed. A register of amber ratings with no decisions attached records that someone looked; it does not manage anything, and it will not satisfy an auditor who asks what happened as a result.
What is a model card?
Documentation covering a system's intended use and explicit out-of-scope use, performance broken down by subgroup rather than in aggregate, known limitations, data provenance, and the human oversight arrangements as they actually operate. It is written for readers you do not control — auditors, customers, regulators, and the engineer who inherits it.
How is this different from our existing risk process?
Enterprise risk asks what could harm the organisation. AI risk assessment has to also ask what could harm the person affected by the system, whether harm falls unevenly across groups, whether that person has any route to contest the outcome, and whether anyone would notice the failure. Those questions rarely appear in an operational risk template.
How often should assessments be repeated?
On triggers rather than on a calendar: a material change to the model, its data, its purpose or its user population, plus a periodic floor so nothing is left indefinitely. Calendar-only review is late when something changes and wasteful when nothing has.
Who should be involved?
Engineering, the business owner, and representatives of whichever functions deal with the people affected — support, HR, legal, clinical, whatever applies. Engineering-only assessments reliably miss whole categories of affected party, because the people who built the system are not the people who field the complaints.
Often paired with this
Most clients combine two or three engagements from the AI Governance, Security and Compliance pillar. These are the ones that most often run immediately before or after.
Responsible AI and Bias and Fairness Audit
The fairness definition chosen explicitly, disparities measured on real subgroups, remediation tested not asserted.
Read more →AI Governance Framework Design
Proportionate risk tiers, named accountability and decision rights — a framework people follow rather than route around.
Read more →AI Inventory and Model Registry
A complete inventory including shadow and vendor AI, kept current by discovery rather than by annual survey.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.