KPIs and Reporting to Leadership

A balanced scorecard

Report a handful of metrics that cover four angles. Value: share of tickets handled by AI (containment), time saved, adoption. Quality: accuracy or grounded-correct rate, escalation rate, customer satisfaction. Risk: incidents by severity, overdue reviews, fairness gaps, policy exceptions. Cost: spend against budget and cost per resolved task. Show trends and targets, and include bad news. A dashboard that only shows good numbers loses leadership trust.

Three KPIs, run

I ran this with example numbers: 6,000 of 10,000 tickets were handled by AI (containment 0.6); 600 of those were escalated (escalation rate 0.1); spend of 4,500 over 5,400 resolved tickets gives 0.83 per resolved ticket.

tickets, ai_handled, escalated, cost = 10_000, 6_000, 600, 4_500
print(round(ai_handled / tickets, 2),                 # containment
      round(escalated / ai_handled, 2),               # escalation rate
      round(cost / (ai_handled - escalated), 2))      # cost per resolved ticket

Output:

0.6 0.1 0.83

Pair every value metric with a guardrail metric

If you push containment up, also watch satisfaction and escalation quality. Optimising one number alone can quietly damage what actually matters.

Quick check: Why pair a value metric like containment with a quality metric like satisfaction?

• Containment is not measurable
• Satisfaction is always lower
• Raising one alone can hide damage to the thing that matters
• Pairs of metrics are required by law