Custom KPIs
A rubric grades on a 1-5 scale. A custom KPI is a yes/no question. They complement each other — rubrics measure quality, KPIs measure did this specific thing happen.
When to use a KPI vs a rubric criterion
| You want to know… | Use |
|---|---|
| How well the agent did on empathy | Rubric criterion (1-5) |
| Whether the agent quoted the legal disclosure | KPI (yes/no) |
| Whether the call ended with a clear next step | KPI (yes/no) |
| How efficient the resolution was | Rubric criterion |
| Whether sentiment improved by the end | KPI |
| Whether a script keyword appeared | KPI |
Types of KPIs
| Type | Example |
|---|---|
| Keyword presence | Transcript contains “your call may be recorded”. |
| Phrase contains | Agent acknowledged frustration within first 30s. |
| Numeric threshold | Sentiment score at end ≥ sentiment at start. |
| Tool used | Agent called transfer_to_human. |
| Custom (LLM-judged) | “Agent offered an alternative product.” |
Build a KPI
Open KPIs
Go to QA → Custom KPIs → New KPI.
Pick a type
For most cases, start with Custom (LLM-judged) — it’s the most flexible and you can always switch to a rule-based KPI later if precision matters.
Define the question
A single yes/no question:
“Did the agent reference the customer’s account by name within the first 60 seconds?”
Set scope
| Scope | When |
|---|---|
| All conversations | Default. Runs on every scorecard. |
| By channel | Voice-only or chat-only KPIs. |
| By agent group | KPI applies to a specific team. |
| By rubric tag | KPI applies only when a tag is present. |
Save and backfill
By default, new KPIs run on incoming conversations. Click Backfill to run it on the last N days too.
Example KPIs
Did the agent quote the cancellation policy verbatim?
Did the agent confirm the customer's preferred contact method?
Did the agent offer a self-service link before transferring?
Did the agent end the call with a clear next step?
Did the customer's tone shift from frustrated to satisfied?
Did the agent stay under 4 minutes for a refund?
Did the agent attempt at least one save before processing the cancellation?Reading KPI results
KPIs appear on every scorecard as green ✅ or red ❌ chips. Dashboards roll them up:
- Per-agent pass rate for each KPI.
- Per-team trends week-over-week.
- Outliers — KPIs that pass <50% of the time across the team often indicate a process problem, not a coaching problem.
Custom KPIs are cheap to add. Start with 2-3 high-signal ones rather than 20 low-signal ones — they’re easier to act on and less noise on dashboards.
Pitfalls
- Compound questions. “Did the agent acknowledge AND quote policy?” — split into two KPIs.
- Vague phrasing. “Did the agent do a good job?” — that’s a rubric criterion. KPIs need a clear yes/no answer.
- Stale KPIs. Review your KPI list quarterly — if a KPI passes 99% of the time, it’s no longer measuring anything useful.
Don’t tie compensation to KPI pass rates without a calibration period. LLM-judged KPIs drift over time and are noisy on small samples.
Open in Omniflow
Related
| If you want to… | Go to |
|---|---|
| Read scorecards | Scorecards |
| Tune the rubric | Scoring Rubric |
| Send KPI alerts | Notifications |