Last updated: September 12, 2026
What a call scoring rubric is
A call scoring rubric is the set of criteria a reviewer scores a call against, plus the scale each criterion is measured on. Most criteria are binary, because the behavior either happened or it did not. Anchored scales earn their extra complexity only where a behavior has real degrees, and every criterion needs evidence a second reviewer can point at.
Every QA program eventually argues about what one line is worth. A supervisor wants the recording disclosure at 15 points, because a missed disclosure is what somebody pulls two years later. Her counterpart wants it at 3, since the line takes four seconds and the call that actually went wrong last week was the one where the callback number came out wrong. Neither will win, because the argument is about how much a behavior matters in the abstract.
What settles it is the cost of the failure. Ask who is holding the problem an hour after an agent skips the line. Both criteria come out heavy under that question, for different reasons, and the argument moves off taste.
Pick the scale before you write the criteria
The scale is the decision most programs make by accident. Somebody opens a template, sees 1 to 5 down the right hand column, and the criteria get written to fit it, which means they arrive vague, because a behavior with five levels has to be described loosely enough that five levels exist.
| Scale | What it measures well | Where it falls apart | An anchor written for it |
|---|---|---|---|
| Binary, yes or no | Behaviors with a findable moment: a line delivered, a number read back, a hold that ran past the account standard, a term off the do-not-say list | Behaviors with genuine degrees, where a yes hides the gap between adequate and good | Yes: read all of the callback number back digit by digit and the caller confirmed it. No: anything short of that |
| Three point anchored | Behaviors where the distance between adequate and good changes the outcome of the call, such as discovery depth | Anything a reviewer would have scored yes or no anyway, which now costs three written descriptions to maintain | 2: asked at least two probing questions before proposing an action. 1: asked one. 0: proposed an action with nothing asked |
| Five point anchored | Long calls where a behavior has five levels a reviewer could actually tell apart, with the time to place them | The middle. An unsure reviewer takes the 3, and twelve five point criteria produce a stack of 3s that describes nothing | 5 through 1 written out per criterion, which is sixty descriptions on a fifteen line card, and nobody maintains sixty descriptions |
The five point scale looks generous and behaves badly. An odd numbered scale hands every uncertain reviewer a safe midpoint, and uncertainty is the normal state of a supervisor scoring their ninth call before lunch. Once the middle fills up, every agent lands in the same band.
The clean version of that argument puts the whole card on binary and goes home. One or two criteria per rubric resist it. Discovery is the usual holdout: an agent who asked one question and an agent who asked four both score a yes, and those two need different coaching. That line earns three points of scale, and the other fifteen do not.
Writing a behavioral anchor
An anchor tells a reviewer what a level looks like, and it works when every level names something a second person can count or find in the recording.
"Confirmed the caller's details" becomes a binary line: read the callback number back digit by digit and got a confirmation. A reviewer points at the read back and whatever the caller said next.
"Showed urgency" becomes three levels. Two points when the agent named the next action and when it would happen, one when the action came with no time attached, zero when the call ended without one.
"Controlled the call" is the hardest of the three and still lands. Two points for bringing a wandering caller back to the required question inside two turns, one for a slower redirect, zero when the call ran to length with the question unanswered.
Any level description carrying a word like appropriately or properly has not been written yet. That is where a reviewer's judgment hides, and the agent who disagrees has nothing to argue with.
Allocating the points
Write one sentence per criterion describing what happens when it fails, naming who holds the problem afterward and what the fix costs. Sort those sentences by consequence, hand points down that order, and keep the total round, because that is what lets a score compare across months.
A criterion you cannot write a failure sentence for is not measuring anything. Take it off the card instead of parking it at 2 points.
Cap everything on an anchored scale at roughly a quarter of the total. That cap is our judgment rather than a published standard, and the arithmetic is plain: past a quarter, two reviewers off by one level on three lines swamp what the binary lines got right.
Say a Medicare agency starts from a general rubric with 25 points on communication quality and 10 on compliance. A missed TPMO disclaimer is a recorded call somebody can pull years later, and a Scope of Appointment collected after the plan discussion is a finding with a paper trail.
That moves compliance to 30 and communication quality to 10. The general 100 point scorecard we publish went the other way, from 20 down to 10 on compliance, because its critical fail list already carries the breaches. Same method, opposite answer.
How many criteria a card can carry
Twelve to eighteen is where most published rubrics sit, ours included, and the ceiling is a reviewer's attention rather than a rule. A supervisor scoring a 45 second answering service call against 30 criteria reads the first ten and pattern matches the rest, which shows up as identical scores on the bottom third of the card.
Checking yours is cheap. Pull the last fifty scored calls and look for criteria that never vary, since those lines are decoration.
Auto fails sit outside the arithmetic
A weight says one failure is worse than the others on the card. Critical fail means the call is void whatever else happened, which makes it a flag rather than a number, and it belongs outside the point arithmetic.
The test is whether the call could still be called good with everything else perfect. A lower score covers it when the answer is yes. When the honest answer is that the call should never have been logged that way, the line is a critical fail.
Scoring a breach as a 15 point deduction and a zero at once is the usual mistake, since the zero already said it.
List length is its own health check. Five items and a critical fail reads as an emergency to the floor. Past fifteen it mostly records who won an argument in a rubric meeting.
Download the blank builder
The builder is a blank card with four reference rows at the top, one per scale decision.
| Section | Criterion | Scale | Points | Critical fail | Evidence the reviewer points at |
|---|---|---|---|---|---|
| Opening | Delivered the required recording disclosure in full before any other business | binary | 5 | no | The disclosure text matched line by line with its timestamp |
| Information capture | Read the callback number back to the caller digit by digit | binary | 8 | yes | The read back and the caller's confirmation |
| Discovery | Asked probing questions before proposing an action | three point anchored | 6 | no | The count of questions asked ahead of the first proposal |
| Resolution | Followed the on-call schedule in force at that time of day | binary | 7 | no | The dispatch decision against the schedule for that hour |
Download call-scoring-rubric-builder.csv
That file carries the four rows above plus blank ones underneath, ungated. Fill the evidence column before the points column.
Two reviewers, one call
Before a rubric goes live, take ten calls and have two reviewers score them separately with no conversation in between. An afternoon of that tells you which criteria you actually wrote.
Read the per criterion disagreement rather than the gap between the two totals. Two reviewers can land a point apart while disagreeing in opposite directions on four lines, and the total hides all of it.
A criterion that splits your reviewers every round needs its anchor rewritten so each level names something countable. When that fails twice, two criteria were probably hiding inside one, or the line belongs in coaching notes and off the card.
As of September 2026 we could not find a published threshold for acceptable variance worth quoting, so set the target from your own first three rounds. Session mechanics sit on running a calibration session.
Where machine scoring fits in a rubric
Binary criteria with a named phrase or a countable event automate cleanly. A disclosure matched against the required text, a callback number compared against the record, a term off the do-not-say list. Those are close to arithmetic, and a machine holds steadier on them than two supervisors at the end of a Friday.
Anchored tone criteria are a different story, and automated scoring is measurably less reliable on subjective tone than on rule based criteria, which is what Aircall and StackAI both report. We would rather put that on our own site than have a buyer find it in month two.
That is a design constraint before it is a tooling question. Put 40 points on anchored tone criteria and no product will score the card for you, which quietly commits your supervisors to a manual pass forever.
Any criterion turning on who said something also depends on speaker separation holding up, and recordings that put both sides on a single channel make that harder, so write those lines to let a reviewer confirm the speaker from context. CallSniper scores against the transcript for that reason, stamping a timestamp and a confidence score on every line so a supervisor spends the pass on the low confidence ones (how scoring works in the product).
Start from a rubric somebody already argued about
A blank builder suits a program defending a weighting decision. Starting from nothing, edit a finished card instead.
For a general inbound contact center, the finished 100 point scorecard runs 17 criteria with five critical fails. A floor answering calls under client account scripts in under a minute wants the answering service version, which weights capture and message accuracy much harder. Once the rubric holds, building the file in a spreadsheet covers the formulas and the tabs, and anyone arriving cold should start with what call center QA covers.
When to change a rubric and when to leave it alone
Review the card on a schedule and change it on evidence. Once a quarter suits most programs, and once a year is plenty where the call mix has not moved.
Four things justify a change. A criterion that never varies across fifty scored calls has stopped measuring. Reviewers still splitting on a line after its anchor was rewritten means the criterion needs splitting or cutting. New obligations on the call, such as a disclosure requirement that changed, need a line. The clearest signal is a failure that hurt a client and has no line on the card.
An agent's average coming in lower than a manager expected justifies a coaching conversation and nothing else.
Then the trap. Change a weight in the middle of a quarter and every score written before it stops comparing to every score after, while the dashboard keeps drawing one line through both. That is worse than having no chart, because a supervisor walks into a one-on-one and reads a 6 point reweighting as an agent's improvement.
What to do instead is unglamorous. Version the rubric, v1.3 with the date it took effect, and stamp that version on every scored call. Let changes land on a period boundary and start the trend line there, with a change log carrying what moved and why. When you need the comparison across the boundary, rescore a sample of the old period under the new card, because the numbers mostly do not travel on their own.
If arguing a weighting in a meeting has stopped being productive, we can run your draft rubric over six months of your own recordings and show you which criteria never moved, for a fixed pilot price.
Frequently asked questions
What is a call scoring rubric?
A rubric is the criteria a reviewer scores a call against together with the scale each one uses. One binary criterion might ask whether the required disclosure was delivered in full, while discovery depth gets three levels, written out so two reviewers read them the same way.
How do you score a call out of 100?
Check the critical fail list first, because one flag voids the card regardless of points. If the call is clean, divide the points earned by the points available, which is the reason to keep the total at 100.
What is the difference between a rubric and a scorecard?
The rubric is the design: which criteria exist, what scale each uses, and what each is worth. A scorecard is the instrument a reviewer fills in for one call, and the 100 point scorecard is a worked example.
How many points should each criterion be worth?
Weight each line by what its failure costs, sorted by who holds the problem afterward. Hold anchored criteria to about a quarter of the total, which is our judgment rather than a standard.
Should QA scores use a 1 to 5 scale?
For behaviors that either happened or did not, a five point scale adds a middle for unsure reviewers to sit in and gives nothing back. Anchored scales earn their place on the one or two criteria where adequate and good genuinely differ.
Sources
- Aircall and StackAI on automated scoring holding up better on rule based criteria than on tone. Third party, unverified.
- No published threshold for acceptable inter rater variance. Our own research pass, 2026-09-08.
- Scorecards with points per criterion, critical fail flags, transcript timestamps, and per line confidence scores. Trinsic feature sheet, verified internally.
- The quarter of the total cap, and the twelve to eighteen criteria range. Editorial judgment.