Designing performance measures and targets

Begin with the decision and the strategic purpose

A useful performance measure helps someone judge progress or choose an action. Start with the organisation's purpose and the decisions needed to achieve it. Data already available in a spreadsheet is a possible input, not a sufficient reason to create another indicator. A concise set that directs attention to important decisions is preferable to a long report that nobody uses.

The fictional Lumen Tool Library lends equipment to local members. Its purpose is dependable access to suitable tools at an affordable cost. It needs to know whether reservations are ready as promised, whether issued equipment passes its checks, and whether members can use the service. Counting website visits alone would not answer those questions.

A measure specifies what is observed; a target states the desired result. Lumen's reservation-readiness rate is a measure. A requirement to achieve at least 94% during a named month is a target. The rate can be calculated even if no target has been set. Conversely, a target without a defined measure cannot be evaluated consistently.

Concept sources: ACCA, Performance indicators — linking indicators and objectives with strategic performance.

Write a definition that another person can apply

Before reporting a ratio, settle its population, timing and counting rules. Two teams can use the same label but report different things: one might count each reservation once, while another counts every tool within it. Document the agreed calculation and keep the underlying records available to authorised reviewers.

The manager also specifies a report date and correction procedure. If a late record changes the result, preserve the original report and explain the revision. A change in definition requires a dated note; otherwise an apparent improvement might simply reflect a changed calculation.

Concept sources: ANAO, Reporting Meaningful Performance Information — documented methods, data sources, responsibilities and evidence for reported results.

Calculate the result without hiding missing information

In February, Lumen has 1,000 reservation records. Forty are genuine member cancellations before the cut-off, leaving 960 eligible reservations. Of these, 864 were ready on time, 48 were ready late, 32 were incomplete and 16 have unresolved readiness records.

Confirmed readiness rate = 864 ÷ 960 × 100 = 90%. The report should also disclose the 16 unresolved records. They remain in the population but cannot yet be counted as confirmed successes. This is a conservative evidence rule, not a claim that all 16 were definitely late.

Dropping the unresolved records would produce 864 ÷ 944 = 91.5%, rounded, without changing any member's experience. Counting only collected reservations would create a different bias by excluding people who could not collect their tools. Ask whether each exclusion follows the definition rather than whether it makes the percentage look better.

If no eligible reservations exist in a reporting period, the rate is not applicable, not 0% or 100%. Show the numerator and denominator alongside the percentage so a reader can distinguish a small service from a large one.

Combine different kinds of evidence

Financial and non-financial measures answer different questions. Lumen needs a sustainable operating cost, but cost alone cannot establish that its tools are usable. Equally, excellent readiness would not establish that the service can afford to continue. Select complementary evidence and make the proposed connection between measures explicit.

An input such as staff hours, an activity such as checking equipment, an output such as a loan, and an outcome such as a member completing their task are different points in the service process. More activity does not automatically mean a better outcome.

Leading and lagging describe a measure's relationship to a particular result. Overdue maintenance may warn of future availability problems; last month's failed reservations record something already experienced. A non-financial measure can be lagging. A proposed leading indicator needs a credible explanation and testing, not merely an earlier reporting date.

Consider efficiency, effectiveness and equity together

Efficiency asks what resources are used to deliver output. Effectiveness asks whether the intended result is achieved. Equity asks how access, treatment and outcomes differ across relevant groups, taking their circumstances and needs into account. These perspectives can reveal trade-offs that a single overall ratio conceals.

Concept sources: Productivity Commission, Approach to performance measurement — the distinction between efficiency, effectiveness and equity, and the importance of context when interpreting service comparisons.

Suppose Lumen completes 900 loans using 450 staff hours: two loans per staff hour. Increasing that ratio by cutting checking time could undermine dependable service. It is therefore necessary to examine quality alongside throughput.

Within February's 960 reservations, 666 of 720 weekday reservations were ready on time, compared with 198 of 240 weekend reservations. The rates are 92.5% and 82.5%, a ten-percentage-point difference; together they still produce 90%. The overall figure obscures the weaker weekend experience.

This difference warrants investigation, not an immediate conclusion of unfair treatment. Weekend demand, equipment mix and staffing may differ. Ask whether opening arrangements disadvantage members with limited weekday availability. Equal treatment does not necessarily require identical resources where needs differ, and improving access may increase average cost.

Test whether the evidence deserves confidence

Validity concerns whether the measure captures the issue being evaluated. Counting reminders sent is a weak substitute for knowing whether members receive usable tools. Reliability concerns consistency of measurement: two trained reviewers applying the same rules to the same records should reach the same result. A consistently produced reminder count can still be an invalid measure of dependable access.

Test these qualities before attaching significance to small differences. Have two reviewers classify a sample of reservations independently, investigate disagreements and refine ambiguous rules. Reconcile totals to the source register. For surveys, inspect response rates and coverage rather than assuming that a precise percentage represents everyone.

Information must also arrive in time to affect a decision. Lumen's monthly readiness report supports resource planning, while a same-day list of missing tools supports immediate recovery. The latter is useful even before the monthly result is final. Label provisional information clearly.

Keep explanations understandable and give authorised users access to the supporting detail. Separate what a manager can influence from external conditions and shared responsibilities. A service manager can improve handover arrangements but cannot personally control every supplier disruption. The disruption remains relevant to the service result; it should be explained rather than deleted.

Turn an aspiration into a SMART target

SMART provides a practical check: make a target specific, measurable, achievable and agreed, relevant, and time-based. In the guide's treatment, timely reporting also matters because an expired opportunity cannot be recovered by a later report. Agreement should include resources and responsibilities, not merely a signed number.

“Make reservations much better” leaves both success and the deadline unclear. Lumen instead proposes: “Achieve at least 94% confirmed readiness for eligible June reservations, using the agreed definition, with weekly exception reviews and no relaxation of equipment checks.”

The percentage is specific and measurable; the service purpose makes it relevant; June fixes the period. Achievability still needs evidence. Lumen should test whether improved return reminders, spare equipment and revised staff coverage can close the gap from the 90% baseline. Calling a target SMART does not establish that the improvement plan is feasible.

Review interactions with other commitments. A higher readiness target that can be reached only by refusing difficult reservations might damage access. Changes in resources or demand may justify a new target, but record the reason and retain the earlier target for a transparent comparison. Do not rewrite it retrospectively merely to report success.

Compare the benefit of measurement with its cost

Include staff attention, data checking, systems and lost opportunities when assessing a measure's cost. A report earns its place when it informs a worthwhile action or fulfils a genuine accountability need. Review old reports as well as proposed additions; a measure can outlive the decision it once served.

That difference includes staff capacity released, not necessarily cash saved. Check whether the hours can be productively reassigned or paid overtime reduced. Include training, error correction and ongoing support before approval. Automation that produces an invalid measure faster has little decision value.

Self-check: evaluate a proposed improvement

Question: Lumen removes all unresolved reservations from the denominator and reports that readiness improved from 90% to 91.5%. The manager says the result proves the service has become more reliable. Is that conclusion supported?

Explained answer: No. The higher percentage results from an exclusion, not evidence of better service. Keep the agreed population, investigate the missing records and disclose their effect. A revised definition may sometimes be appropriate, but it needs justification and comparable restated data. Review readiness, quality, cost and access together before judging performance.

Sources and further reading

Study-guide scope: Module 5, designing performance measures and targets, printed pages 269–278. Lumen Tool Library and all its figures are fictional. These examples apply the guide's concepts to independent situations.

• ACCA, Performance indicators — strategic relevance and SMART objectives.
• ANAO, Reporting Meaningful Performance Information — evidence, methods and ownership.
• Productivity Commission, Approach to performance measurement — interpreting service performance across several dimensions.