Conducting a performance audit

Accounting Ahead | Original study resource | accountingahead.com.au

Conducting a performance audit

MODULE 5 | 5.5.2 | STUDENT COMPANION

A performance audit has the elements of any assurance engagement, but its mandate comes from

legislation and its report is tabled in parliament. It runs through five stages: selection, planning,

fieldwork, reporting and follow-up. Criteria may be established or developed, but must be relevant,

complete, reliable, neutral and understandable, and are best agreed with the agency. Fieldwork looks at

the management systems behind performance, and findings explain the gap, its cause and what should

change.

The elements of a performance audit

Element In a performance audit

Intended users Parliament and, through it, the public

Responsible party The head of the agency that runs the activity

Practitioner The Auditor-General, whose mandate comes from legislation rather than an engagement

letter: the agency can't refuse the audit or narrow its scope, and independence doesn't

depend on fees the agency pays

Subject matter The activity or program

Criteria The benchmarks of good performance it is measured against

Result A report tabled in parliament

The five stages of a performance audit

1. Selection

Choose topics for the

annual work program:

money and people

involved, risk of poor

administration, likely

benefit, parliament's

interest, and whether it

can be audited

2. Planning

Study the program; set

the objective, criteria,

scope and methods;

discuss the criteria with

the agency

3. Fieldwork

Gather and analyse

evidence against each

criterion

4. Reporting

Draft findings,

conclusions and

recommendations; the

agency comments on a

proposed report

5. Follow-up

Parliamentary committees

may examine the report;

later audits check

whether

recommendations were

put into effect

Commonwealth audits

Any written comments the agency gives within 28 days of receiving the proposed report must be included in the final report tabled in parliament

The mandate comes from legislation, so the agency can't refuse the audit or narrow its scope

Reporting and follow-up at the Commonwealth level. Under section 19 of the Auditor-General Act 1997 the

agency receives a proposed report, and any written comments it gives within 28 days must be included in the

final report tabled in parliament. Parliament's Joint Committee of Public Accounts and Audit must then examine

every Auditor-General report, and it chooses some for a closer inquiry. Each state and territory's legislation sets

its own rules.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.2 | STUDENT COMPANION

1

Accounting Ahead | Original study resource | accountingahead.com.au

Cooler Suburbs: objective, criteria and evidence

Selected two years in: large, highly visible, and early reports suggest many trees have died

Audit objective

Is the department administering Cooler Suburbs effectively to increase shade in the suburbs most at risk from heat?

Criterion 1: designed to target

Was the program designed to target the

hottest and most vulnerable suburbs, with

clear objectives and measures?

Evidence

Program guidelines; the department's heat

vulnerability mapping; its funding decisions

Criterion 2: assessed and paid

properly

Were grants assessed and paid in line with the

guidelines?

Evidence

A sample of grant files; assessment scores;

payment records

Criterion 3: monitored and reported

Does the department monitor and report on

tree survival and shade?

Evidence

Council progress reports; survival counts from

site visits; aerial canopy data; the department's

annual report

Breaking the objective into a few high-level criteria, each with the evidence that will test it, keeps the fieldwork

focused and makes the eventual conclusion easy to follow.

Where criteria come from, and the tests they must pass

Established criteria

Legislation; government rules for grants and

procurement; the program's own guidelines

Developed by the auditor

From better practice guides, expert advice or

the results of similar programs; the auditor

explains why they're suitable

Either way, criteria must be:

Relevant to the audit objective

Complete enough to cover what matters

Reliable different auditors would reach similar conclusions

Neutral not biased

Understandable to parliament and the public

Agree them with the agency during planning, so the findings are hard to dispute when they come

Fieldwork: the systems behind performance

Management systems: governance and oversight; planning and risk management; selecting and paying

grant recipients; contract management; the performance information the agency collects and reports; and

how it uses that information to decide. Weaknesses here usually explain poor performance.

Techniques: examining files and documents, interviewing staff and stakeholders, analysing data, running

surveys, visiting sites, comparing results with benchmarks, and drawing on experts, such as an arborist for

Cooler Suburbs.

Corroboration: evidence is often persuasive rather than conclusive, so a finding should rest on more than

one source.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.2 | STUDENT COMPANION

2

Accounting Ahead | Original study resource | accountingahead.com.au

What the Cooler Suburbs team found

Finding Why it happened Recommendation

35% of the funding went to suburbs that

weren't on the department's own heat

vulnerability list

Grants were awarded first come, first

served

Target funding using the heat

vulnerability list

About 40% of the trees planted in the first

year have died, mostly where councils had

no watering contract

The guidelines never required

watering plans

Require a two-year watering and

maintenance plan before a grant

is paid

Councils report how many trees they plant,

but nobody measures how many survive

The guidelines never required

survival reporting, and staff only

checked that invoices matched

planting numbers

Make councils report survival and

canopy growth, not just planting

If the agency disagrees

The auditor must consider the agency's comments carefully and, if they bring new evidence, re-examine the

finding. But if the evidence still supports it, the finding stays, and the agency's disagreement is published

alongside it for parliament and the public to judge. That independence, protected by legislation, is what gives a

performance audit its authority.

Practice questions

MODULE 5 | 5.5.2 | TRY THESE BEFORE THE WORKED ANSWERS

Question 1 - An objective and criteria for Kellerton

The City of Kellerton (imagined) spends about $2.1 million a year on a graffiti removal service run by a

contractor. The contract requires offensive graffiti to be removed within 24 hours of a report and other graffiti

within five working days, on council property and, with the owner's consent, on private property visible from the

street. Residents report graffiti through the council's app or call centre, which creates a work order; the

contractor marks each order complete with a photo and is paid per square metre removed. Complaints have

risen for two years running, and the state's Auditor-General has chosen the service for a performance audit.

Write an audit objective and three high-level criteria.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.2 | STUDENT COMPANION

3

Accounting Ahead | Original study resource | accountingahead.com.au

Question 2 - Gathering the evidence

For each of your three criteria, choose the best evidence-gathering methods and explain why.

Your criterion Best evidence-gathering methods Why these methods?

1.

2.

3.

Question 3 - The council disagrees

The draft report on Kellerton includes this finding: 'The council does not effectively monitor its contractor. In

a sample of 120 reports of offensive graffiti, 37 (31%) were not removed within 24 hours, and the council

pays invoices without checking them against completed work orders.' The council's chief executive replies:

'We disagree. The contractor's monthly reports show 96% of offensive graffiti removed within 24 hours.

Your sample included the fortnight after the January storms, when we redirected the crews to storm

clean-up. And our coordinator approves every invoice.' Explain how the auditor should deal with this

response.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.2 | STUDENT COMPANION

4

Accounting Ahead | Original study resource | accountingahead.com.au

Worked answers

MODULE 5 | 5.5.2 | WORKED ANSWERS - ATTEMPT THE QUESTIONS FIRST

Question 1 - Kellerton's graffiti service (sample answer)

Objective: to assess whether the City of Kellerton is managing its graffiti removal service effectively, so that

graffiti is removed within the required times at a reasonable cost.

Criterion 1 - the service is well designed: clear service standards, payment terms and performance reporting in

the contract, and a contract awarded through a sound process at a competitive price.

Criterion 2 - the council monitors the contractor effectively: reliable data on removal times, invoices checked

against completed work, and action taken on poor performance.

Criterion 3 - the service achieves its intended results, and the council reports them: removal times met,

repeat graffiti at hotspots falling, and residents' complaints falling.

Question 2 - Gathering the evidence (suggested)

Criterion Methods Why

1. Well designed Examine the contract, the tender evaluation and

the service standards; compare rates per square

metre with other councils; interview the contract

manager

Documents are direct evidence of design;

benchmarks test whether the price is fair

2. Monitored

effectively

Analyse the council's own work-order data for

every report (time reported against time

completed); match a sample of invoices to work

orders, photos and measured areas; reinspect a

sample of sites

The council's system records are

independent of the contractor;

whole-population analysis beats a small

sample; site visits corroborate the photos

3. Achieving results Analyse trends in reports, repeat sites and

complaints; survey residents; compare with

targets and other councils; visit hotspots

Outcomes need several sources, because

each one alone is weak evidence

Question 3 - Dealing with the council's response (sample answer)

Consider it properly. The council is entitled to comment, and new evidence must be weighed. The test is what

the evidence shows, not whether the council agrees.

The 96% figure. The contractor's own reports are less reliable than the council's system records. Ask for the data

behind them and reconcile the two; they may measure from a different starting point, such as dispatch rather than

the resident's report.

The storm fortnight. Rerun the analysis without those two weeks, or over the whole year's work orders. If the late

removals cluster in that fortnight, revise the finding and explain the redirection; if the rate is still well above target,

the finding stands, with the context noted.

Invoice approval. Find out what the coordinator actually checks. If approval doesn't involve matching invoices to

completed work orders, the finding stands; if it does, review the evidence and correct the report.

Report. Keep the finding as far as the evidence supports it, word it fairly, and publish the council's response

alongside it, as audit legislation usually provides. Document how the response was considered.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.2 | WORKED ANSWERS

5