Performance audits and the three Es

Accounting Ahead | Original study resource | accountingahead.com.au

Performance audits and the three Es

MODULE 5 | 5.5.1 | STUDENT COMPANION

A performance audit asks whether public money is being well spent. In Australia it is carried out mainly by

auditors-general, who report to parliament and never judge the merits of a policy. Economy concerns the

cost of inputs, efficiency the link between inputs and outputs, and effectiveness whether outputs achieve

the intended outcomes. Indicators measure each, but only mean something against a reference point.

Performance audits differ from financial statement audits in their subject matter, criteria, evidence and

reports.

Who carries out performance audits

Parliaments give governments money to carry out their policies, and departments and agencies do the work, so

parliament needs independent assurance about how well it is done. In Australia that comes from the

Auditor-General of each jurisdiction: the Auditor-General for Australia, supported by the Australian National Audit

Office under the Auditor-General Act 1997, and an Auditor-General in each state and territory, each reporting to

its own parliament. In the private sector, internal auditors and consulting firms do similar work, sometimes called

an operational audit.

The boundary. The auditor never judges whether a policy is the right one; that is for parliament and voters. It

examines how well the policy is being put into practice.

The three Es along the chain

Resources

Money and people

Inputs

What is bought or used

Outputs

What is delivered

Outcomes

The change achieved

Economy: spending less

Inputs of the right quality and

quantity, when they're needed, at the

lowest cost

Did councils pay a fair price for the

trees and the planting contracts?

Efficiency: spending well

As much output as possible from the

resources used, or as little as

possible for a given level of service

How many trees were planted and

established for each dollar? Were

watering crews organised sensibly?

Effectiveness: spending

wisely

Do the outputs achieve what the

program was set up to do?

Did the trees survive, did shade

increase, and did summer street

temperatures fall?

Cooler Suburbs (imagined): $40 million over four years for shade trees in the hottest suburbs

Value for money brings them together: do the outcomes achieved justify what was spent?

The Australian standard on performance engagements (ASAE 3500) describes all three as performance

principles, alongside others such as ethics. Only the effectiveness question tells you whether the money

did any good: a fair price and plenty of trees per dollar mean little if the shade never arrives.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | STUDENT COMPANION

1

Accounting Ahead | Original study resource | accountingahead.com.au

Performance indicators for Cooler Suburbs

Input

Measures resources

Dollars spent, contractor hours, the number of trees bought

Output

Measures what was delivered

Trees planted, streets covered

Efficiency

Relates inputs to outputs

Cost per tree planted, or better still, cost per tree still alive after two years

Outcome

Measures the change sought

Growth in tree canopy over the target suburbs, measured from aerial images; the fall in summer surface

temperatures

Quality and timeliness

Sit alongside the others

Share of trees planted in the right season

Compare with a reference point

A target, the program's earlier results, or a similar program elsewhere

Outcomes take time

And are affected by things the program can't control, such as a run of hot,

dry summers

Read indicators with care. Compare each with a reference point. Outcomes take years and are pushed around

by outside factors, so look at trends and corroborate them.

Can a program be efficient and still fail?

Yes. Trees planted cheaply in midsummer with no watering plan look efficient on the output figures, but if half die

within a year, the money spent on them is largely wasted. That is why auditors push past outputs to outcomes

wherever they can.

Performance audit and financial statement audit

Performance audit Financial statement audit

Subject matter An activity or program: is public money being well spent? A set of financial statements: are the numbers right?

Criteria

Often selected or developed by the auditor from legislation,

government rules, the program's objectives or better practice,

and agreed with the agency

A ready-made framework: the accounting standards

Evidence Often persuasive rather than conclusive, from many sources:

files, data, interviews, surveys and site visits

Mainly the entity's records, documents and confirmations,

gathered for each assertion

Report

Long and detailed: findings, conclusions and recommendations;

Commonwealth reports tabled in parliament include the

agency's comments

A standard-form opinion on the financial statements

Engagement

Usually direct: the auditor evaluates the activity itself. If the

agency reports on its own performance, that attestation

engagement falls under the umbrella standard (ASAE 3000)

Attestation: management prepares the statements, and the

auditor reports on them

Standards. The revised ASAE 3500 covers direct engagements; where the agency reports on its own

performance and the auditor concludes on that report, the engagement falls under the umbrella standard, ASAE

3000. At the Commonwealth level, the Auditor-General's own auditing standards build on the performance

engagement standard.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | STUDENT COMPANION

2

Accounting Ahead | Original study resource | accountingahead.com.au

Case note: the Community Sport Infrastructure audit

On 15 January 2020 the Australian National Audit Office tabled its audit of a $100 million Commonwealth

program of grants for community sport facilities, covering 684 projects, run by Sport Australia, with the Minister

for Sport approving the grants. It found that the funding decisions were not informed by an appropriate

assessment process and sound advice. The minister's office had run its own separate assessment; 61% of the

approved projects had scored below the agency's merit cut-off; and there was evidence that grants had favoured

electorates the government had targeted before the 2019 election. The legal authority the minister relied on to

approve the grants was not evident. After weeks of public debate, the minister resigned from the ministry on 2

February 2020, following a separate review that found a breach of ministerial standards, and on 5 February the

Senate set up a select committee to inquire into the grants. The lesson: how public money is allocated, not just

how much is spent, is a central question of effectiveness and accountability.

Source: ANAO Report No. 23 2019-20 (January 2020); Senate Select Committee on Administration of Sports Grants (2020).

Practice questions

MODULE 5 | 5.5.1 | TRY THESE BEFORE THE WORKED ANSWERS

Question 1 - Splash Ready's indicators

Splash Ready (imagined) is a state program that pays accredited swim schools to give primary school children

ten free lessons. Its aim is that every child finishing primary school can swim 50 metres and stay afloat for two

minutes (the state's water safety benchmark), with places offered first in suburbs where few children have had

lessons. Classify each indicator and link it to one of the three Es.

Indicator Input, output,

efficiency or

outcome?

Economy,

efficiency or

effectiveness?

(a) Number of lessons delivered

(b) Cost per child who completed the course

(c) Total funding paid to swim schools

(d) Share of children who couldn't swim at the start and met the

benchmark by the end

(e) Average fee paid per lesson, compared with the swim schools'

usual price

(f) Number of children who completed all ten lessons

(g) Average number of children per instructor in each class

(h) Share of children finishing primary school who meet the

benchmark, compared with before the program

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | STUDENT COMPANION

3

Accounting Ahead | Original study resource | accountingahead.com.au

Question 2 - How is Splash Ready performing?

Here are Splash Ready's first-year results. Assess the program's economy, efficiency and effectiveness, and say

what further evidence you would want.

Measure Target Year 1 result

Funding paid to swim schools $12.0 million $11.4 million

Average fee per lesson (the swim schools' usual price is $24) $21.00 $19.50

Children enrolled 60,000 60,000

Children completing all ten lessons 51,000 (85%) 38,000 (63%)

Cost per child completing the course $235 $300

Average children per instructor in each class 5 7

Non-swimmers meeting the benchmark at the end of the course 75% 58%

Places taken up in the priority suburbs At least 40% 18%

Question 3 - Which audit would answer it?

Members of an imagined state parliament raise four concerns.

Concern Financial statement audit or performance audit?

Why?

(a) Has the state housing authority recorded all the rent

its tenants owed at year end?

(b) Is the free trade-training program actually helping

graduates find jobs in the trades the state is short of?

(c) Did the arts department pay too much for its new

collection storage building, and manage the contract

well?

(d) Is the land held by the state's development agency

shown at the right value in its financial statements?

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | STUDENT COMPANION

4

Accounting Ahead | Original study resource | accountingahead.com.au

Worked answers

MODULE 5 | 5.5.1 | WORKED ANSWERS - ATTEMPT THE QUESTIONS FIRST

Question 1 - Splash Ready's indicators

Indicator Type Links to

(a) Lessons delivered Output Efficiency (the output side); says nothing yet about

results

(b) Cost per child completing Efficiency Efficiency

(c) Total funding paid Input Economy

(d) Non-swimmers reaching the benchmark Outcome Effectiveness

(e) Fee per lesson against the usual price Input Economy: the price paid for each input

(f) Children completing all lessons Output Efficiency (the output side)

(g) Children per instructor Efficiency Efficiency, though it also bears on quality and safety

(h) Benchmark rate at the end of primary

school, against before

Outcome Effectiveness: the program's real aim, best judged

over several years

Question 2 - Splash Ready's first year (sample answer)

Economy: good on price. Lessons cost $19.50 against a $21 target and a usual price of $24, and spending was

$0.6 million under budget. But classes averaged 7 children per instructor against 5, so some of the saving may

have come from lower quality.

Efficiency: weak. Only 63% of children finished the course, so the cost per child completing was $300 against

$235, about 28% higher. Money spent on children who dropped out bought little.

Effectiveness: weak. Only 58% of non-swimmers met the benchmark, against 75%, and only 18% of places went

to the priority suburbs, against at least 40%, so the program is missing many of the children it was meant for.

Overall: economical, but not yet efficient or effective.

Further evidence: why children dropped out (surveys of parents; lesson times, places and transport); who tests

the benchmark, and an independent check of a sample of results; whether bigger classes produced worse results,

swim school by swim school; why priority suburbs took up so few places (how enrolment works, access to pools);

a baseline and comparison with children outside the program; and benchmark rates at the end of primary school

over several years.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | WORKED ANSWERS

5

Accounting Ahead | Original study resource | accountingahead.com.au

Question 3 - Which audit would answer it?

Concern Answer

(a) Rent owed Financial statement audit. Whether all rent receivable is recorded is a question about the

completeness and accuracy of amounts in the statements.

(b) Trade-training jobs Performance audit. It asks whether the program achieves its outcome, which is

effectiveness. A financial statement audit would only confirm what was spent.

(c) Storage building Performance audit. Paying a fair price is economy, and contract management is a

management system. The statements would show the cost, not whether it was too much.

(d) Land values Financial statement audit. Valuation of an asset in the statements is an assertion the

auditor's opinion covers.

The pattern: questions about whether the numbers are right belong to the financial statement audit; questions

about whether money was well spent belong to the performance audit.

ADVANCED AUDIT AND ASSURANCE | M5 | 5.5.1 | WORKED ANSWERS

6