Audit and Risk Committee (ARC)
Subject: Compensation Quality Assurance Program 2019-20
Item:
and update on Quality Online (QOL) Pilot
Endorsement
Decision X Discussion
X Information
Discussion led by: Luke Brown, Assistant Secretary Business Improvement
& Quality Assurance
Paper cleared by: Natasha Cole, First Assistant Secretary Client Benefits
Division
Recommendation:
The Committee note that commencing in the 2019-20 financial year, the QA Program will
include:
1. a revised approach to counting Work Element errors
s 22 - Out of scope
The Committee note the update on the Quality Online (QOL) Pilot.
Background/Issues:
The key purpose for the Compensation Quality Assurance Program (
QA Program) is to
ensure that clients receive the correct decision, the first time. The QA program creates
reports that provide assurance data for Divisional performance reports, financial reporting
and the DVA Annual Report. The reports are also used to inform decisions within the
Compensation business and identify business improvement opportunities.
It is important that the QA Program reporting is accurate, informative and structured to
present information in an accessible format for managers and decision makers. To this end
the QA Program reporting has been refreshed to include both qualitative and quantitative
data, and focus on trend identification and analysis.
The Quarterly Reports (
Attachment A: Compensation QA Quarterly Report 2018-19 Q4)
have been expanded in early 2019 to include sections on Issues and Trends and Error
Causation.
Entering into the 2019-20 financial year consideration has been given to amending the
approach to counting errors and how they are classified, to ensure a consistent approach to
the year’s reporting.
Page 1 of 4
Key Points/Status Update:
Work Element Errors
A work element is a discrete activity performed in the process of determining or finalising
a claim. The number of work elements in a case depends on a type of case and vary
according to the number of items determined (e.g. conditions determined).
The current approach to work elements established in 2006, is where for example a case
is split into five work elements and three of the elements have errors, it is counted as 1/5
error in the final statistics. With this approach there is no change in the correctness rate if
there are three elements with errors or five elements with errors.
The proposed approach, commencing in 2019-20 financial year, is that the elements with
errors are accounted for in the correctness rate. So in the same case outlined above, if
there were errors in three elements, it is counted as 3/5 error in the final statistics. This
approach increases the accuracy of in calculating error rates and although there is a risk
that there will be a decrease in the reported correctness rates, there should only be a
slight fluctuation. The proposed methodology was applied to Quarter 3 & 4, 2018-19
results to demonstrate the fluctuations caused by the change in approach (
Attachment B:
Work Element Calculation Comparison).
s 22 - Out of scope
s 22 - Out of scope
s 22 - Out of scope
Risks and Opportunities
The risks associated with the proposed approach to counting Work Element errors, is that
there may be a fluctuation with the error rates, which poses a reputational risk. The
possible fluctuation was demonstrated in the comparison of 2018-19 Quarter 3 results in
Attachment B, which indicates that the fluctuations are not substantial and have a small
impact on the publically reported HIEs.
These risks are offset against the benefit of more accurate reporting and the consistency
across the Client Benefits Divisions Quality Assurance programs. Additionally the context
provided by identifying financial and non-financial errors, will ensure that more informed
decisions can be made from the QA Program reports.
The reductions in the IPS sample size will not impact on the assurances provided to you
and the Chief Financial Officer, in relation to accuracy of payments made to a client, in the
process of signing off the end of year financial statements and meeting related ANAO
requirements.
Financial Implications
Nil
Consultation
The following stakeholders were consulted during the preparation of this paper:
Income Support Governance and Training Section
Internal Audit
ANAO
s 22 - Out of scope
Attachment A -
Compensation QA Quarterly Report 2018-19 Q4
Attachment B -
Work Element Calculation Comparison
Attachment C - Administrative Errors, LIE and HIE definitions
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
Chair’s Notes
Compensation Quality Assurance Reporting 2019-20
Speaker:
Natasha Cole, First Assistant Secretary - Client Benefits Division will address this item.
Talking points are provided for you to introduce this item.
Observers:
Luke Brown, Assistant Secretary – Business Improvement & Quality Assurance
Outcome Sought:
That members:
1.
AGREE to the proposed approach to counting Work Elements errors
s 22 - Out of scope
5.
AGREE these changes are to come into effect for the 2019-20 financial year.
Chair talking points:
The Compensation Quality Assurance Program (QA Program) samples compensation
cases from Client Benefits Division and assess the correctness and quality of decision
making.
The QA Program is functioning in accordance with external auditing and financial
reporting requirements.
It is recognised that error rates for the majority of case types in the compensation
business have exceeded portfolio performance indicators for a number of years, but
there are strategies in place to reduce the error rates and improve client experience.
It is essential that the QA program reporting is accurate, informative and structured
to present information in a format that is accessible and useful for manager and
decision makers.
This item seeks agreement to four changes in the approach to QA Program reporting
to come into effect in the 2019-20 financial year.
Page 1 of 2
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
IF ASKED (NOT to be raised)
Nil
Page 2 of 2
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
Compensation Quality Assurance Reporting 2019-20
Led by: Natasha Cole
Recommendations:
That members:
1.
AGREE to the proposed approach to counting Work Elements errors
s 22 - Out of scope
5.
AGREE these changes are to come into effect for the 2019-20 financial year.
Purpose:
The purpose of this paper is seek your agreement for the 2019-20 Compensation Quality
Assurance reporting to include:
a revised approach to counting Work Elements errors;
s 22 - Out of scope
Issues for discussion:
The key purpose for the Compensation Quality Assurance Program (
QA Program) is to
ensure that clients receive the correct decision, the first time. The QA program creates
reports that provide assurance data for Divisional performance reports, financial reporting
and the DVA Annual Report. The reports are also used to inform decisions within the
Compensation business and identify business improvement opportunities.
It is important that the QA Program reporting is accurate, informative and structured to
present information in an accessible format for managers and decision makers. To this end
the QA Program reporting has been refreshed to include both qualitative and quantitative
data, and focus on trend identification and analysis.
The Quarterly Reports (
Attachment A: Compensation QA Quarterly Report 2018-19 Q4)
have been expanded in early 2019 to include sections on Issues and Trends and Error
Causation.
Page 1 of 5
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
Entering into the 2019-20 financial year consideration has been given to amending the
approach to counting errors and how they are classified, to ensure a consistent approach to
the year’s reporting.
Work Element Errors
A work element is a discrete activity performed in the process of determining or finalising a
claim. The number of work elements in a case depends on a type of case and vary according
to the number of items determined (e.g. conditions determined).
The current approach to work elements established in 2006, is where for example a case is
split into five work elements and three of the elements have errors, it is counted as 1/5
error in the final statistics. With this approach there is no change in the correctness rate if
there are three elements with errors or five elements with errors.
The proposed approach, commencing in 2019-20 financial year, is that the elements with
errors are accounted for in the correctness rate. So in the same case outlined above, if there
were errors in three elements, it is counted as 3/5 error in the final statistics. This approach
increases the accuracy of in calculating error rates and although there is a risk that there will
be a decrease in the reported correctness rates, there should only be a slight fluctuation.
The proposed methodology was applied to Quarter 3 & 4, 2018-19 results to demonstrate
the fluctuations caused by the change in approach (
Attachment B: Work Element
Calculation Comparison).
s 22 - Out of scope
Page 2 of 5
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
s 22 - Out of scope
Page 3 of 5
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
s 22 - Out of scope
Risks and Opportunities:
The risks associated with the proposed approach to counting Work Element errors, is that
there may be a fluctuation with the error rates, which poses a reputational risk. The possible
fluctuation was demonstrated in the comparison of 2018-19 Quarter 3 results in
Attachment B, which indicates that the fluctuations are not substantial and have a small
impact on the publically reported HIEs.
These risks are offset against the benefit of more accurate reporting and the consistency
across the Client Benefits Divisions Quality Assurance programs. Additionally the context
provided by identifying financial and non-financial errors, will ensure that more informed
decisions can be made from the QA Program reports.
s 22 - Out of scope
Financial Implications: Nil
Page 4 of 5
OFFICIAL: Sensitive
OFFICIAL: Sensitive
EXECUTIVE MANAGEMENT BOARD
Tuesday, 17 December 2019
Consultation:
The following internal bodies and business areas were consulted during the preparation of
this paper:
Audit and Risk Committee
Client Services Committee
Income Support Governance and Training Section
The following external bodies were consulted during the preparation of this paper:
ANAO
Next Steps and Critical Dates:
The revised approach will be used in the first Quarterly Report of the 2019-20 year, due for
distribution during December 2019.
Attachments:
A. Compensation QA Quarterly Report 2018-19 Q4
B. Work Element Calculation Comparison
C. Administrative Errors, LIE and HIE definitions
Page 5 of 5
OFFICIAL: Sensitive
OFFICIAL: Sensitive
Client Services Committee
Monday, 28 October 2019
COMPENSATION QUALITY ASSURANCE REPORTING 2019-20
Led by: Luke Brown, Assistant Secretary Business Improvement and Quality Assurance
Recommendations:
That members:
1.
AGREE to the proposed approach to counting Work Elements errors
s 22 - Out of scope
4.
AGREE these changes are to come into effect for the 2019-20 financial year.
Purpose:
The purpose of this paper is seek your agreement for the 2019-20 Compensation Quality
Assurance (QA) reporting to include:
a revised approach to counting Work Elements errors;
s 22 - Out of scope
Issues for discussion:
The key purpose for the Compensation Quality Assurance Program (QA Program) is to
ensure that clients receive the correct decision, the first time.
The QA program creates reports that provide assurance data for Divisional performance
reports, financial reporting and the DVA Annual Report. The reports are also used to inform
decisions within the Compensation business and identify business improvement
opportunities.
It is important that the QA Program reporting is accurate, informative and structured to
present information in an accessible format for managers and decision makers. To this end
the QA Program reporting has been refreshed to include both qualitative and quantitative
data, and focus on trend identification and analysis.
The Quarterly Reports (
Attachment A: Compensation QA Quarterly Report 2018-19 Q4)
have been expanded in early 2019 to include sections on Issues and Trends and Error
Causation. Entering into the 2019-20 financial year consideration is required to amending
the approach to counting errors and how they are classified, to ensure a consistent
approach to the year’s reporting.
Page 1 of 3
OFFICIAL: Sensitive
OFFICIAL: Sensitive
Client Services Committee
Monday, 28 October 2019
Work Element Errors
A work element is a discrete activity performed in the process of determining or finalising a
claim. The number of work elements in a case depends on a type of case and vary according
to the number of items determined (e.g. conditions determined).
The current approach to work elements established in 2006, is where for example a case is
split into five work elements and three of the elements have errors, it is counted as 1/5
error in the final statistics. With this approach there is no change in the correctness rate if
there are three elements with errors or five elements with errors.
The proposed approach is that the elements with errors are accounted for in the
correctness rate. So in the same case outlined above, if there were errors in three elements,
it is counted as 3/5 error in the final statistics. This approach increases the accuracy of in
calculating error rates and although there is a risk that there will be a decrease in the
reported correctness rates, there should only be a slight fluctuation. The proposed
methodology was applied to the 2018-19 Quarter 3 results to demonstrate the fluctuations
caused by the change in approach (
Attachment B: Comparison Data Q3 2018-19).
s 22 - Out of scope
Page 2 of 3
OFFICIAL: Sensitive
OFFICIAL: Sensitive
Client Services Committee
Monday, 28 October 2019
s 22 - Out of scope
Risks and Opportunities:
The risks associated with the proposed approach to counting Work Element errors, is that
there may be a fluctuation with the error rates, which poses a reputational risk. The possible
fluctuation was demonstrated in the comparison of 2018-19 Quarter 3 results in
Attachment B, which indicates that the fluctuations are not substantial and have a small
impact on the publically reported HIEs.
These risks are offset against the benefit of more accurate reporting and the consistency
across the Client Benefits Divisions Quality Assurance programs. Additionally the context
provided by identifying financial and non-financial errors, will ensure that more informed
decisions can be made from the QA Program reports.
Financial Implications: Nil
Consultation:
The following internal bodies and business areas were consulted during the preparation of
this paper:
Income Support Governance and Training Section
Internal Audit
The following external bodies were consulted during the preparation of this paper:
Nil
Next Steps and Critical Dates:
With the Client Service Committee’s agreement, the revised approach will be used in the
first Quarterly Report of the 2019-20 year, due for distribution at the end of October 2019.
Attachments:
A. Compensation QA Quarterly Report 2018-19 Q4
B. Comparison Data Q3 2018-19
s 22 - Out of scope
Page 3 of 3
OFFICIAL: Sensitive
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
3.1 Quality Assurance Reporting 2019-2020
Members were provided with an update on the Compensation Quality Assurance Program (QA Program).
It is essential that the QA program reporting is accurate, informative and structured to present
information in a format that is accessible and useful for manager and decision makers. The errors are
classified into work element errors, Low Impact Errors including administrative errors and High Impact
Errors. s 22 - Out of scope
There was strong support from
the members and a strategic discussion be held on the reporting at the January EMB.
2 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
4 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
5 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
6 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
7 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
8 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
9 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
10 of 12
Executive Management Board
Tuesday, 17 December 2019
9:00 – 11:00am (AEDT)
ACT LEVEL 6, ROLFE ROOM
s 22 - Out of scope
11 of 12
Client Services Committee
Monday, 28 October 2019
11:30am – 12:45pm (AEDT)
Millen Room, ACT Level 8
s 22 - Out of scope
Page 3 of 3
Client Services Committee
Thursday, 5 December 2019
10:30am – 12:30pm (AEDT)
Saunders Room, ACT Level 6
s 22 - Out of scope
Page 4 of 4
From:
, Anne
s 47E, s 47F
on behalf of DEPUTY.COMMISSIONER.TAS.COORD
To:
s 47E, s 47F , Sally
Subject:
FW: For Action: Budget Estimates Questions on Notice & review of allocations [SEC=OFFICIAL]
Date:
Monday, 17 January 2022 10:42:09 AM
Attachments:
image001.jpg
image002.jpg
Anne V s 47E, s 47F
Executive Officer to Luke Brown
Deputy Commissioner Tasmania
Assistant Secretary, Business Improvement & Quality Assurance
Client-Focused/Responsive/Connected
Lest we forget
Ph when WFH s 47E, s 47F or s 47E
Please do not print this email unless absolutely necessary
From: s 47E, s 47F, Alexander <Alexanders 47E, s xxx@xxx.xxx.xx>
Sent: Wednesday, 11 November 2020 4:22 PM
To: s 47E, s 47F, Lana <Lana.s 47E, s xxx@xxx.xxx.xx>; DEPUTY.COMMISSIONER.TAS.COORD
<xxxxxx.xxxxxxxxxxxx.xxx.xxxxx@xxx.xxx.xx>; CLIENTS.BENEFITS.DIV.COORD
<xxxxxxx.xxxxxxxx.xxx.xxxxx@xxx.xxx.xx>
Subject: RE: For Action: Budget Estimates Questions on Notice & review of allocations
[SEC=OFFICIAL]
Hi All,
Apologies for the delay. The Committee have advised that the list of QoNs I sent them (which
did
not include the below) is complete.
NFA required on the below.
Kind regards,
Alexander s 47E, s 47F
Senior Parliamentary Officer
Ministerial and Parliamentary Support Section
Parliamentary & Governance Branch
Department of Veterans’ Affairs
e:
m: s 47E, s 47F
s 47E
s 47E
p: s 47E
From: s 47E, s 47F, Lana <Lana s 47E, s xxx@xxx.xxx.xx>
Sent: Monday, 9 November 2020 10:46 AM
To: DEPUTY.COMMISSIONER.TAS.COORD <xxxxxx.xxxxxxxxxxxx.xxx.xxxxx@xxx.xxx.xx>;
s 47E, s 47F Alexander <Alexanders 47E, s xxx@xxx.xxx.xx>; CLIENTS.BENEFITS.DIV.COORD
<xxxxxxx.xxxxxxxx.xxx.xxxxx@xxx.xxx.xx>
Subject: RE: For Action: Budget Estimates Questions on Notice & review of allocations
[SEC=OFFICIAL]
Anne
Natalie will chase these up and get back to you.
Regards
Lana s 47E, s 47F
Executive Officer to
Natasha Cole
First Assistant Secretary
Clients’ Benefits Division
Australian Government Department of Veterans’ Affairs
Gnabra House, 21 Genge Street Canberra ACT 2601
Phone s 47E
Mobile: s 47E, s 47F
s 47
Email: lana
@dva.gov.au
s 47E, s 47F
Part-time days Monday to Thursday
From: s 47E, s 47F Anne <Annes 47E, s xxx@xxx.xxx.xx>
On Behalf Of
DEPUTY.COMMISSIONER.TAS.COORD
Sent: Monday, 9 November 2020 9:49 AM
To: s 47E, s 47F Alexander <Alexander s 47E, s xxx@xxx.xxx.xx>; s 47F, s 47E Lana <Lanas 47E, s xxx@xxx.xxx.xx>
Subject: FW: For Action: Budget Estimates Questions on Notice & review of allocations
[SEC=OFFICIAL]
Importance: High
Morning Lana and Alexander, We are becoming a bit anxious about getting the QoN we are
expecting for Quality Assurance as we are not sure of the actual scope and cannot pull data until
we know that. It will take at least to working days to pull the data and rework it once we know
what Sen. Ayers has actually requested.
Please let me know how to proceed given how short time is becoming.
Thanks
Anne
From: s 47E, s 47F, Anne
On Behalf Of DEPUTY.COMMISSIONER.TAS.COORD
Sent: Friday, 6 November 2020 12:45 PM
To: s 47E, s 47F Natalie <Natalie.s 47E, s xxx@xxx.xxx.xx>
Subject: FW: For Action: Budget Estimates Questions on Notice & review of allocations
[SEC=OFFICIAL]
Natalie, this is the info re QA QoN I expected somebody to be chasing us for …….
From: s 47E, s 47F, Anne
On Behalf Of DEPUTY.COMMISSIONER.TAS.COORD
Sent: Wednesday, 4 November 2020 12:56 PM
To: s 47E, s 47F, Alexander <Alexander s 47E, s xxx@xxx.xxx.xx>
Subject: FW: For Action: Budget Estimates Questions on Notice & review of allocations
[SEC=OFFICIAL]
Alex, we note that Hansard indicates a QoN re QA methodology that is not mentioned in the
Notice. Could you add it and forward please?
It is page 115-116
Senator AYRES: In a recent media report military compensation lawyer Brian Briggs indicated that
there was a lack of consistency among delegates and that decision error rates have increased. Do you
have data that supports or undermines that contention?
Ms Cole: We measure error rates through a QA program. The QA program essentially uses a statistical
methodology we call the Saunders methodology to sample across the claims type. Basically, if there are
more claims types in MRCA PI, for example, we will sample more there than war widows, which is a
very small category.
For example, last year, following a remark in that very same ANAO audit that you mentioned earlier,
we change the methodology about how we calculate our error rate in terms of how we count the number
of errors per case. By a case I mean an individual. We have essentially made it more transparent, in that
calculation rate. So while it might appear that our QA rates are worse than in the past, there is this
element where we have instituted a more transparent and straightforward methodology in our QA rates
from this year onwards. So you can't do a direct comparison between this year and previous years.
Senator AYRES: I remember a similar discussion in earlier estimates about changing accounting
methods in a budget context. So error rates have increased, but you say it is an apples and oranges
comparison?
Ms Cole: That's correct.
Senator AYRES: Are you able to say whether, if the measurement methodology had not changed, error
rates would have increased or decreased or stayed the same?
Ms Cole: I don't believe that it's entirely due to this methodology, a change in error rates, but until we
actually calculate it out based on the previous methodology, versus the current methodology, I couldn't
answer that question.
Senator AYRES: So your view, without having reviewed that, is that error rates may have increased—
Ms Cole: In some categories it's possible.
Senator AYRES: But the data shows them increasing more than they otherwise might have—is that a
fair way of putting it?
Ms Cole: That's correct.
Ms Cosson: If I can add there, Ms Cole and her division have implemented a lot of lessons that we're
learning from the QA, from the error rate that we're seeing, particularly mandatory training. I think over
1,100 of the delegates have undergone the e-learning module to assist with learning from those errors
that are identified, plus also some mentoring sessions for senior delegates and for junior delegates. So,
importantly from my perspective, we are learning from the errors that are identified. Page 116 Senate
Monday, 26 October 2020
FOREIGN AFFAIRS, DEFENCE AND TRADE LEGISLATION COMMITTEE
reassign PDRs and update the group responsible as necessary. Advice of a nil response would
also be appreciated if applicable.
QoNs are due Deputy cleared to PSU on
Wednesday 18 November 2020. PSU will then facilitate
clearances through the Secretary and Minister’s Office.
Please note that Senators are able to ask further questions in writing and PSU will assign these
questions as they become available.
For further context about questions, in the first instance please refer to the Hansard (also
attached).
Otherwise If you have any questions please do not hesitate to contact PSU
Kind regards,
Alexander s 47E, s 47F
Senior Parliamentary Officer
Ministerial and Parliamentary Support Section
Parliamentary & Governance Branch
Department of Veterans’ Affairs
e: s 47E @dva.gov.au x:
m: s 47E, s 47F
s 47E
s 47E
