Document 2 Revised Redacted.pdf

Commercial‐in‐Confidence
2
TABLE OF CONTENTS
EXECUTIVE SUMMARY ............................................................................................................. 3
CAPABILITY .............................................................................................................................. 5
Ability to meet Statement of Requirements ........................................................................................... 5
Our understanding of the required services ........................................................................................... 6
Proposed methodology ........................................................................................................................... 7
Proposed timetable ............................................................................................................................... 18
Proposed project team .......................................................................................................................... 19
PRICE ..................................................................................................................................... 21
OUR REFEREES ....................................................................................................................... 23
Experience providing similar services ................................................................................................... 23
Quality assurance plan .......................................................................................................................... 30
CAPACITY ............................................................................................................................... 31
APPENDIX A: CVS OF CORE PERSONNEL .................................................................................. 33
APPENDIX B: ADMINISTRATION DETAILS ................................................................................ 42

P2406 Department of Veterans’ Affairs – Proposal to conduct the 2018 Client Satisfaction Survey

Commercial‐in‐Confidence
3
s 22
Summary of our proposed approach
Our approach extends beyond meeting the Statement of Requirements – ORIMA Research intends to
work in close partnership with DVA to design a comprehensive and tailored client satisfaction
research program which forms an integral part of achieving the longer‐term goals of the
Department’s strategic plan and transformation efforts.
In particular, our approach will provide real insight into the experiences and needs of DVA’s clients.
Our proposed analysis approach and suite of reporting products will provide the Department with a
clear sense of the priority areas for service delivery improvement. The findings will also be presented
in a user‐friendly format that will track client satisfaction over the 2014, 2016 and 2018 surveys.
Based on our experience in conducting research with this client group and the Statement of
Requirements, we propose to utilise a telephone methodology for the survey fieldwork. This
methodology has worked extremely well ensuring results which are representative of DVA’s client
base whilst allowing significant ‘drill‐down’ capacity in to client groups with particular profiles or
concerns (for example, the under 45 veteran cohort has historically recorded significantly lower
levels of satisfaction with DVA). We propose:
 a 20‐minute Computer Assisted Telephone Interview (CATI) survey for n=3,000 veterans, with a
sample designed to ensure representative coverage of key client cohorts;
P2406 Department of Veterans’ Affairs – Proposal to conduct the 2018 Client Satisfaction Survey

Commercial‐in‐Confidence
4
  The random sampling methodology (stratified by gender, age, and state at a minimum)
will be designed to collect statistically robust survey.
  Compared to self‐completion methodologies, telephone surveys will help minimise
non‐response bias (response rates for CATI surveys are generally higher than self‐
completion surveys), reduce respondent burden, and minimise environmental impact
(noting that paper‐based surveys typically achieve lower response rates). It also
enables interviews to be translated if required (in 2016, a handful of interviews were
conducted in Cantonese and Vietnamese).
  a final major report which forms includes time‐series tracking of the 2014, 2016 and 2018
results, sophisticated key drivers analysis to provide DVA with greater insight into those
aspects of service delivery which are driving client satisfaction for specific ‘touch points’.
Importantly, the analysis will also identify mutually exclusive client segments that differ in
terms of behaviours and attitudes, allowing DVA to get an in‐depth understanding of clients’
needs, wants, barriers and enablers in relation to their experience transitioning from the
defence force (as applicable), seeking assistance from support services, and interacting with
DVA.
  The major report will be supplemented by:
  an Executive Summary, summarising the major findings and suitable for public release;
  A Fact Sheet in infographic style describing the key findings, suitable for public release;
  A Fact Sheet summarising the client segmentation, suitable for public release if
required;
  A master presentation deck suitable for the in‐person presentations;
  The raw data; and
  The de‐identified verbatim comments.
  As in 2016, we will remain flexible with our reporting suite and can tailor or add
reporting deliverables to suit changing needs.
Another key feature of our proposed approach includes a significant focus on effective questionnaire
design.
  We have provided for an optional, but strongly recommended, qualitative component to guide
the development of the 2018 questionnaire, through consultation with DVA clients and
members of Ex‐Service Organisations as appropriate. We feel that the questionnaire is due for
a solid refresh in order to ensure it is capturing the feedback clients wish to provide about
DVA.
  Putting clients at the centre of this questionnaire development process will yield
significant benefits for DVA through a stronger and up‐to‐date understanding of client
needs, wants and pain points, whilst reflecting the true intent of the transformation
efforts. It will also ensure the questionnaire is meaningful (language, terminology, etc)
to clients and hence will strengthen the validity of the instrument.
  As a core service, we aim to assist DVA co‐design the questionnaire, ensuring it meets DVA’s
needs whilst maintaining comparability to key historical data for tracking purposes. Significant
work was undertaken with the DVA project team in 2016 to identify the ‘core’ (asked every
survey), ‘topical’ (asked on an ‘as needs’ basis) and ‘seasonal’ (asked every second survey)
questions going forward. This means that the 2018 instrument will start on a very strong
footing as we will bring this context to the questionnaire development process, though we
anticipate that the survey will change in a range of ways (including through asking questions
that will help to segment the client population).
P2406 Department of Veterans’ Affairs – Proposal to conduct the 2018 Client Satisfaction Survey

s 22

Commercial‐in‐Confidence
9
s 22
Sample design
In Phase 1 we will work closely with the DVA project team to finalise the CATI sample design in light
of the research objectives and likely questionnaire content, to ensure that there is scope in the
sample for us to ‘drill down’ into the findings for specific cohorts within the client base to undertake
more tailored and sensitive analysis
Table 1 below provides an indicative CATI sample, designed to provide representative coverage of
the client base with a total of n=3,000 interviews being conducted. This example provides a level of
statistical precision for survey estimates of around +/‐ 2 percentage points at the 95% confidence
level, and robust estimates at a cohort level.
  Compared to other data collection approaches, a CATI methodology will provide not only
representative coverage (including minimising non‐response bias), but it is also the most
universally accessible given the diverse nature of DVA’s client base.
We have based our indicative CATI sample design on the design used in 2016, stratifying by age,
gender, and State/ Territory. Our sample design also distinguishes between those clients who have
contacted DVA through the Client Contact Facility, and those with no recorded contact in the last 12
months – in 2016 and 2014, this was important given that there were multiple sources of data for the
DVA project team to collate to form a representative client sample frame (from which to select the
survey respondents).
  However, we are happy to tailor the sample design to suit DVA’s analysis requirements and
sample list accessibility.
  The final sample design will allow DVA to ‘drill down’ further into the results to identify
nuances in client experiences and therefore tailor action strategies accordingly.
  The final sample design will form the parameters for the contact list the DVA project team will
need to provide. We will provide detailed guidelines for the project team on the sample list
specifications, including list randomisation, data fields to include, and number of records
required (given response rate estimates and required number of interviews).

P2406 Department of Veterans’ Affairs – Proposal to conduct the 2018 Client Satisfaction Survey

Commercial‐in‐Confidence
11
As a core service, we will hold a questionnaire development meeting with the DVA project team to
develop the first draft of the questionnaire.
  ORIMA will bring a template questionnaire to the development meeting, based on the 2016
instrument, and on the information obtained at the inception meeting, and incorporating the
findings from the qualitative research (if undertaken). ORIMA will work closely with the project
team adapt and refine this version of the questionnaire at the meeting, ensuring that key
variables are maintained for tracking purposes.
  As noted above, significant work was undertaken with the DVA project team in 2016 to identify
the ‘core’ (asked every survey), ‘topical’ (asked on an ‘as needs’ basis) and ‘seasonal’ (asked
every second survey) questions going forward. This information will be brought to the
questionnaire development discussion for consideration by the team for the 2018 survey.
  We propose to include a range of attitudinal, behavioural and demographic questions to
facilitate value‐added analysis of the survey results and to allow for the targeting of follow‐up
strategies where appropriate. The attitudinal questions will be designed to facilitate
segmentation of respondents based on their underlying attitudes and confidence in DVA
service delivery and to combine this with the demographic profile of respondents to provide a
robust picture of key variations in priorities between cohorts within the client population.
We expect that the core survey questions will cover clients’ dealings with DVA’s various touchpoints,
as well as overall satisfaction with the service provided by DVA.
  Filter and drill‐down questions will be used to ensure that respondents are only asked
questions relevant to their experiences (minimising the burden on individual respondents), and
that as much information as possible is obtained about their experiences and expectations.
To maximise the likely response rate and enable delivery of the survey within the given budget, our
approach involves targeting a survey length of around 20 minutes, which equates to around 75
questions. For ease of response, most questions will be close‐ended, however, there will also be
scope for a few free‐text questions. Average completion time will be tested during the pilot testing
phase.
We will ensure regular telephone and email contact with the DVA project team to effectively and
efficiently finalise the draft Word version of the questionnaire, based on the outcomes of the
development meeting. As the questionnaire nears its final draft, we will work closely with the project
team to carefully review the questionnaire in terms of the following aspects:
  wording, terminology, clarity and understanding of questions;
  appropriateness of response scales and variety of options;
  questionnaire flow and skips;
  facilitating effective analysis; and
  length (average completion time).

We will then seek the project team’s assistance to conduct a 2.5 hour workshop with key internal
stakeholders, to further refine the draft by ensuring that key issues are included, and that all parties
are ‘on the same page’ in relation to questionnaire intent, structure and broad analysis plan. We will
rely in the project team to provide the venue and invite relevant attendees. A senior ORIMA
consultant will run the workshop, assisted by a note‐taker.
  The note‐taker will update the questionnaire ‘live’ during the workshop, such that at the end of
the workshop a revised draft is available for circulation and final comment, and then approval by
the project team that the questionnaire is ready for CATI programming.
P2406 Department of Veterans’ Affairs – Proposal to conduct the 2018 Client Satisfaction Survey

s 22