FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
EXECUTIVE BRIEF
3.2. While the Agency seeks approval to continue the Copilot trial from July 2024 to June
2025, further work will be undertaken to understand the risk exposure that AI poses
to the Agency and will evaluate key mitigation strategies that are in place on
completion of the trial.
3.3. The majority of DTA trial participants across government have elected, where
possible to continue using Copilot. Departments such as Home Affairs have a
Protected network and are unable to continue the trial in this environment. The
Agency doesn’t have this restriction and so is not limited by its operating
environment and policy settings.
3.4. Continuation of the trial will allow further opportunities for the EDN to explore the
accessibility benefits of using Copilot in their business-as-usual activities. This will
provide further data on the use of Copilot as an accessibility assistant and provide
opportunities for the Agency to work with Microsoft as an active partner in improving
accessibility features.
3.5. An extension of the trial will be separate from the current DTA trial, which finished in
June 2024. Microsoft has quoted
per Copilo
s47G - busines
t licence to allow a continuation of
the trial, with our current pool of 300 licences, equating to a cost of s47G - business
inform
(GST Inc.) for 2024/25.
3.6. The extended trial at the NDIA will utilise all 300 licences by including most NDIA
SES, and their Executive Assistants. This will expand the trial participants into more
business areas, further testing the capability, value, and risks of Copilot.
4. Risk Mitigation
Risks
4.1. There is a recognised potential for Copilot to generate inaccurate, misleading, or
inappropriate content that may affect the quality and integrity of NDIA's documents
and communications.
4.2. The possible legal and ethical implications of using Copilot-generated content
without proper verification, attribution, or consent, especially when dealing with
personal or sensitive information of participants, staff, or stakeholders is problematic
where use of the tool is unregulated, and users are not trained and supported in
their use.
4.3. The continued development of a robust governance framework and accountability
mechanism for the continued use of Copilot, including clear roles and
responsibilities, escalation and approval processes, and feedback and evaluation
mechanisms is essential.
4.4. Such a governance framework is further enhanced by ongoing communication and
engagement with Copilot users to raise awareness of functions and risks inherent in
the tool, whilst encouraging best practice and soliciting feedback on the benefits and
challenges of using Copilot.
Copilot's access and separation
4.5. Prior to the implementation the trial the Agency conducted a Cyber Security risk
assessment (Authority to Operate) on the Microsoft suite of applications that
incorporate Copilot. The DTA has also undertaken an assessment of Copilot, which
was reviewed to identify risks that may impact the Agency. The Cyber Security risks
of using Copilot are considered LOW due to the following:
OFFICIAL: SENSITIVE
Extension of the Microsoft Copilot Trial
2
Page 8 of 76
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
EXECUTIVE BRIEF
4.5.1. Copilot operates as a private tool, which is separated from other Microsoft
customers and the public. Copilot only accesses information and data in the
Microsoft suite of applications which the user has authorisation to access.
4.5.2. Sensitive NDIA systems and data (such as PACE, SAP CRM and Protected
Enclave) cannot be accessed by Copilot.
4.5.3. The NDIA’s use of Copilot is designed to work within the boundaries of the
NDIA ICT environment and does not access the wider internet.
4.5.4. Microsoft has made a commitment to the Australian Government that the use
of Copilot will never train its large language model to gain insights and
intelligence from Government processes and decision making.
Artificial Intelligence and Machine Learning Policy
4.6. Risk will continue to exist where a virtual office assistant is being used based on a
Large Language Model of Artificial Intelligence (LLM). LLM is the deep learning
algorithm in AI tools which summarise, translate and generate replicated human
responses, ideas and concepts.
4.7. The Use of Artificial Intelligence, Generative AI, and Machine Learning – Interim
Policy was implemented in April 2024 to address the inherent risks of using AI and
machine learning within the Agency (Interim Policy)
4.8. The Interim Policy, which was developed in consultation with the DTA to support
users of the technology and is accompanied by mandatory learning modules for
users that address the ethical, legal and security risks (
Attachment B).
4.9. The Interim Policy has been compared with the National Framework for the
Assurance of Artificial Intelligence in Government published by the DTA on June 21,
2024, to determine if adjustments are necessary.
4.10.
The review concluded that no modifications to the Interim Policy are needed
at this time regarding Copilot usage. Another review is expected by December 2024
to include insights from the DTA’s Copilot survey conducted across all participants in
the trial and the Privacy Assurance Advice provided to the Agency on August 8,
2024 (
Attachment C).
Privacy and data protection measures
4.11.
Some to the updates anticipated are:
4.11.1. Establishing clear and transparent criteria for the selection of Copilot users
and ensuring that they are adequately trained and informed of their privacy
obligations.
4.11.2. Implementing a Privacy Impact Assessment (PIA) process for any new use
cases of Copilot that involve personal information or sensitive data.
4.12.
Developing and enforcing clear guidelines and protocols for the use of
Copilot, including the types of documents and data that can or cannot be accessed
or generated by Copilot, the retention and deletion of Copilot outputs, and the
reporting and resolution of any privacy incidents or breaches.
4.13.
Conducting regular audits and reviews of Copilot's performance, accuracy,
and compliance with privacy laws and NDIA policies.
Risk management and governance
OFFICIAL: SENSITIVE
Extension of the Microsoft Copilot Trial
3
Page 9 of 76
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL: SENSITIVE
EXECUTIVE BRIEF
4.14.
The Agency’s AI Working Group collaborated with the Agency Risk, Audit and
Resilience Division to undertake a risk assessment (
Attachment D), consolidating
existing risk mitigation strategies and to identify any emerging risks (
Attachment E).
4.15.
The project team will engage with the Agency Ethics Committee anticipated to
be established in October 2024 through the Integrity Agencies and Ethics team in
the Legal Services Branch. The Agency Ethics Committee will maintain a standing
item on the use of AI and will establish a review mechanism for all new AI and
machine learning technology proposals. The project team will provide updates in
status reporting to that Committee.
4.16.
The Agency will continue to consult with the DTA on the Copilot trial to
support the adoption of government wide practice and policies guard railing the use
of AI.
5. Recommendation
5.1. The COO:
5.1.1. notes the preliminary findings of the Copilot trial and the evolving AI
governance and policy.
5.1.2. approves the extension of the DTA Copilot trial through to July 2025.
5.1.3. notes that the project team will continue to engage on the ethical and
responsible use of Copilot and inform our evolving governance and policy
position on AI under consultation with the DTA.
6. Authorisation
Approval in PDMS History
……………………………………..
Date: 08 July 2024
Natasha Murphy
Prepared by: Carol s47F - personal privacy
Branch Manager
Enterprise Architecture and Governance
Mobile: s47F - personal privacy
Phone / Mobile s47F - personal privacy
OFFICIAL: SENSITIVE
Extension of the Microsoft Copilot Trial
4
Page 10 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
the primary production tenancy of participating agencies which has been available
for deployment since January 2024.
What were the Minimum requirements for trial participation?
The terms and minimum commitment for trial participation are an Agency commitment of:
• 300 Copilot for Microsoft 365 licences from agencies with 2,000 Microsoft 365
Qualified Users or more; or
• The greater of 15% of users or 10 Copilot for Microsoft 365 licences for agencies
with fewer than 2,000 Microsoft 365 Qualified Users
• Commitment for the term of trial.
The overall benefit from Copilot for Microsoft 365 will be impacted by how heavily invested
agencies are in their usage of Microsoft 365.
Agencies may opt to extend the term of their Copilot for Microsoft 365 licences after the
trial for a further 12 months. The DTA will be contacting agencies with more information
about extending their commitments as part of the annual licence reconciliation process.
Key steps and timeline
OFFICIAL
Page 13 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
Training and Support
• The DTA and Microsoft will be working with participating agencies to support them,
through onboarding and readiness workshops, and additional training.
• Readiness workshops and training will be at no extra cost to agencies.
• An information pack containing FAQs addressing whole-of-government guardrails,
possible use cases, evaluation approach and other information about the trial will be
provided to trial participants early in the trial process.
Useful information
The following resources have been developed to assist implementing AI in the APS
Interim guidance on government use of public generative AI tools | aga (digital.gov.au)
Australia’s AI Ethics Principles | Australia’s Artificial Intelligence Ethics Framework -
Department of Industry, Science and Resources
How might artificial intelligence affect the trustworthiness of public service delivery -
PM&C (pmc.gov.au)
Product information
Copilot for Microsoft 365
Microsoft Copilot Copyright Commitment Security Information
Data, Privacy, and Security for Copilot for Microsoft 365
Microsoft Purview data security and compliance protections for Microsoft Copilot
Support and Feedback on the trial
The NDIA project team responsible for conducting the trial can be contacted via:
s47E(d) - certain operations of agencies
OFFICIAL
Page 14 of 76
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
End of Trial Report
1
Copilot for Microsoft 365
1
January – June 2024
1
1. Executive summary
4
1.1 Background
4
1.2 Objectives of the trial
4
1.3 Purpose
5
2. Copilot at the NDIA
6
3. Implementation of Copilot for Microsoft 365
8
3.1 Approach
8
3.2 Planning and Configuration
8
3.3 Training and Support
9
3.4 Implementation and Monitoring
9
4. Results
10
4.1 Using Copilot
10
4.2 Accessibility
11
Neurodivergent User
12
JAWS User
12
Dyslexic User
12
Hearing Impaired User
12
4.3 Ongoing Training, User Monitoring, and Guidance
13
4.4 Network Integration
13
5. Summary of Benefits
14
6. Key Findings
16
6.1 User Productivity
Error! Bookmark not defined.
6.2 User Satisfaction
Error! Bookmark not defined.
6.3 Accessibility Usage
Error! Bookmark not defined.
6.4 Stakeholder Engagement
Error! Bookmark not defined.
7. Perceived Risks and Mitigation Strategies
17
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
2
Page 16 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
8. Controls for Mitigating Risks During the Trial
19
8.1 Data Privacy Controls
19
8.2 User Training and Support
19
8.3 Ethical Guidelines
19
9. Summary of Recommendations
20
10. Conclusion
20
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
3
Page 17 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
1. Executive summary
1.1 Background
In the wake of rapid advancements in Artificial Intelligence (AI) technologies, the Agency
recognises the potential of AI to transform public service delivery, enhance operational
efficiency, and foster innovation.
The Australian Public Service (APS) commenced a six-month trial of Copilot for Microsoft
365 in January 2024 to test new ways to enhance productivity and develop skills,
capabilities and preparedness for the use of generative AI tools by APS staff. The trial was
coordinated by the Digital Transformation Agency (DTA) and involved over 50 participating
Commonwealth agencies.
The Agency is participating in the trial alongside numerous Australian Government
Agencies and Departments and obtained 300 licenses. The trial started as a Proof of
Concept (POC) for four months, and then continued as a Trial for the remaining trial period.
A smaller group of users were chosen for the er group of users were chosen for the POC,
more users were included for the Trial. Staff who participated were urged to use Copilot to
help with their daily administrative tasks.
Over the six-month trial period, data was collected and analysed to assess security posture,
user productivity, satisfaction, and accessibility improvements.
1.2 Objectives of the trial
•
Evaluate effectiveness: Assess staff using Copilot impacts productivity and user
satisfaction.
•
Determine value for money: determine whether future use by all NDIA staff
represents value for money.
•
Ongoing AI risk mitigation: Assess if having an internal AI product like Copilot
reduces the likelihood of staff accessing external/unapproved AI services, and the
corresponding threat to Agency data that arises from unapproved use.
•
Policy development: Use the trial to develop a policy position and unified approach
to the use of Artificial Intelligence within ICT services.
•
Accessibility impact: Measure the accessibility improvements for users requiring
support.
•
Gather data for informed decisions: Collect quantitative and qualitative data to
inform future AI tool implementations.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
4
Page 18 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
1.3 Purpose
This purpose of this report is to:
• Describe the implementation of Copilot for Microsoft 365 at the Agency
• Illustrate use cases of Copilot at the Agency
• Evaluate the effectiveness of Copilot for Microsoft 365 on productivity and
accessibility within the Agency.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
5
Page 19 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
2. Copilot at the NDIA
A description of how Copilot can assist NDIA staff with their daily tasks is tabled below.
Use
Description
Ensuring policy
Assistance with preparing documents and emails and aligning
compliance
with NDIA policies and guidelines by:
• interpreting NDIA policies and generating a summary of the the
purpose
• identifying amendments to a policy document and the
responsible person
Meeting scheduling
Quickly schedule and reschedule meetings:
• without manually comparing participants’ calendars for
availability, and
• creating meeting invites, adding participants and setting the
meeting date and time based on availability
Checking document Improve the accessibility of documents by providing suggestions
accessibility
for:
• readability and simplicity
• formatting
• complying with the NDIA’s accessibility standards
• colour contrast for colour vision deficiencies
• converting into structured PDF or plain text for accessibility
• accessibility best practices
Email summary and Assist users with:
creation
• summarising key points and decisions from large email chains
• drafting email responses
• managing email inboxes (eg flagging important emails)
Assisting with data
Assist users with:
analysis
• creating complex formulas in Excel spreadsheets
• generating charts and graphs to visualise data
• interpretating numbers of trends
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
6
Page 20 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
Use
Description
• sorting and filtering data
• reminding users to keep data up to date
Providing in-context Assist user with learning Microsoft 365:
training
• navigating tools and processes by providing step-by-step
instructions
• helping learn more about advanced features
Document
Assist visually impaired staff with reviewing extensive documents
summarisation and
and reports by:
reading aloud
• providing summaries of Word documents with the key points,
decisions and action items; and
• text-to-speech functionality.
Presentation
Assist staff with creating engaging and informative presentations
creation assistance
by:
• suggesting a slide structure from an outline of the presentation
• using the appropriate NDIA templates
• retrieving and including relevant statistics, images and case
studies from NDIA data
• assisting with presentation rehearsals
• assisting with managing live Q&A sessions by quickly providing
information which answers the audience’s questions
Accessible data
Assist visually impaired users interpret complex data sets, charts
analysis and
and graphs by:
reporting
• interpreting data and providing simple and understandable
insights in a format which is understandable without visual aids
• analysing trends, summarising data points and predicting future
patterns based on historical data
• creating accessible reports through the translation of the data
analysis into descriptive narratives in a format which is inclusive
for all
• allowing reports to be easily shared through Microsoft 365
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
7
Page 21 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
Use
Description
• ensure documents created are compliant with accessibility
standards and easy to navigate
Accessible project
Assist hearing impaired users with managing projects by:
management
• enabling real-time transcription during Teams meetings so the
user receives live captions and allow for confirmation of any
discussion points which may have been missed
• generating notes, list tasks and assist the user in other ways
during the meeting
• organising project tasks and timelines, the user can provide
Copilot verbal instructions to update task statuses, add new
assignments, and set reminders for deadlines.
3. Implementation of Copilot for Microsoft 365
3.1 Approach
The trial faced difficulties due to:
• scepticism about the usefulness of generative AI tools such as Copilot
• concerns for the security and protection of Agency data Copilot would access
• the perception that the trial was being conducted to evaluate it’s effect on
productivity as part of a plan to reduce staff numbers
• findings of the recent “Robodebt” Royal Commission
To mitigate, the Copilot trial was reduced to a Proof of Concept (POC) for the initial four
months of the trial. This provided the additional time required to carefully examine and
mitigate risks before conducting the trial. Add some points in here about what was scaled
down for the POC. A communication ban was enforced that impacted the ability to
communicate broadly.
The POC was extended to a full trial in May 2024. Add some points in here about what was
scaled up for the Trial. What happened to the communication ban?
3.2 Planning and Configuration
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
8
Page 22 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
•
Planning phase: Detailed planning sessions were held to define the scope,
objectives, and success criteria for the trial.
•
Infrastructure configuration: Necessary IT infrastructure and security measures
were put in place to support the deployment of Copilot.
•
Agency AI Working Group formation: Approximately 10 executives across
different NDIA Business Areas, including staff requiring accessibility support.
•
POC Group selection: Select up to 300 NDIA staff across different Groups,
Divisions and Sections, including users requiring accessibility support.
•
Trial Group selection: Select up to 300 NDIA staff across different Groups,
Divisions and Sections, including users requiring accessibility support.
3.3 Training and Support
•
Information sessions: Comprehensive training sessions were conducted to ensure
users were familiar with Copilot features and functionalities.
•
Learning modules: A mandatory learning module ensured that all users were taken
through the ethical and professional obilgations of using AI.
•
Support channels: Dedicated support channels were established, including a
helpdesk, FAQs, and user guides.
3.4 Implementation and Monitoring
•
Phased rollout: The implementation was carried out in phases to manage any
potential issues effectively.
•
Monitoring and feedback: Continuous monitoring of usage and performance, along
with regular feedback collection from users, was conducted to identify areas for
improvement.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
9
Page 23 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
4. Results
4.1 Using Copilot
The following benefits were demonstrated by users using Copilot, and could also be
realised by users across all Australian Government Departments:
Benefit
Copilot Use
Enhanced productivity Copilot assisted users achieve their tasks faster. For
example, capturing summaries of meetings, assisting with
summarising and prioritising email trails, assisting with
scheduling meetings. This outcome indicates significant
improvements to staff productivity.
Improved accessibility Copilot enhanced accessibility support by providing guidance
to enhance the accessibility of document content; interpreting
complex text, graphs and analytical data; providing
assistance during and after meetings through transcripts and
summaries; and other capabilities that increased a staff
member’s independence to complete tasks. This improved
engagement and productivity of staff relying on accessibility
tools.
Data-driven decision
Copilot has the ability to search through NDIA data for
making
relevant statistics, images, and case studies. This feature
ensures that the content is accurate and visually appealing,
aiding in informed decision-making.
Presentation
Using Copilot's assistance to prepare and rehearse
preparation and
presentations was a key finding. Copilot can generate slides
delivery
with suggested titles and content, provide a script based on
the slide content, and offer feedback on delivery, which
analyses pacing, clarity, and tone.
Streamlining
All use cases indicate that Copilot will streamline routine
administrative
administrative activities, such as writing briefs and initial
activities
drafts of correspondence. This has the potential to provide
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
10
Page 24 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
Benefit
Copilot Use
significant time savings to administrative staff within the
Agency.
4.2 Accessibility
The participation of Employee Disability Network (EDN) members in the NDIA trial provided
valuable insights into Copilot's impact on their work. The tool received overwhelmingly
positive feedback, with notable benefits.
User Group
Copilot Use
Neurodivergent Users
Copilot significantly improved task management,
communication, and focus, offering tailored suggestions and
automating routine tasks.
JAWS Users
Copilot's integration with JAWS screen readers enhanced
accessibility, making information more accessible and
interactions more efficient.The ability to use Copilot to recap
meetings allowed visually impaired users the freedom to
participate fully in meetings rather than concentrating to
ensure that transcript matched the tone of the discussion.
Hearing Impaired
Copilot’s application within Microsoft Teams was the biggest
Users
gamechanger for this group of users. The recap function, the
live transcript in meetings, and the ability to ask Copilot for
clarification when the conversation moved too quickly were
revolutionary for hearing impaired users.
Users with Dyslexia
The ability to draft content from simple prompts or rewrite
written content allowed users with Dyslexia the comfort of
knowing that their work was correct, raising their confidence
and shortening editing and review time across the board.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
11
Page 25 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
4.3 Ongoing Training, User Monitoring, and Guidance
Feedback from users indicated that Copilot made it very easy to slip into usage patterns
outside of the trial and interim policy parameters. Ongoing education around the ethical use
of AI would be required to ensure that users complied with data protection expectations.
Training and
All use cases indicated that the implementation of any AI tool
Support
required ongoing and comprehensive training sessions. A new
tool like Copilot also required dedicated support channels,
including a helpdesk, FAQs, and user guides.
Ethical Use and
The establishment of clear guidelines, transparency, and
Data Privacy
accountability measures to monitor the ethical use of AI tools,
along with strict data privacy controls, are critical use cases that
demonstrate responsible AI implementation.
4.4 Network Integration
Microsoft Managed
The Agency's (MMD) program, coupled with its greenfield
and Device (MMD)
relationship with Microsoft, facilitated a smooth and efficient
program
activation of Copilot.
This advanced setup allowed for quick deployment and reduced
downtime, ensuring minimal disruption to ongoing operations.
Identity
Identity management functions were configured effectively,
Management
allowing users to access data and information strictly within
Identity and Access Management (IDAM) rules.
This setup not only enhanced security but also ensured
compliance with internal data governance policies, providing a
robust framework for secure and efficient use of Copilot across
the Agency.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
13
Page 27 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
5. Summary of Benefits
The following benefits were realised during the POC and trial of Copilot:
Benefit
Description
Enhanced Collaboration
Copilot facilitates better collaboration by
summarising conversations, providing real-time
suggestions, and enabling seamless sharing of
information, thus enhancing team productivity.
Process Optimisation
By automating routine tasks and providing
intelligent suggestions, Copilot helps streamline
workflows, reducing manual effort and
minimising errors.
Time efficiencies
Copilot saves time by automating repetitive
tasks, drafting emails, and summarising
lengthy documents and conversation threads,
allowing users to focus on more complex
tasks.
Skill Development
Copilot assists users in learning and improving
their skills by providing context-aware tips and
recommendations, thus enhancing their overall
proficiency in using Microsoft 365 tools.
Accesibility Improvements
Copilot's accessibility features significantly
improved the experience for all users with
disabilities within the trial.
Cyber Security Mitigations
By providing an in-house AI tool like Copilot,
the Agency mitigated the risk of users seeking
potentially insecure, web-based AI tools,
ensuring Agency data remains within the
secure government ecosystem.
Improved Decision Making
Copilot supports better decision-making by
providing data-driven insights, analysing trends,
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
14
Page 28 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
Benefit
Description
and generating reports, enabling users to make
informed and timely decisions.
Enhanced User Satisfaction
The intuitive interface and helpful suggestions
from Copilot lead to higher user satisfaction
and engagement, contributing to a more
positive work environment.
Reduced Workload
By automating repetitive and time-consuming
tasks, Copilot reduces the overall workload on
employees, helping them manage their time
more effectively and reduce burnout.
Consistency in Work Output
Copilot ensures consistency in documents and
communications by providing standard
templates and suggesting improvements,
leading to a more uniform and professional
output.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
15
Page 29 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
6. Key Findings
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
16
Page 30 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
7. Perceived Risks and Mitigation Strategies
Risk Category
Risk Description
Likelihood
Impact
Mitigation Strategy
Data Privacy External
Potential risks of data
Medium
High
Strict adherence to data
breaches or misuse of
privacy regulations, regular
sensitive information by
security audits, and user
members
awareness training.
Data Privacy Internal
Risks of accidental data
Medium
High
Implementation of robust
exposure or
access controls, continuous
unauthorised access by
monitoring of data access and
internal users
usage, regular internal audits,
and comprehensive training
for employees on data
handling practices.
Integration Issues
Technical challenges in
Low
Low
Thorough testing, phased
integrating Copilot with
rollout, and collaboration with
existing systems
Microsoft for troubleshooting. |
User Resistance
Resistance to adopting
High
Medium
Comprehensive training,
new AI tools
ongoing support, and
communication plan that
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
17
Page 31 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
Risk Category
Risk Description
Likelihood
Impact
Mitigation Strategy
highlight benefits and lowers
expectations
Dependence on AI
Over-reliance on AI tools Low
Low
Encouraging balanced use,
might reduce critical
where Copilot is seen as a
thinking skills
support to repetitive tasks and
not a replacement of human
judgment.
Ethical Concerns
Ethical implications of
High
High
Preparation of the Interim Use
using AI in decision-
of AI Policy and establishing
making processes
clear supporting guidelines
and ensuring transparency in
how the Agency assesses AI.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
18
Page 32 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
8. Controls for Mitigating Risks During the Trial
8.1 Data Privacy Controls
Encryption: All sensitive data was encrypted both in transit and at rest.
Access Controls: Strict access controls were implemented to ensure only authorised
personnel could access sensitive information.
Regular Audits: Periodic security audits were conducted to identify and mitigate potential
vulnerabilities.
8.2 User Training and Support
Comprehensive Training: Detailed training programs were conducted to ensure users
were well-versed in using Copilot.
Continuous Support: Ongoing support was provided through helpdesks, user guides, and
FAQs to address any user concerns promptly.
8.3 Ethical Guidelines
Transparency: Clear guidelines were established to ensure transparency in how AI tools
were used within the organisation.
Accountability: Accountability measures were put in place to monitor the ethical use of AI
tools.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
19
Page 33 of 76
OFFICIAL
FOI 24/25-1345 - DISCLOSURE LOG
OFFICIAL
9. Summary of Recommendations
Extend the Trial: Extend the trial for an additional 12 months to gather more data and
further evaluate the long-term impact.
Expand user base: Include more users, particularly those requiring accessibility support, to
get a broader understanding of the tool’s impact.
Enhance training: Continue to provide comprehensive training and support to maximise
user engagement and effectiveness.
Strengthen partnerships: Foster stronger relationships with Microsoft and other AI tool
providers to ensure continuous improvements and support.
Establish clear guidelines: Develop and implement clear guidelines for the ethical use of
AI tools within the organisation.
10. Conclusion
The Copilot for Microsoft 365 trial has shown significant potential in enhancing productivity,
user satisfaction, and accessibility within the NDIA.
The six-month trial of Copilot for Microsoft 365 demonstrated significant benefits for the
Agency, particularly for employees with accessibility needs. The tool's positive impact on
neurodivergent users and those relying on JAWS underscores its potential as an inclusive
productivity aid.
Additionally, robust data security measures ensured the protection of Agency and
participant data throughout the trial.
Extending the trial period and expanding the user base will provide more comprehensive
data, allowing the agency to make informed decisions about the long-term adoption of AI
tools.
The controls and mitigation strategies implemented during the trial have effectively
managed risks, and the lessons learned will guide future initiatives to further leverage AI
tools for organisational benefits.
Based on these findings, it is recommended that the Agency consider wider adoption of
Copilot to further enhance productivity and accessibility for all employees.
ndis.gov.au
17 June 2024 | Copilot – End of Trial Report
20
Page 34 of 76
OFFICIAL
DOCUMENT 7
Senate Community A
FOI
ffai
24/25-1345
rs Legislation Committee
- DISCLOSURE LOG
ANSWER TO QUESTION ON NOTICE
2024-25 Budget Estimates
The Hon Bill Shorten MP
Minister for the National Disability Insurance Scheme
Minister for Government Services
Topic: Use of Artificial Intelligence - Intergovernmental Guidance
Question reference number: NDIA SQ24-000177
Question asked by: James McGrath
Outcome Number: 3
Type of Question: Written
Hansard Page/s:
Date set by the Committee for the return of answer: 25 July 2024
Question:
Question 1: The Department of Prime Minister and Cabinet has stated that the Digital Transformation
Agency and the Department of Industry Science and Resources have released interim guidance to ‘all
government agencies’ on the proper and adequate utilisation of Artificial intelligence.
Has the NDIS employed this guidance?
Question 2: What internal steps have been taken to employ this guidance?
Question 3: Has the NDIS passed on this advice from the DTA to their third-party contractors?
Question 4: Does the NDIS require that third party contractors adhere to the DTA’s guidance on the
utilisation of AI? If not, is there another policy in place?
Answer:
Yes. The National Disability Insurance Agency (NDIA) first employed this guidance in the
development of the interim policy on the use of Artificial Intelligence (AI) and provided advice to
NDIA staff who are taking part in the trial of Copilot for Microsoft 365. The Agency is subsequently
in the process of updating this policy to align with the ‘National framework for the assurance of
artificial intelligence in government’ (AI Framework) released on 21 June 2024.
All third-party contractors, including labour hire staff, are required to adhere to the Agency's policies
while providing services to the Agency. This includes the Agency’s Use of AI, Generative AI, and
Machine Learning – Interim Policy (Interim Policy), which was developed using the Digital
Transformation Agency advice and guidance that form the basis of the Framework.
The Agency requires that anyone, regardless of engagement type, must comply with all Agency
policies and practices including the interim policy and undertake any required training. Third party
contractors have this stipulated in their contracts.
Page 76 of 76
Page 1 of 1
