Australian government workers save an hour a day in large generative AI trial
Australian public servants who took part in a six-month trial of generative AI reported that it saved them an average of an hour a day.
The trial made Microsoft 365 Copilot, a generative AI chatbot, available to 7,600 staff across more than 60 government agencies, and is thought to be one of the largest by any government in the world.
The reported time saved fell into task categories including notetaking, minute-taking and various other administrative duties, according to an evaluation.
The Australian Public Service (APS) announced Copilot trial in November 2023. It lasted from January to June this year, and cost more than AUS$1.2m (US$615,210).
The trial was coordinated by Australia’s Digital Transformation Agency (DTA). It was non-randomised and required agencies to nominate staff to be allocated a Copilot licence.
The broader survey evaluation showed that most trial participants (77%) were satisfied with having an integrated AI tool. A greater proportion (86%) expressed a wish to continue using it. In addition, 69% of users said Copilot sped up the process of finishing tasks. Nearly as many (61%) said they believed the tool enhanced the quality of their work.
The review also captured a difference in user confidence between those who had undergone one type of training versus those who had undergone multiple forms. Training formats included online resources, hands-on experimentation, and attending agency-facilitated and Microsoft-led training sessions. Three-quarters of participants who received three or more forms of training reported a 28-percentage point lead in their confidence with using the service over those who underwent just one form of training.
Training was found to be most effective when tailored to the APS, the users’ role and the agency context.
Read more: Exclusive research sets out how UK government can capitalise on the opportunities of AI
Not a total slam dunk
The evaluation of the trial also found some need for improvements, however. Although the programme did save public servants time at work, some of that time was frequently spent reviewing AI-generated content for errors.
According to Lucy Poole, the DTA’s general manager of strategy, planning and performance, the trial also revealed the potential for social biases in AI training to impact government services.
“AI models might replicate these biases from their training data, resulting in misleading or unfair outputs, insights or recommendations,” she said.
The report details how this could include outputs that may be “biased towards western norms” and “may not appropriately use cultural data and information such as misusing First Nations images and misspelling First Nations words”.
Read more: Australian public servants urged to use AI at work
The impact on women and junior roles
Other concerns about discrimination raised in interviews with agencies included the potential for generative AI to threaten the jobs of many female and junior public servants
It was suggested that AI’s ability to assist with administrative tasks could impact entry-level positions and administrative jobs, which are mostly filled by these two groups, further down the line.
“The potential for generative AI to reduce employment opportunities or significantly alter job roles elicited strong responses,” the report finds. It recommends that agencies should “engage with staff to manage reactions and provide assurances about generative AI’s impact”.
Additionally, it says, generative AI’s impact on employment opportunities across different demographics should be regularly monitored to ensure that specific groups do not experience unfair rates of job disruption or displacement.
Poole said that concerns were raised in the review about how such AI systems could affect government jobs as a whole.
She said retaining new talent requires tools that can “improve the efficiency and quality of some types of work”.
She warned, however: “This cannot be outweighed by losing key skills around processing and synthesising concepts or knowledge of the work undertaken by an agency.”
“That is why we continue to reiterate the importance of keeping a human in the loop.”
About Jack Aldane
Jack is a British journalist, cartoonist and podcaster. He graduated from Heythrop College London in 2009 with a BA in philosophy, before living and working in China for three years as a freelance reporter. After training in financial journalism at City University from 2013 to 2014, Jack worked at Bloomberg and Thomson Reuters before moving into editing magazines on global trade and development finance. Shortly after editing opinion writing for UnHerd, he joined the independent think tank ResPublica, where he led a media campaign to change the health and safety requirements around asbestos in UK public buildings. As host and producer of The Booking Club podcast – a conversation series featuring prominent authors and commentators at their favourite restaurants – Jack continues to engage today’s most distinguished thinkers on the biggest problems pertaining to ideology and power in the 21st century. He joined Global Government Forum as its Senior Staff Writer and Community Co-ordinator in 2021.
Related Posts
