'Algorithm discrimination': How the use of AI in job hiring may disadvantage some people

Almost two-thirds of Australian organisations are using AI in their hiring processes, raising concerns about discrimination risks.

Person in a hoodie typing on a backlit keyboard in a dark setting.

Robo-interviews are becoming more common, with candidates self-recording their responses to questions posed by an AI program. Source: AAP / Tim Goode/PA

Have you applied for a job and landed an interview? You might need to prepare differently — as this time, it may not be a human deciding your fate.

In 2024, 43 per cent of Australian organisations used Artificial Intelligence (AI) "moderately" in their recruitment, while 19 per cent used it "extensively" when hiring new staff, according to The Australian Responsible AI report.

The use of technology in the hiring process can happen in different ways, such as by analysing CVs or evaluating applicants.

'Confronting' interview with AI

But nowadays, AI can be used more directly in the process through so-called "robo-interviews".

Melbourne-based social enterprise, Sisterworks, experienced this late in 2024, when a group of its graduates found themselves in a situation neither they nor their teachers had prepared for.

"We actually got caught on surprise, when towards the end of last year, we sent about nine sisters to an interview and they didn't make it, they didn't pass the interview," The group's CEO, Ifrin Fittock, told SBS News.

"We found out that they're actually not being interviewed face to face, but actually being interviewed by videos slash AI interviews.

"I think they just fail because they don't know what to do."
Sisterworks has been helping migrant and refugee women enter the workforce for more than a decade, and their recent experience with AI is not unique, with job seekers increasingly sharing similar experiences.

Robo-interviews are essentially interviews where a candidate is assessed by an AI system instead of a human. Typically, candidates record themselves and answer a series of questions, which are then evaluated by AI.

The use of technology in recruitment is often praised for its potential to save both time and money, but there are concerns that this type of hiring process can create new barriers for migrants and refugees.

"The challenges with these AI recruitment or AI interview for some of our sisters is really, first of all, English is not their first language, but also the level of digital literacy that they may or may not have," Fittock said.

"It's all really quite confronting."

Concerns about 'discrimination'

These are not the only existing concerns about AI hiring systems.

The way that these systems operate "creates serious risks of algorithm-facilitated discrimination", according to a study published by the Law School at the University of Melbourne.

Natalie Sheard, who led the study, told SBS News: "In my research I heard of systems not being accessible to job seekers with disabilities, heard of CV screening systems using things like gaps in employment history to screen candidates out.

"Having a gap in your employment history is a well-known proxy for gender as women tend to have time out of the workforce to care for family, parents, children."
Previous reports indicate that the new hiring process might discriminate against applicants wearing headscarves, individuals with disabilities, and women.

Sheard's research also highlights the issue around data sets and algorithms for the AI models.

"A lot of these systems are built overseas, so they might've been trained on data on populations that aren't comparable to the Australian population," she said.

"The system won't perform as well for particular demographic groups that we have in Australia. For example, refugees, migrant women, First Nations ..."

There are benefits too

Andreas Leibbrandt, a professor at Monash University, has also researched the role of AI in recruitment.

He says there are benefits to the technology, and if deployed correctly, a system can be less biased than a human.

In one study, conducted in the US, he found women and non-anglo candidates are more likely to apply for a job if they know AI tools are being used.

"It's not that both women or ethnic minorities feel there's no bias in the AI algorithm, but they feel that this bias is less so than when they're faced with a human recruiter," he said.
In the same time, he is concerned about a lack of transparency, regulation, and potential bias built into algorithms.

"These AI algorithms are fed with training data sets or data sets from corporations. But their training data in itself may be biased. It may come from an organisation where there was, for instance, [a] bias against women," Leibbrandt said.

While it remains uncertain whether AI hiring systems reduce bias or reinforce discrimination, applicants from various backgrounds appear to be adapting.

Video interview training is now part of the Sisterworks job course, and Fatemeh Zahra Hazrati, originally from Iran, is among those taking part.

"AI has [a] real big effect on our lives nowadays," she said.

"All of us have to learn it, [and we] just need to adapt ourselves for new things and accept challenges to learn new things."


For the latest from SBS News, download our app and subscribe to our newsletter.

Share
5 min read

Published

By Abbie O'Brien, Niv Sadrolodabaee
Source: SBS News


Share this with family and friends


Get SBS News daily and direct to your Inbox

Sign up now for the latest news from Australia and around the world direct to your inbox.

By subscribing, you agree to SBS’s terms of service and privacy policy including receiving email updates from SBS.

Download our apps
SBS News
SBS Audio
SBS On Demand

Listen to our podcasts
An overview of the day's top stories from SBS News
Interviews and feature reports from SBS News
Your daily ten minute finance and business news wrap with SBS Finance Editor Ricardo Gonçalves.
A daily five minute news wrap for English learners and people with disability
Get the latest with our News podcasts on your favourite podcast apps.

Watch on SBS
SBS World News

SBS World News

Take a global view with Australia's most comprehensive world news service
Watch the latest news videos from Australia and across the world