“There are plentiful examples of face recognition software in the past that have proven to be biased,” said Toby Walsh, a professor of AI at UNSW.
The technology is known to vary in accuracy depending on ethnicity, colour and gender.
“If I was a person of colour I would be very concerned about all the false positives,” he said.
- Toby Walsh, AI Professor
If I was a person of colour I would be very concerned about all the false positives.
“That this app would be failing to recognise me and then the police would be coming around knocking on my door.”
Similar concerns have been raised by the Australian Human Rights Commission, which in May called for a moratorium on the use of facial recognition in policing and circumstances that involve high stakes decisions.
“I haven’t seen the evidence from this company that they’ve done the appropriate work to make sure that it isn’t biased against people of colour, or even, there is plentiful evidence that it’s more likely to misrecognise woman than men,” Professor Walsh said.
The trial has been expanded this week to include 90 Australian Defence Force officers returning from overseas, while 50 people from interstate have completed the program or are currently in home quarantine.
The South Australian Premier is expected to report back to National Cabinet on Friday.
“What we learn from the domestic and international Home Quarantine SA trials could help the entire nation to safely repatriate more Australians coming home from overseas,” Premier Steven Marshall previously said.
Western Australia has already mandated the use of a similar app and the Northern Territory has bought that technology.
Using the app to quarantine at home would make it possible for Vikas Sharma to return from India to Adelaide following an 18-month separation from his wife and son.
Mr Sharma returned to India’s Punjab province before the start of the pandemic to be with his elderly mother. In April, he was hospitalised with COVID-19.
“This is a very grim situation I haven’t faced in my life,” he said.
“Now it is the reality the Australian government has stopped the flights so I have no option left to just wait for the situation to be normalised and be part of my family.”
While the app could mean he is reunited with his family sooner than planned, Mr Sharma isn’t comfortable with the biometric monitoring.
“I’m not happy with this - although I have no other options - that is why I would be excepting it,” he said.
“I am of the opinion that it will raise the racism.”
Mr Sharma is also concerned about privacy.
“All the data will be stored with the agencies, which I do not like, because already we have seen that all the time, information has been leaked all over the world.”
Civil liberty groups agree, prompting calls for strong legislation to govern the usage, storage and deletion of data.
“I would applaud anything like facial recognition technology which will have the effect of enabling us to establish the protocols for a person to return to this country safely,” said South Australian Council for Civil Liberties president Rick Sarre.
“But that can only happen, not through some ad hoc police trial, but when the government sits down and gets some legislation.”
Mr Sarre suggests a national approach, similar to the legislation that accompanies facial recognition data used at international airports.
But for many Australian citizens and permanent residents stranded overseas, a home monitoring app is a welcome prospect.
“I’d use it in a heartbeat, absolutely, because that would mean I could be back in Melbourne,” said Darren Brealey, who is currently in quarantine in Sydney after returning from the UK to see his 80-year-old mother who is unwell.
He was told there were no options for hotel quarantine in Melbourne.
“I think that would allow somebody else who needs a hotel room to quarantine; so now there’s double the number of Australians back in the country,” he said.
Another Australian citizen, Susan Holland, has packed her life in the US into boxes as she desperately tries to rebook a ticket home after her flight to Perth was cancelled due to border caps.
“We’ve got tracking cameras all over the place in America, it doesn’t bother me,” she said.
“This is maybe a step up, it maybe seems a little bit more invasive, but if it’s the price to make sure everyone is safe and that I am looked after within the regulations of public health, then I don’t see an issue.”
The South Australian Government said information collected via the app is encrypted immediately and stored on a secure server.
“The Home Quarantine SA app has stringent security features that ensures participant information is kept secure throughout the duration of quarantine,” a spokeswoman for the premier said.
“The safety and wellbeing of the community remain the number one priority. A thorough evaluation of the trial will inform future expansion of home quarantine.”
Mr Brealey said he is comfortable with existing laws.
“If I want to get home - and I’m not too concerned because I trust the policies, the laws, the regulations that are already in place will protect my privacy - then I’m more than happy to use the app and isolate at home.”
But Mr Sarre says the current privacy promises are not good enough.
“I would suggest that anyone that suggests they’ll be deleted at the end of the pandemic is in a fool’s paradise.”