"Whether they're black, white or brindle, that makes them refugees," the NSW Senator told reporters in Canberra on Tuesday.
"They would get jobs, pay taxes, contribute to society far more than, for example, Rohingya would."
His remarks come the day after Myanmar's leader Aung San Suu Kyi discussed solutions to the ongoing humanitarian crisis engulfing the Rohingya ethnic minority in her country with Prime Minister Malcolm Turnbull at Parliament House.
Hundreds of thousands of Rohingya have fled targeted violence in Myanmar, crossing the border into Bangladesh.
Senator Leyonhjelm claimed Australians worried about "assimilation and integration" and had "a soft spot for farmers".
Whether they came as skilled migrants or on humanitarian visas, white South Africans "would integrate far more easily," he said.
"There's a good chance if it keeps going the way it's going at the moment there will be white refugee farmers from South Africa."
Australian Greens immigration spokesman Nick McKim said Senator Leyonhjelm's comments were racist.
"Discriminating against and making generalisations about people based on their ethnic background is racism, pure and simple," the Tasmanian said.
"You'd expect someone who calls themselves a libertarian to welcome more freedom of movement, especially for those fleeing persecution, without showing bias towards certain skin colours and ethnicities."
Labor immigration spokesman Shayne Neumann said Australia's non-discriminatory migration program was vital to its vibrant multiculturalism.
"If people are facing persecution, regardless of where they are from or the colour of their skin, they are able to apply to Australia's humanitarian visa program which will be assessed on its merits," he told AAP.
The question of special treatment for white South African farmers first flared up when Mr Dutton said they "deserve special attention" for facing "horrific circumstances" last week.
South Africa is seeking changes that would allow the appropriation, without compensation, of farms for redistribution to black South Africans.
The policy has led to media reports that white South African farmers are being targeted by violence and are experiencing higher murder rates.
Mr Dutton's comments earned a sharp rebuke from the South African government.
They have also polarised the party room - with conservative Tony Abbot arguing for the proposal and senior figures Julie Bishop and Mathias Cormann arguing against.