Liberal frontbencher Josh Frydenberg is fully aware of the sensitivities surrounding the Racial Discrimination Act and says needed changes to the contentious section 18C would have to be approached carefully to have public backing.
But Greens Leader Richard Di Natale described it as an "insider debate" with most Australians more worried about house prices, and access to health care and education.
He described it as "solution looking for a problem", and watering down 18C would just make it easier for people to be racists.
A case involving three Queensland University of Technology students under section 18C was thrown out by Brisbane's Federal Court on Friday, sparking fresh calls for 18C to be amended to remove the words "offend" and "insult".
Mr Frydenberg said he has been on the record for sometime supporting such changes to 18C.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"I don't think the balance has been struck, I think that the threshold with offend and insult is too low," he told Sky News.
But he said Prime Minister Malcolm Turnbull was absolutely right to proceed carefully because of the sensitivities of these issues and the need have the public on board for any proposed changes.
Senator Di Natale warned Mr Turnbull not to go down this route, saying it would just make it easier for people to be racists.
"That's the outcome to a change to 18C. Let's make it easier for people to be bigots and racists," Senator Di Natale told Sky News,
However, former Human Rights commissioner turned Liberal MP Tim Wilson believes 18C has resulted in "joke cases".
"This just can't go on. When it comes down to it, every Australian has a right to freedom of speech," Mr Wilson told ABC television.
At the same time, Mr Wilson said people have a right to be free of public harassment.
"They're not incompatible things, but we have a badly designed law," he said.
"Having some sort of inquiry that would slowly, methodically look at how to reform the law and bring everybody together, would then provide opportunities for a change in the law in the future."
