Community groups around the country have fought this battle before.
Two years ago, it was their backlash against changes to the Racial Discrimination Act that forced former Prime Minister Tony Abbott to abandon such a move.
Now the issue's being looked at by a parliamentary inquiry.
Assistant Multicultural Affairs Minister Zed Seselja says there are important arguments being made both for and against the changes.
"I think fundamentally we can all agree that we condemn racism, that we do support free speech and we always just have to work out exactly where the line should be drawn as to when the law intervenes."
Earlier this month, the Prime Minister agreed to a parliamentary inquiry into whether section 18C of the Racial Discrimination Act imposes unreasonable restrictions on freedom of speech.
It makes it illegal to "offend, insult, humiliate or intimidate" because of a person's "race, colour, national or ethnic origin".
Rod Little, from The National Congress of Australia's First Peoples, is among almost a dozen representatives from community groups who've visited Canberra this week to make their message clear.
They want the Racial Discrimination Act left alone.
"Here there's authorities and legislation in place to protect us and you want to take that away again? I'd say no... We don't have a problem reviewing, having a look at the processes or procedures, but fundamentally the legislation is there and it should stay there because it's protected people for a very long time."
Community leaders warn that changes would only incite violence and unrest across the country.
They're hoping that face-to-face meetings with federal MPs and senators in the nation's capital this week will make a difference.
Kenrick Cheah, the President of the Chinese Australian Forum, says the Racial Discrimination Act in its present form makes for a better society that's more cohesive and harmonious.
"What we don't want to see happen is casual racism being accepted as the norm. We don't want to see future generations of children living in a culture where they think it is okay to judge someone on the colour of their skin, make fun of other kids - I mean that's the main focus."
Khaldoun Hajaj, from the Arab Council Australia, says changes would affect every Australian, no matter what their ethnicity.
"18C and 18D are essentially functions that protect the rights of all communties. It's not an ethnic community-specific provision, rather it's actually an entire community-specific provision which actually is the basis of protecting Australia's harmony across all communities. We ask that the provisions be protected and, if anything, be bolstered."
Some politicians disagree.
Liberal Democratic Senator David Leyonhjelm says the Act is weak and doesn't curb racism at all.
"We don't need the sledgehammer of the law to go creating a polite society - we can do that without the law. If you really want to change minds then stopping them from speaking their mind isn't going to help. The productive way to deal with racists is to let them say what they're going to say as long as it's not incitement to commit violence. Let them say what they're going to say and then jump all over them."
President of the United Indian Association, John Kennedy, wants the Racial Discrimination Act to be strengthened instead.
He says those within the Indian community are too often the target of racism.
"This is happening in all of the communities, yeah, particularly the Indian community. There are lots of racial discrimination happening in the streets and workplaces."
A parliamentary inquiry into freedom of speech is due to report back in February.
