Speaking at the Sydney Institute on Thursday night, Mr Costello said that anyone not prepared to accept Australian values, and who had citizenship of another country, should not remain an Australian citizen.
He said anyone who believes Islamic sharia law could co-exist with Australian law should move to a country where they might feel more comfortable.
Muslim leaders reacted by calling on the Prime Minister to censure the treasurer over the remarks.
However Mr Howard told Southern Cross Broadcasting that the treasurer’s comments were similar to some of his own and he should not be censured.
Asked why not, Mr Howard said: "Because what he said was fundamentally accurate."
Mr Howard said he supports multiculturalism if it simply means showing respect and tolerance for other people's cultures.
But he said Australia could not have a federation of cultures.
"Over the years, at its zenith, the more zealous multiculturalism base said that this country should be a federation of cultures," he said.
"You can't have a nation with a federation of cultures.”
"You can have a nation where a whole variety of cultures constantly influence and mould and change and blend in with the mainstream culture," he said.
Mr Waleed Aly, a council board member of the Islamic Council of Victoria, said the treasurer’s comments are part of an ongoing government theme.
He said the attitude only served to isolate Australian Muslims and make them feel "rejected" by their government.
"It seems quite clearly calculated at marginalising a part of mainstream Australia that's been part of mainstream Australia for 50 years, but suddenly it's some sort of hideous problem," he said.
"Australian Muslims have made a fantastic contribution to this country," he said.
The opposition Australian Labor Party said Mr Costello's comments about citizenship are just a political pitch to conservative voters on the eve of John Howard's 10th anniversary as prime minister.
Opposition foreign affairs spokesman Kevin Rudd described Mr Costello's comments as "an exercise in politics. It's not an exercise in real policy," he said.
Mr Rudd said the government should have already acted if it has recognised problems with citizenship.