An Islamic community leader says calls from the prime minister for leaders to speak out against extremism aren't justified and that Mr Abbott shouldn't tell them how to do their job.
The prime minister has reiterated calls for Muslim community leaders to speak out against extremism.
Mr Abbott was asked about the government's response to combating home-grown terrorism during parliamentary question time this week and responded that Islamic community leaders needed to speak out against extremism and people needed to remember their citizenship pledge.
"It is not enough to say it, people have to mean it," he said.
"It's not enough to mean it once, you have to mean it always. You have to live it. "
The comments mirrored Mr Abbott's national security statement made on Monday, in which he warned the threat from terrorism at home and abroad was much greater and becoming harder to combat.
But Islamic community leaders said they were doing all they could to speak out against extremism.
"We are doing our job so don't come...and tell me to do my job...that's really offending us," the Australian Arabic Association of Western Sydney's Sheik Adid Alrubai told SBS.
"Actually we say it, we work. If he's not listening and not seeing the real picture that's his problem, but we're doing our job in very good ways."
Muslim group Hizb ut-Tahrir said yesterday's national security statement made Muslims "scapegoats."
In a statement the group said calls for leaders to do more "is a slap in the face of those Muslims who have for years already been saying, 'Islam means Peace'."
Multicultural community leaders gathered in Sydney's Blacktown on Tuesday to "Stand Together."
There was concern recent events, including the Sydney siege, could be dividing the community.
"If we don't build an inclusive society we will always have the problems of isolation and terrorism simple as that," said Mayor of Blacktown, Stephen Bali.
The Vice President of SydWest Multicultural Services, Om Dhungel, said it was important members of the community took responsibility and ownership for what happened in Australia.
"We have to stand up together and don't allow anything to happen to this wonderful country," he said.
- With AAP