Their activity - first reported by the Age and Sydney Morning Herald - drew the attention of local police and intelligence officers from Victoria’s Counter-Terrorism Command.
Labor’s Home Affairs spokesperson Kristina Keneally has questioned whether the government is taking strong enough action to address the concerns.
"The intimidation tactics of the National Socialist Network in the Grampians is unfortunately not new in Australia," she told SBS News.
"These groups have been proliferating across the country for a number of years."
Mr Dutton said he "categorically" condemned the activity of the right-wing group.
“I categorically condemn any such actions," he told SBS News in a statement.
He added his government had given the Australian Federal Police and ASIO more funding and “constantly raised with them the threat of lunatic fringe groups like this one.”
The domestic spy agency ASIO has said far-right extremism now accounts for up to 40 per cent of its counter-terrorism workload, up from 10 to 15 per cent before 2016.
AFP commissioner Reece Kershaw has also spoken of a “steady increase” in operations directed towards right-wing extremism.
Treasurer Josh Frydenberg - speaking about anti-Semitism as Australia marked Holocaust Remembrance Day - recently urged people to "take on hate wherever we see it".
Senator Keneally said the federal government needed to take up his call to arms.
"The question is, will Scott Morrison listen to his Treasurer, the ASIO Director-General and countless other experts and finally take action on this issue?" she said.
'It's not surprising to see this'
The group has been identified as the far-right National Socialist Network by the Sydney Morning Herald and the Age.
The images show them burning a cross – a ritual associated with the Ku Klux Klan – and in another photo presenting the Nazi salute to the camera.
Terror expert Greg Barton said the blatant sighting of the group was not "surprising" given Australia's reported rise in right-wing extremism.
"They are deliberately positioning themselves as a white-supremacist fascist group that is expressing hatred towards everyone outside of their circle," he told SBS News.
He said Australia's security agencies were taking the threat seriously, but he believed there was a question to be asked about whether "in aggregate" the government is doing all it can in response.
"Whether we shouldn’t change legislation to make that sort of incitement of hatred and intimidatory behaviour illegal," he said.
"We currently have situation where people have to actually engage in an act of violence before we can deem it to be criminal activity.
"We seem to be in a bind where we can’t stop this sort of behaviour because it’s not breaking the law."
A parliamentary inquiry into extremism in Australia started late last year after a push for the government to look at the rising threat specifically posed by right-wing extremists.
Treasurer Josh Frydenberg, who is Jewish, said the behaviour has no place in our "freedom loving democracy".
“What is very clear is we need to constantly be vigilant against extremism and intolerance and racism in all its forms – I know our intelligence and law enforcement agencies are very seized of that issue," he told reporters.