"What is happening between Canberra and Manus has ceased to be government and has become terror," Mr Bandt told a rally in Melbourne's CBD attended by hundreds of people on Saturday.
"If the definition of terror is to use violence and threaten people's lives for political purposes, then Peter Dutton is a terrorist."
More than 600 refugees have barricaded themselves in the mothballed detention centre, which closed on Tuesday.
