About 40 protesters from the Love Makes A Way movement staged a sit-in in the building's entry foyer for more than five hours on Wednesday before security intervened.
They were peacefully escorted one by one from the building, each singing the hymn Were You There.
More than 100 members of the group, which takes its campaign to end Australia's "inhumane" asylum-seeker policies to locations around the country, have been arrested over the past year.
Spokesman Craig Farrell, a Salvation Army captain, said the group wanted to see a change of heart from both major parties on the treatment of asylum seekers.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
They also want children and families released from offshore processing centres and into the community.
"We won't stop until all children are released," Mr Farrell told reporters.
The Greens declared support the protest, saying Australia's current refugee policies are "devoid of compassion" and that protesters should be commended.
"This action demonstrates how the Australian community feel about the Government’s approach to some of the most vulnerable people, particularly children," said Greens immigration spokesperson Sarah Hanson-Young. "They need our help, not our condemnation or cruelty."
"It's time to listen to these voices," she said.
Mr Farrell urged the prime minister to adopt a more compassionate view on treatment of children and families in offshore processing centres.
"I would hope that perhaps he might have a second come-to-Jesus moment and change his heart," he said while asking for Labor to do the same.
Mr Farrell acknowledged Parliament House security had been respectful of the protest, but the group would have like to have stayed longer.
