"We are deeply concerned that the Morrison government has done yet another dodgy deal that opens the door to an increased number of temporary workers being exploited when we should be prioritising hiring and training of local workers," ACTU president Michele O'Neil said.
Labor is yet to decide if it will support the agreement and is facing pressure from unions to oppose it.
Labor Senator Katy Gallagher would not be drawn further.
"There is no hesitation it's just that I'm not here to endorse anything, I'm a member of a caucus and caucus has its processes as does the parliament," she said.
Trade Minister Simon Birmingham said there would no exemptions from requirements to conduct labour market testing.
He urged politicians to consider the benefits of deepening the relationship between the two countries.
"Why should working holiday-makers from Indonesia not be welcome in Australia when we’re welcoming in working holiday-makers from other parts of the world?" he said.
The Indonesia agreement is one part of the government's push to increase the supply of short-term foreign workers to regional areas, despite concerns about exploitation.
Working holidaymaker visa program expansion
Talks are underway with 13 countries to allow citizens to apply for working holiday visas to add to the existing 44 with deals in place.
Immigration Minister David Coleman announced on Wednesday a 20 per cent increase in the number of working holiday makers granted a second-year visa.
"They got that visa because they worked and committed to working in Regional Australia," Mr Coleman told reporters at a farm in South Australia.
That brings the number of young people scattered across regional centres to 43,000.
"A big focus of our government's immigration program is delivering immigration to the areas of regional Australia where it is most needed," the minister said.
The National Farmers Federation, which has been calling for a separate agricultural work visa, welcomed the increase.
"We still think there's more work to do so we're very keen to get backpackers and make sure they can continue to work on farms but we need more measures that are going to address what we think is pretty severe labour shortage in rural and regional Australia," NFF CEO Tony Maher told SBS News on Wednesday.
"Backpackers come and stay for two or three years and it’s a great cultural exchange, they can earn some money that but we do need to get measures that allow people to stay for a longer period of time."
The tourism industry said accommodation providers were benefitting from the increase in backpackers completing three months of regional work to be eligible for a second year in the country.
“Working Holiday Makers are an important component of the workforce for Australia’s accommodation providers and this can be particularly the case in regional areas where a sufficient local workforce may not be available,” Tourism Accommodation Australia CEO Michael Johnson said.
The countries engaged in talks with the government on reciprocal working holiday-maker visa arrangements are Andorra, Brazil, Croatia, Fiji, India, Latvia, Lithuania, Mexico, Monaco, Mongolia, Philippines, Solomon Islands and Switzerland.
Changes to the program recently announced will also mean eligible visa holders will be able to stay a third year.
There are concerns the exploitation of migrant workers is rife in the agricultural sector with researchers estimating the scale of unclaimed wages totals more than a billion dollars.
Mr Birmingham said the Home Affairs Department was cracking down on farms doing the wrong thing.
"We had 89 people recently, under Operation Battenrun, who had their visas cancelled because of inappropriate conduct in relation to labour-hire and so on."
Additional reporting by AAP