Treasury secretary John Fraser says it would be a "tough task" to rely only on governments to fund new road, rail and ports.
Mr Fraser told a Senate estimates hearing on Friday that infrastructure and housing would be key drivers of the national and global economy after a lull in recent decades.
"There's a lot of talk and we need to get on and start digging things," Mr Fraser said.
Greens senator Janet Rice asked whether the budget papers were at odds with this idea, as they showed federal payments to the states for infrastructure were due to fall from $7 billion in 2015/16 to under $4 billion in 2019/20.
Senior Treasury official Michael Brennan said it reflected "milestone payments" being made to the states in the next few years from to the asset recycling plan.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
However, Mr Fraser said if it was left purely to government to fund infrastructure "it will be a tough task".
"Where infrastructure can really blossom is the private sector's interest in [it]," he said.
He said there was an encouraging growth in interest from investors in infrastructure, which took between 12 and 15 years to bring a return.
Not only private sector funds but retail investors were looking at infrastructure, which could be due to low interest rates.
"People are starting to seek out other investment opportunities," Mr Fraser said.
He said housing construction would also be an important element in driving economic growth, as well as consumption.
But he did not expect an oversupply of housing.
"The worry is keeping up the land release and the infrastructure."
