The Indonesian Ambassador was recently withdrawn in protest against Australia's decision to give temporary protection visas to 42 refugees from the Indonesian province of Papua.
Mr Downer met with his Indonesian counterpart, Hassan Wirajuda, for more than three hours in Singapore last night.
Mr Downer says Australia will not say when it would like the Ambassador to return.
"There's no doubt the Indonesian ambassador will be coming back.
"The timing of that and how they handle that is a matter of domestic sensitivity for them and I will leave all of that to them."