The decision to grant protection visas to 42 West Papuan asylum seekers has caused some sensitivity in Indonesia, with Jakarta withdrawing its ambassador to Australia and Indonesian President Susilo Bambang Yudhoyono saying the relationship between the two countries was full of “challenges”.
Mr Downer has told ABC radio that the government will take a cautious approach to rebuilding communication.
"We just let things plod along for a little while and gradually rebuild our communications. It is just best to take things cautiously and calmly and not rush into anything too much at this stage," he said.
But, in a positive turn for the strained relationship, Mr Downer added that President Yudhoyono did renew his commitment to the bilateral relationship with Australia.
"We can understand why they are upset but of course what we are trying to explain to them is that this has no implication for our recognition of Papua as a full part of the republic of Indonesia," he said.
Security games withdrawal
But tension has reportedly led Indonesia to withdraw as an observer from the Proliferation Security Initiative (PSI) exercise to be held around Darwin, the Northern Territory, this week.
PSI aims to limit the spread of weapons of mass destruction and materials through developing procedures for intercepting illicit shipments.
Mr Downer said he was not surprised by Indonesia’s snub as they weren’t expected to attend.
"I am not sure about a specific invitation. They may or may not have. But they certainly have not supported the Proliferation Security Initiative so if we gave them an invitation we would not have expected a very positive reply unless they changed their policy."
"We have lobbied them on PSI but they have felt up until now a bit uncomfortable," he added.
But Ron Huisken, an expert on international relations with the Strategic and Defence Studies Centre in Canberra, said the timing of the rejection was no coincidence.
"Probably a calculation was made in Jakarta that even observing this exercise might dilute the opposition that we are expressing to this Australian position," he told ABC radio.
"That observer status, my understanding is, has become very popular as sort of halfway house where you can show political interest in the initiative but also maintain political distance from it."
Serious problem
Australian Labor Leader Kim Beazley said the Howard government needed had to acknowledge the serious problems it had created.
"(Foreign Minister) Alexander Downer is out there saying that we can all sit back and relax on this one, it doesn't matter all that much," Mr Beazley told reporters in Melbourne.
"Well, it matters a lot. Border security is not something that you take lightly. (Prime Minister John) Howard and Downer have got to start to earn their money and get their relationship back on an even keel." He said.
Mr Beazley said the problem was that Indonesia had believed the Howard government's largely political claims in recent years that they, Indonesia, controlled Australia's borders.
"The Indonesians believed they had absolute control over it or wanted absolute control over it," he said.
Attorney-General Philip Ruddock says he doesn't believe talks with Indonesia on fighting terrorism and people-smuggling are in jeopardy despite a breakdown in relations.
Mr Ruddock says the two countries will continue to work on issues of mutual interest.