The sixth-ranked Australians now need to beat Brazil, who opened their campaign with a 3-0 win over Jamaica, to stay on course for a top-two finish in the group and an automatic spot in the last 16.
"We've got to look at both ends of the field," coach Milicic told reporters.
"We created enough chances to score a few more. But in the end, defensively, I felt we gave away too many chances and, again, that's another area we are going to have to address.
"We've got three days now to regroup and look forward to the Brazil game."
Captain Sam Kerr, who scored Australia's goal when she followed up on her own penalty miss, said the team were frustrated but there were still plenty of positives to build on.
"It's obviously disappointing to lose but we have still got two games and we are definitely not out," said Kerr. "It's the first game. Get the jitters out, we'll focus on Brazil."
Milicic also added that while a draw against Brazil could mean they would have to look at qualifying for the knockout phases as one of the four-best third placed teams, they were not going to get eliminated without a fight.
"We're not going to pack it in after one defeat. There are still two games to go," he said.
"We have the potential to get through, we've just decided to go the long way, the hard way about it.
"Maybe that's just the Australian way. We'll stare it straight in the eyes and go for it."
(Reporting by Greg Stutchbury in Wellington; Editing by Nick Mulvenney)
