Futsalroos captain Greg Giovenali got the leveller after Katsutoshi Henmi's opener, only for Kaoru Morioka to score and give Japan a half-time lead.
Kazuhiro Nibuya 34th minute effort was the only goal of the second half as Japan topped Group D with a perfect record.
Advertisement
“We defended well as a team as you come to expect from Australia, I have to say that Japan were deserved winners but I’m pleased with our performance,” Australia coach Rob Varela said.
“Japan are one of the best teams in Asia if not the world and they make you work hard in defence.
“While you’re never happy with a loss I’m delighted with the fact that we gave a lot of players some game time tonight and that they all followed the goals we’d given them at the outset.”
Australia face Thailand on Wednesday evening in what is essentially a 2016 FIFA Futsal World Cup play-off with the top five sides qualifying for the tournament in Colombia.
The other last eight battles are: Uzbekistan v Iraq, Iran v Kyrgyzstan and Japan v Vietnam.
The four winners progress to the semi-finals while the beaten quarter-finalists play off for fifth spot.
