In a statement, the defence department said the soldier received a single gunshot wound on the afternoon of Friday 21 April and was taken to a United States military hospital where he later died.
It is the first death of a member of the Australian armed forces in Iraq since the US led invasion in 2003.
The Department has not given details of what happened to the man, but that his death was not combat related and no-one else was hurt.
Defence Minister, Brendan Nelson said the shooting incident did not involve a confrontation with enemy forces.
He added the soldier was an experienced marksman with particularly good rifle skills.
It is unclear whether the bullet came from the soldier’s own weapon but Dr Nelson ruled out the possibility of suicide.
"The soldier was simply handling his weapon and maintaining his weapon as soldiers are required to do, and for some unexplained reason the firearm discharged and the bullet unfortunately entered the soldier's head,” he said.
"Several hours after the injury, despite receiving the best of medical care, he unfortunately passed away."
Army Chief Lieutenant General Peter Leahy said the accident happened inside the Australian barracks in Baghdad's secure green zone.
Tributes flow
Dr Nelson said the soldier was "highly regarded by his colleagues, he was highly regarded by his commanding office.
He was serving with Australia's Operation Catalyst Security Detachment in Baghdad, which is working on the reconstruction of Iraq.
Dr Nelson said he was seen as one of the most outstanding soldiers who served in the security detachment.
He described the man's death as a tragedy, but also said it was a risk faced by Australia's troops who had served in Iraq.
"The entire Australian Defence Force community is saddened by this loss and offers their sympathy to the soldier's family," the ADF said.
Speaking in the Solomons Island capital, Honiara, foreign minister Alexander Downer said he is deeply saddened by the death.
"I feel very sad about it," he said.
"My condolences go out to the soldier's family, this is the first Australian member of the Australian Defence Force who has been killed in Iraq."
The RSL says the death is a tragic reminder of the dangers faced by all military personnel serving overseas.
Bill Crews, RSL national president, has offered sympathies to the soldier's relatives.
Major-General Crews says Australian military personnel have a difficult job and face considerable risks in the line of duty.
Young family
General Leahy says the soldier was a member of the 3rd Battalion of the Royal Australian Regiment, based at Holsworthy outside Sydney.
Army officials had made contact with the man's family, which includes a wife and child.
The soldier is understood to have family based in Victoria.
His body is expected to be returned to Australia within two days.
The soldier's identity has not been released at the family’s request.
A military and coronial inquest will be launched into the death.
Australia currently has 460 troops serving in southern Iraq.