Two officers swooped on a busy market square at Westfield Hornsby in Sydney's north shortly before noon on Thursday following reports a young man was acting erratically, armed with a large carving knife.
A confrontation allegedly erupted when the man, aged in his 20s, lunged at the officers after being told to put the knife down.
The officers both opened fire on the man, who was reportedly injured in the stomach and arm.
Three women standing nearby, aged between 60 and 82, were also hit by police bullets or shrapnel.
"In a life and death situation police had to make a very critical decision on the actions they took," Assistant Commissioner Denis Clifford told reporters at the scene.
"It's alarming (that innocent bystanders were injured), but I wonder what may have happened had the police not intervened and stopped this person with a knife."
Asst Comm Clifford said the drama unfolded within seconds and an investigation will examine whether a taser could have been used.
"I must say, look, if Taser was available, a decision was made by those officers in a life-and-death situation," he said.
A stallholder at Hornsby farmer's market says he was selling bread to an elderly woman when she started screaming and said, "I've been shot," as police fired at the psychiatric patient.
Steve Middleton, who runs the Brasserie Bread stall, saw a male and female police officer draw their guns and fire when the knife-wielding man ran at them.
Mr Middleton said the elderly woman was hit by a bullet he believed bounced off a fountain in the pedestrian part of Florence St, where the markets were held.
"I was standing there with the lady, she started to scream," Mr Middleton told AAP.
"Then she says, 'I've been shot'. When I looked down there's certainly blood coming out from underneath her, dark red blood, but then some lighter really thick blood."
Mr Middleton said the attacker didn't say anything but was waving a large knife at the officers.
"He just had a little smile on his face, no shoes, tracksuit pants, baggy jumper."
Magdy Abdalla from the Java Lava Cafe on Florence St told AAP he heard gunshots and immediately fell to the floor.
"I saw some people running and I just had a look," he said.
"I found three older ladies on the floor bleeding, one injured in the leg, one in the arm and the other one maybe in the back."
Police have confirmed nobody was stabbed in the incident, despite earlier reports, with all four injured in the gunfire.
All four people injured are in stable conditions in hospital, where the offender remains under police guard.
Critical Incident officers have launched an investigation into the shooting, including why a Taser and other options available were not used by the officers.
Thursday's incident occurred metres from a restaurant that was the scene of a stabbing in March, which left one man dead and a woman seriously injured.
