The group were on the plane when it appeared to suffer catastrophic engine failure moments after taking off from Essendon Airport on Tuesday morning.
Pilot Max Quartermain, and US citizens Greg Reynolds De Haven, Russell Munsch, Glenn Garland and a fourth American whose identity is not yet known, were all on board the Beechcraft Super King Air twin-engine aircraft.
Mr Reynolds De Haven's family said on social media the men, who had already played golf at exclusive courses Royal Melbourne and Kingston Heath as well as in New Zealand, were on the holiday of a lifetime.
"Dear friends and family, my handsome athletic big brother was killed today in a plane accident while on his 'once in a lifetime' trip to Australia," Mr De Haven's sister Denelle Wicht posted.
"It was a charter flight with 2 of his friends flying to another island to play golf."
Mr Munsch was a lawyer specialising in bankruptcy and banking law and a shareholder of Munsch Hardt, a Texas law firm he helped start in 1985.
It's been reported the men were travelling with their wives, who had planned on seeing the Great Ocean Road while their husbands flew to King Island for golf.
King Island's golf courses have become an international destination.
The Herald Sun reports entrepreneur and investor Glenn Garland is believed to be among those killed in the crash.
Mr Quartermain, 63, ran Corporate and Leisure Aviation with his wife Cilla.
Bas Nikolovski, chief executive of Australian Corporate Jets, told AAP Mr Quartermain was "one of the most experienced pilots".
Victoria Police say they have spoken to and are providing support to the immediate families.
The plane issued a mayday before smashing into the Direct Factory Outlets in Essendon at around 9am, an hour before opening time.
Staff were preparing to open stores but no one was injured by debris that fell onto nearby freeways, or when a fireball engulfed the plane and burned through the stores and into a car park.
But many suffered shock as they searched frantically for colleagues.
No one on the plane survived.
"Looking at the fireball, it is incredibly lucky that no one was at the back of those stores or in the car park of the stores that no one was even hurt," Police assistant commissioner Stephen Leane said.
Ash Mayer, a storeman at The Good Guys, felt the crash reverberate.
"We felt just everything shake and a massive explosion and a fireball go up," he told AAP.
"We just knew this was bad and there was nothing we could do.
"One of the boys actually saw it go down and he's now left because he can't deal with it."
The plane, owned by Myjet, had been hired by Corporate and Leisure Travel, a company owned by Mr Quartermain.
Australian Corporate Jets CEO Bas Nikolovski says news of the accident "smashed him for six".
"Every aspect of my body had goosebumps" Mr Nikolovski told AAP.
Emergency Management Commissioner Craig Lapsley said witnesses had been treated for shock and trauma.
Essendon Airport was closed, the DFO building is being investigated for structural flaws and the freeways around the crash site were partly closed due to debris from the crash.
The outlet won't reopen until Thursday.
The Australian Transport Safety Bureau has taken over the investigation, while police have spoken to the immediately family of all victims.
It's the wost aviation crash in Victoria since 1978 when a light plane crashed into an Airport West home, killing six people.
Victorian Premier Daniel Andrews has called the crash a "desperately sad day" for the state.
He says it's the worst civil aviation accident Victoria has seen for 30 years.
"We are currently reaching out to (victims') families to provide them with the support that they need to try and comfort them at what must be such a horrible moment," he says.
"Beyond that, there are a number of people who've witnessed this terrible accident and they've been given the support, the psychological first aid that they need by Ambulance Victoria."
Premier Andrews praised the efforts of Victoria Police and the 90 firefighters who attended what he called a "complex and unpredicatble fire".