There are reports he suffered head injuries and a cardiac arrest.
A Victorian Police spokeswoman has confirmed a prisoner has died, but failed to name the victim.
An ambulance spokeswoman says paramedics arrived at Victoria's high security Barwon Prison just before 1.20pm to find the man in a critical condition.
Two independent sources have said the man is Williams.
Underworld figure Mick Gatto said he had heard unconfirmed rumours about Williams having died but said that was all he had heard.
"I would rather let dogs lie," he told AAP.
"If you hear any more let me know but I would rather not comment on him."
Gatto said Williams was "certainly not" a friend.
Williams was serving a life sentence with a non-parole period of 35 years for ordering the murders of three underworld rivals and the failed conspiracy to murder a forth person.
He was found guilty of murdering Jason Moran, Lewis Moran and Michael Marshall.
He had one child, a daughter, with ex-wife Roberta Williams, a convicted drug trafficker.
Williams' father George was released from prison in June 2009 after serving a sentence for drug trafficking, while his mother Barbara was found dead in 2009 after an apparent overdose.
A Melbourne newspaper had reported on Monday that Victorian police had paid $8000 to allow Williams' daughter to attend a top private school.
