A new book by award-winning investigative journalist Ron Suskind, an excerpt of which was published on the Time website, claims that al-Qaeda were 45 days off staging the attack when it was called off by the number two in command, Ayman al-Zawahri, even though it could have killed thousands of people.
The informant said operatives had planned to use a small, easily concealed delivery system to release hydrogen cyanide into multiple subway cars.
The gas is similar to that used in Holocaust-era gas chambers.
US officials had already discovered plans for the device in the computer of a Bahraini jihadist arrested in February 2003, and they had been able to construct a working model from the plans.
The simple device, known as "the mubtakkar", meaning "invention" or "initiative", represented a breakthrough in weapons technology that "was the equivalent of splitting the atom", Suskind writes.
The White House has refused to confirm or deny the story.
However a New York Police Department spokesman said authorities had known of the planned attack.
"We were aware of the plot and took appropriate precaution," Paul Browne said.
According to the Time report, US President George W Bush was shown a model of the weapon in March 2003 and ordered alerts be sent through the US government.
When intelligence arrived that al-Zawahri had called off the attack, Mr Bush was concerned it was because they were planning something even worse, according to Suskind.
At least two of the journalist's sources recalled Mr Bush saying: "This is bad enough. What does calling this off say about what else they're planning? ... What could be the bigger operation Zawahri didn't want to mess up?" the author said.
The informant, who had become disgruntled with al-Qaeda's leadership, linked the organisation's top agent on the Arabian peninsula to the attacks, Suskind writes.
The man was later killed in a violent standoff with Saudi authorities.
The excerpt of Suskind's forthcoming book, The One Percent Doctrine, will appear in Monday's print edition of Time magazine.
Suskind is a former reporter for The Wall Street Journal.