Mr Kozlov, known for his strict regulation of the country's banks, was shot with an automatic weapon about 8.30pm local time, Moscow's deputy prosecutor Vladimir Ponevezhsky said.
"An assassination attempt took place on the deputy chairman of the central bank, Andrei Andreyevich (Kozlov), who was injured and is in a serious condition in hospital. His driver was killed," he said.
Surgeons operated on Mr Kozlov for head injuries but he did not survive.
Moscow's general prosecutor, Yury Syomin, said the motive for the shooting was unclear, with Mr Kozlov's work in the central bank one of several possibilities being considered.
"We are looking into all possible versions, including his professional activities, a mistaken target, personal relations," he said.
The attack took place on a secluded road outside the Spartak Moscow football club in the northeast of the city following a football match involving employees of the central bank.
"After the end of the game, Kozlov got changed and left the building with his driver-bodyguard, when an unknown attacker ran up to him and opened fire," said a witness.
"Kozlov immediately fell to the ground. His guard tried to run away, but was also shot," he said.
Kozlov a tough operator
Some commentators speculated that the attack may have been linked to the tough line taken by Mr Kozlov at the central bank, where he was responsible for licensing commercial banks.
"Of course it could be connected with his professional activities," said Anatoly Aksakov, the deputy head of the Russian parliament's committee on credit organisations and the financial markets.
"He had a very tough position in relations to banks which ... laundered money and engaged in other illegal activities," he said.
Mr Aksakov mentioned Mr Kozlov's involvement in deciding which banks to allow to take part in a national deposit insurance scheme as one incident when Mr Kozlov made enemies in the sector.
Since the start of the year Mr Kozlov, who is responsible for the supervision of the banking system, rescinded the licenses of 44 of about 1,200 banks, mostly for engaging in illegal activities.
"Kozlov is one of the bravest and most active people who took responsibility and tried to change the banking system, so that it functioned," said Alexander Murychev, the president of the Association of Regional Banks of Russia.
Mr Kozlov, 41, became deputy chairman of the Central Bank in 2002, according to his biography on the organisation website. He was married with three children.