Andy Coulson had faced months of calls to quit over allegations - which he denies - that while editor of the News of the World he encouraged reporters to hack the cellphone messages of royals, politicians and celebrities.
"I stand by what I've said about those events but when the spokesman needs a spokesman, it's time to move on," Coulson said in a statement announcing his resignation.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"Unfortunately, continued coverage of events connected to my old job at the News of the World has made it difficult for me to give the 110 percent needed in this role," he added.
The 43-year-old - a key figure in Cameron's inner circle who helped propel the Conservative leader to power in elections in May 2010 - later left Downing Street without speaking to reporters.
Cameron said he was "very sorry" Coulson was going but added that he could "understand that the continuing pressures on him and his family mean that he feels compelled to do so".
"Andy has told me that the focus on him was impeding his ability to do his job and was starting to prove a distraction for the government," he said.
Opposition Labour leader Ed Miliband said: "He's now done the right thing but he should have done it earlier. I think it raises real questions about David Cameron's judgment that he hung on to Andy Coulson for so long."
Stellar rise
Coulson had a stellar rise from humble beginnings, working his way through the ranks of Australian-born tycoon Murdoch's British tabloid stables and becoming editor of the News of the World at just 34.
But it is now the second time that the phone-hacking saga has forced him to resign from a top job.
The affair has refused to die down since the News of the World's royal correspondent and a private investigator were jailed in 2007 for conspiracy to access mobile phone messages involving Princes William and Harry.
Coulson resigned as editor of Britain's biggest selling newspaper, although he insisted he knew nothing about the scandal.
He signed up as Cameron's communications director later in 2007, with the state-school educated former newspaperman bringing a uniquely populist perspective to the Conservative team.
Case flares up
Police however revived their investigation in September and interviewed Coulson as a witness last year after a journalist said Coulson had encouraged him to hack voicemails.
They said in December there was no evidence to bring charges after several witnesses refused to testify.
The case flared up again earlier this month when Ian Edmondson, the paper's assistant editor for news, was suspended over what it called "serious allegations."
Prosecutors said a week ago they would review police material after actress Sienna Miller sued the paper for harassment and for breaching her privacy.
A document lodged at the High Court in London links Edmondson with the interception of voicemail messages from Miller's phone.
Accusations of burying bad news
As speculation mounted over Coulson's replacement, the outgoing media chief immediately faced accusations of trying to bury bad news.
His resignation came as former premier Tony Blair made a second appearance at the Iraq war inquiry and as the fallout continues from the departure of opposition finance spokesman Alan Johnson on Thursday due to personal problems.
"The Coulson issue must be urgent to (be) announced mid-TB (Tony Blair) Iraq inquiry evidence," Blair's former media chief Alastair Campbell said in a sarcastic Twitter message.
"The issue will move straight from Coulson's situation to Cameron's judgment."
