No reason was given for the resignation which was announced by President George W. Bush at the White House.
Me Goss took over the Central Intelligence Agency in September 2004 on a mission to rebuild it after it faced a storm of criticism over the failure to detect the September 11, 2001 terrorist strikes and the information supplied to justify the invasion of Iraq in March 2003.
With Mr Goss sitting beside him, Mr Bush paid tribute to the "integrity" and "honor" of the outgoing director.
"He has got a five-year plan to increase the number of analysts and operatives, which is going to help make this country a safer place and help us win the 'war on terror'. He's instilled a sense of professionalism."
Mr Goss, who worked as a CIA agent himself for 10 years before going into politics, said he was leaving the agency working on an "even keel".
But lawmakers have said Goss is leaving the foreign intelligence agency in turmoil.
Mr Goss quickly became a controversial figure at the CIA for his hardline efforts to clamp down on leaks and create a more effective intelligence organisation.
The agency is under an ethics cloud after its number three official, executive director Kyle "Dusty" Foggo, became the subject of a CIA Inspector General investigation.
It was not clear however whether a link exists between the probe and the resignation of Goss.
Speaking on CNN television, former Congressman Bob Barr said the ethics charges, which reportedly involve ties to an alleged prostitution ring, may prove key.
"I think there's going to be more coming out. We don't know the whole story," Mr Barr said.
"It's starting to reach into the CIA," said Mr Barr who described the widening scandal as being "like a sore that's been festering, that could very well burst out and maybe that's a reach into the top levels of the agency."
Although he doubted that Goss was directly involved, Barr said that the scandal "certainly reflects on the leadership."
The agency has been demoralised by widespread criticism of its efforts to assess worldwide terror threats from Islamic terrorists, the insurgency in Iraq, the return of the Taliban in Afghanistan, and other global threats.
The CIA director's position, historically one of the most influential in Washington, was considerably less powerful during Goss's tenure because of reforms imposed by the US Congress to reshape intelligence operations.
Bush named John Negroponte as director of national intelligence in charge of all 16 US intelligence agencies, including the CIA.
Mr Goss's efforts to reform the agency have been criticized by opposition Democratic lawmakers and reportedly led to the resignations of key officials, including deputy CIA director John McLaughlin in November 2004.
His shakeup also led to the departure of the CIA official who oversaw foreign operations, deputy director of operations Stephen Kappes and several other senior clandestine service officers.
Mr McLaughlin, who led the CIA on an interim basis before Goss took over, said he was "surprised," by the resignation.
"I don't think there was any indication that this was coming," McLaughlin told CNN television.
"He came in at a difficult moment, and I think got off to a rocky start," he said.
"He came in at a time when a whole new structure had been created to manage the intelligence community, and the CIA in the process ... had to step back from some of the leadership responsibilities that it had traditionally exercised," McLaughlin said.
After September 11, the agency was also faulted for the flawed intelligence that the US administration used to justify the invasion of Iraq in Maarch 2003, to find President Saddam Hussein's weapons of mass destruction.
The weapons programmes were never found.
Mr Negroponte, as the new intelligence czar, now gives the president a daily intelligence briefing -- a duty which once fell to the CIA director.
Mr Goss also reportedly has clashed with Negroponte over policy questions, according to new accounts.
The outgoing director said however that the CIA has "made great strides on all fronts, from our field-forward approach, to welcoming record numbers of new employees who are today receiving better training than ever before."
He said the agency was "reinvigorating and enhancing" its analysis and "there is no question, that CIA remains the leader of cutting edge research and technology, which enables our security mission."