"I extend my hand in peace to Mahmud Abbas, elected president of the Palestinian Authority. On behalf of the state of Israel, we are willing to negotiate with a Palestinian Authority," he said.
"In a few years they could be living in a Palestinian state, side by side in peace and security with Israel," Mr Olmert told members of the Senate and House of Representatives.
President George W. Bush hosted Mr Olmert for a summit on Tuesday during which they discussed in length Olmert's plan to withdraw Israeli settlers and troops from vast areas of the occupied West Bank while retaining several large settlement blocs in the Palestinian area.
During the talks Mr Olmert praised Abbas as a "genuine" and "sincere" partner for peace.
Mr Olmert, during the Washington visit, has also reiterated Israel's demand that the recently formed Hamas government recognise Israel's right to exist and renounce violence as a precondition for any peace agreement.
"But no one can make it happen for them if they refuse to make it happen for themselves."
Middle East roadmap
The newly-elected prime minister repeated Israel's commitment to the US-sponsored roadmap to peace, which calls for a negotiated settlement with the Palestinians, but said Israel will not wait forever.
"Our deepest wish is to build a better future for our regions, hand in hand with a Palestinian partner," he said. "But if they refuse, we will not give a terrorist regime a veto over progress, or allow it to take hope hostage."
"Should the Palestinians ignore our outstretched hand for peace, Israelwill seek other alternatives to promote our future and the prospects of hope in the Middle East."
Mr Olmert's three-day visit to Washington has been aimed at reaffirming Israel's strong ties with its chief ally following the Israeli leader's election victory.
"Our countries do not just share the experience and pain of terrorism. We share the commitment and resolve to confront the brutal terrorists."
"We believe that terrorism not only leads to war but that terrorism is war," Mr Olmert told congress.
Mr Olmert also also urged the United States and the international community to act "now" to resolve the Iranian nuclear threat, which he qualified as "an existential threat" for Israel.
"A nuclear Iran is an intolerable threat to the peace and security of the world," he said. "Our moment is now. History will judge our generation by the actions we take now," the Isralei prime minister said.
Israeli raid
As Mr Olmert was giving his address four Palestinians were shot dead in Ramallah during an Israeli raid in the heart of the West Bank's administrative capital. The raid also resulted in the capture of a leading Islamic Jihad militant.
The Israeli killings in the West Bank came on the eve of Palestinian cross-party crisis talks in Ramallah designed to put a lid on factional violence that claimed two more lives in the Gaza Strip.
Palestinian foreign minister Mahmud al-Zahar accused Israel of seeking to torpedo Thursday's talks by escalating the violence in the Palestinian territories.
"We condemn this crime and demand that the international community exert pressure to make Israel stop its attacks," he said in a statement.
Around 60 other Palestinians were wounded during the Ramallah operation to arrest Mohammed al-Shubaki, the commander of Jihad in the Qalqilya area of the northern West Bank.