"They shouldn't do it because it would really be a sign that they are not prepared to actually make diplomacy work," said Ms Rice.
"In terms of the next phase, if negotiations have been exhausted, we have the votes, there is a resolution sitting there on the Security Council, we'll vote it," she said.
Ms Rice declined to give a timeline for such action but said "the Iranians are digging their own hole of isolation deeper and deeper."
Her remarks came as Iranian delegates failed to show up at the headquarters of the UN atomic watchdog, the International Atomic Energy Agency, in Vienna for a scheduled meeting to discuss Tehran's plan to resume atomic fuel research, which was suspended two years ago.
Tehran has described the plan as a "program for peaceful nuclear energy". It has said it will re-start the program on Monday.
The plan was revealed in a letter Tuesday to the IAEA, prompting the world nuclear watchdog to seek "clarifications".
The European Union and the United States fear Iran's civil nuclear program could provide cover for developing a nuclear bomb. Tehran denies the accusation, insisting the program is designed solely to meet its electricity needs.
In September 2005, the IAEA Board found Iran in non-compliance with the nuclear Non-Proliferation Treaty, paving the way for the matter to be referred to the UN Security Council if Iran does not halt nuclear fuel work and co-operate fully with an IAEA investigation.
Russia and China, which could veto any US move against Iran in the Security Council, abstained at the IAEA board meeting, where only Tehran and Washington's arch-rival Venezuela voted against the resolution.
Ms Rice said Thursday that the United States would strive to gather wide support from the international community on any planned UN action against Iran.
"We would like to bring as many people to that assessment as possible," she said.
"People want the Iranians to decide whether or not they are prepared to live with a civil nuclear structure that does not raise proliferation risks ... and when it is clear, as it is becoming clear, that they are not prepared to do that, I think you'll have very strong consensus," Ms Rice said.
US State Department spokesman Sean McCormack said there were different ways the Security Council could act against Iran.
"But I think that, certainly if the Iranians are referred to the Security Council, that the Security Council would be prepared to take some action. I'm not going to prejudge what that action may or may not be," he said.
Germany and France, which together with Britain make up the EU-3 team negotiating with the Iranians, had also warned that Iran's latest announcement could delay talks on the issue due to begin again in Austria on January 18.
The three nations are trying to wean Iran off its nuclear ambitions with economic and security incentives.
The Europeans and the United States are backing a compromise offer from Moscow to enrich Iranian uranium on Russian soil, guaranteeing Iran's access to nuclear fuel but for peaceful purposes only.
Iranian officials are to discuss the proposal with Russian officials on Saturday.