"As we move in this new period, we really are looking to transfer responsibility ... we just want to make sure that there is no seam between what we are doing politically and what we are doing militarily," she said.
Both Ms Rice and Mr Rumsfeld are expected to meet with Iraqi prime minister-designate Jawad al-Maliki, just days after he was nominated for the position in a move that ended a four-month long political deadlock.
Earlier, Mr Rumsfeld said Iraq's new leaders should begin discussions on the future of the US military in the country, as well as the pace of the US handover of security responsibilities.
"The question of our forces' levels here will depend on conditions on the ground and discussions with the Iraqi government which will evolve over time," Mr Rusmfeld told reporters.
After meeting with the defence secretary, coalition ground forces commander General George Casey said he is still on his "general timeline" for recommending further reductions in the 132,000-strong US force in Iraq.
Mr al-Maliki on Tuesday said he expects to have his cabinet line-up ready for approval in two weeks, with hectic lobbying for key ministerial posts underway.
He also told CNN he thinks US troops could start pulling out of Iraq within 18 months if Iraqi security forces are in shape.
The Pentagon's press secretary said US President George W Bush asked Mr Rumsfeld "to show our support for the new government".
Mr Rumsfeld's lightning visit comes at a time of public discontent with the war and his position in particular, with a group of retired military generals earlier this month calling for his resignation.
The group included former commanders in Iraq.
The defence secretary is set to meet the top US military commanders in Iraq before the talks with Iraqi political leaders in the Green Zone, the fortified compound in central Baghdad that houses the US embassy and government offices.
His visit comes as violence continues in Iraq, with at least 15 killings on Tuesday across the country.
A judge was one of three people killed in Baghdad, and another six employees of a mobile phone company was shot dead west of the holy city of Karbala.
Seven car bombings and numerous shootings left 20 people dead and about 100 wounded in the capital on Monday, and the defence ministry said eight people have been arrested on suspicion of involvement in the car bombs.