The brazen attack on the sprawling complex follows a spate of hit-and-run strikes on military personnel and officials, as the country struggles to complete a thorny political transition.
The attacks in the capital and in the country's south have been blamed on Al-Qaeda in the Arabian Peninsula (AQAP), which Washington regards as the jihadist network's most dangerous branch.
"At least 25 people have been killed," said a security official, shortly after the defence ministry had put the death toll at 20.
Six doctors, including a Venezuelan and two from the Philippines, and three Yemenis, along with five patients, including a judge, were among the dead, medical sources said.
The were at a hospital that lies within the ministry's complex, which bore the brunt of the attack.
"A car bomb driven by a suicide bomber forced its way into the western entrance of the ministry complex," a security official told AFP.
"It was followed by another car whose occupants opened fire at the complex of buildings," he said.
The attack comes as Defence Minister Mohammed Nasser is heading a military delegation on a visit to the United States.
The ministry said gunmen occupied the hospital after the explosion, but security forces had regained control of the building.
"The assailants took advantage of some construction work that is taking place to carry out this criminal act," it said without elaborating.
A security source said another apparently coordinated attack had been launched and a gunfight was raging outside the complex early afternoon.
State television aired gory footage of torn up bodies of what it said were suicide bombers, strewn in hospital's yard amid charred vehicles.
It also showed immense destruction within the hospital as well as floors stained with blood.
The channel broadcast appeals for blood donations at hospitals treating the wounded.
Plumes of smoke billowed across the ministry complex, situated on the edge of the Baba al-Yaman neighbourhood, immediately after the blast and as a firefight erupted.
"I heard a series of explosions, and then an exchange of fire," a wounded soldier told AFP.
President Abdrabuh Mansur Hadi visited the hospital where his 90-year-old brother, Ahmed, was apparently an in-patient.
He ordered a round-the-clock investigation into the attack, state television said.
Yemen has been going through a difficult political transition since veteran president Ali Abdullah Saleh was ousted in February 2012 after a year of deadly protests against his 33-year rule.
The transition aims to culminate in a new constitution and pave the way for parliamentary and presidential elections slated for February 2014, but it still faces many hurdles.
There are growing demands for the secession of the formerly independent south, in addition to on-off fighting in the far north between Shiite rebels and hardline Sunnis.
A national dialogue that opened in March was originally due to close on September 18, but it is yet to conclude.
Yemen is also battling AQAP militants.
AQAP often attacks members of the security forces, despite suffering setbacks in a major army offensive last year and repeated US drone strikes targeting its commanders.
In an attempt to crack down on hit-and-run attacks, Yemeni authorities last week imposed a temporary ban on motorbikes in the capital to prevent shootings that have killed dozens of officials.
Two gunmen on a bike killed last week a Belarussian defence contractor and wounded another as they left a Sanaa hotel.