According to an official list seen by AFP, many of the inmates are accused of serious offences, including many involved in attacks that killed scores of Afghans and foreigners, with more than 150 of them on death row.
"The decision of the loya jirga has removed the last excuse and obstacles on the way to peace talks. We are on the verge of peace talks," said Abdullah Abdullah, who has been appointed by the government to lead negotiations with the Taliban.
"This is a very happy day. Based on the information I have the intra-Afghan talks would begin within two to three days after the release of the 400 Taliban prisoners," former president Hamid Karzai told the gathering.
The gathering recommended that any foreign nationals among the prisoners should be handed over to their respective countries.
The prisoners' fate has been a crucial hurdle in launching peace talks between the two warring sides, which have committed to completing a prisoner exchange before the talks can start.
The Afghan government has released almost 5,000 Taliban inmates, but authorities baulked at freeing the final prisoners demanded by the Taliban.
US Secretary of State Mike Pompeo pushed for the release of the detainees, while recognising the decision would be "unpopular".
The prisoners include 44 insurgents of particular concern to the United States and other countries for their role in "high-profile" attacks.
Five are linked to the 2018 attack on the Intercontinental Hotel in Kabul that killed 40 people, including 14 foreigners.
A Taliban militant involved in the massive May 2017 truck bombing near the German embassy in Kabul is also on the list, which includes a former Afghan army officer who killed five French troops and wounded 13 in 2012 in an insider attack.
On the first day of the gathering, lawmaker Belquis Roshan, a prominent women's rights activist, protested against the release of the prisoners, unfurling a banner that read: "Redeeming Taliban is national treason."