Search and rescue teams met with grim scenes of bodies strewn across the beach at Obock, a port town down the coast from Godaria where the vessels had set sail on the Horn of Africa nation's northeast coast on Tuesday.
An AFP journalist also saw cadavers in the water before teams placed them in white body bags lined up on the beach.
The International Organization for Migration (IOM) said the boats capsized half an hour into their voyage.
The IOM's chief of mission in Djibouti Lalini Veerassamy told AFP the death toll had reached 38 on Wednesday.
"This tragic event demonstrates the risks that vulnerable migrants face as they innocently search for better lives," she said in a statement.
Sixteen people were rescued following the sinkings, with one survivor telling Djiboutian authorities there were 130 people on his boat.
The number of passengers on the second vessel remains unclear, as do the nationalities of those aboard.
Located across the Bab el-Mandeb strait from Yemen and next to volatile Somalia and Ethiopia, Djibouti has in recent years become a transit point for migrants heading to seek work on the Arabian Peninsula.
Yemen arrivals on rise
Named after the town, the Obock region is unusual in that it sees people passing in both directions -- boatloads of Yemeni refugees fleeing war cross vessels carrying African migrants seeking better opportunities.
In 2017, 100,000 migrants arrived in Yemen, with many aiming to head north to find work in Saudi Arabia and its neighbours.
"The number of new arrivals arriving in Yemen has been continuously increasing since 2012, despite the deepened insecurity and violence following the war that erupted in March 2015," the IOM said in its 2018-20 regional migration response plan.
"Arrivals peaked in 2016, when over 117,000 arrived in Yemen," it added.
This migration continues even though Yemen is facing one of the world's worst humanitarian crises.
UN officials say 80 percent of the population -- 24 million people -- are in need of aid and nearly 10 million are just one step away from famine.
IOM said that most of the journey migrants take to Yemen is by foot, walking across the scorching desert regions of eastern Ethiopia, Djibouti and northern Somalia.
Once they arrive in Yemen they could face torture, blackmail, sexual abuse or forced labour.
In 2017, some 2,900 people, mostly Somalis and Ethiopians, fled the dire conditions they had found in Yemen back to Djibouti.
The sea crossing itself has repeatedly proven perilous.
Last year, at least 30 migrants from Somalia and Ethiopia believed to be headed for Djibouti drowned when their boat capsized off Yemen amid reports of gunfire being used against those on board.
In August 2017, dozens of migrants from Somalia and Ethiopia died after human traffickers forced them off two Yemen-bound boats and into the sea.
