Nishino, who spent 10 seasons in charge of Gamba Osaka and became the JFA's technical director in 2016, will have little time to prepare for the tournament in Russia, where Japan are drawn in Group H alongside Colombia, Poland and Senegal.
Japan have three friendly matches scheduled before opening their World Cup campaign against Colombia on June 19.
Halilhodzic, who took over in March 2015, led the team to their sixth straight World Cup finals after they finished top of Asian qualifying Group B ahead of Saudi Arabia and Australia.
However, a draw with Mali and defeat to Ukraine in friendly matches last month did nothing to ease concerns about the team's form this year and increased the pressure on the 65-year-old.
"After we got into the World Cup we set up some friendly matches but finally we decided to fire Halilhodzic two days ago," JFA President Kozo Tashima told a packed news conference on Monday.
"We did not decide this just because of the results of the games or because of the reporting but of course we listened to the player's voices.
"After the matches against Mali and the Ukraine, the relationship and trust between the coaches and the players had become diluted."
Tashima added that the Bosnian had not been expecting the decision.
"When I told him directly, it was my impression that he was surprised. He didn't expect this situation so he showed a little bit of being upset and angry and he wanted to know the reason why," Tashima said.
Nishino, who made 12 appearances for his country as a player, is most famous in Japan for coaching the national side to a victory over Brazil at the 1996 Olympics in Atlanta.
He also led J-League side Gamba Osaka to the AFC Champions League title in 2008.
"We wanted the new manager to be promoted from within the JFA because we have only two months before the World Cup," added Tashima.
"So we decided on Nishino as the new manager because he knows the team the most having seen them from the inside."
Halilhodzic is the third coach to depart from one of the five Asian sides heading to Russia after they had secured qualification.
Australia and Saudi Arabia also changed coaches soon after they booked their places, while South Korea replaced German Uli Stielike with Shin Tae-yong for the final two matches of qualifying.
Japan's next warm-up game for the World Cup is against Ghana on May 30.
(Reporting by Jack Tarrant; Editing by John O'Brien and Christian Radnedge)
