Nelsinho's fate was sealed after Kobe slumped to a third consecutive loss with a 1-0 defeat by FC Tokyo on Sunday.
"Unfortunately, I couldn't achieve what we were aiming for, but I was able to see the team grow a lot toward earning a title in football," Nelsinho said in a statement.
"As a professional, I feel greatly responsible for not being able to live up to everyone's expectations."
The 67-year-old took charge of Kobe in 2015 and guided them to a 12th-place finish in his first year. Kobe were seventh in the standings last year.
Interim manager Yoshida called for unity from the squad.
"I have no experience as manager, so I told the players that we should work together as one," the 40-year-old was quoted as saying by Kyodo News Agency.
Yoshida's first game in charge is a home fixture against third-placed Yokohama F Marinos on Sunday.
(Reporting by Shrivathsa Sridhar in Bengaluru)
