Larkham, who won the 1999 World Cup and played more than 100 times for the national side, had been part of Cheika’s staff since 2015. He took up a role with Rugby Australia after leaving the team.
The Wallabies endured their worst test campaign in 60 years in 2018, winning just four of 13 tests and slumping to sixth in the world rankings.
"It's been very difficult, obviously," Cheika said on Wednesday at a Australia-Japan diplomacy event in Tokyo.
"But looking at some of the changes that I think we needed to make, on and off the field, going into this year to get the best possible outcome come November 2019, sometimes you have got to make those hard decisions."
With the World Cup only five months away, Cheika said he would finalise his coaching staff after the Super Rugby season has ended.
Cheika also said the Sept. 20 to Nov. 2 World Cup would be a unique tournament and teams would have to get used to playing in Japan.
"It is going to be an open World Cup for sure. I think not just because of the teams and how they have performed up until here but also because it is here," he said.
"I think it is going to be very different, different atmosphere at the games and we are so looking forward to it.
"We have got a bit to do before then but the players – the lucky ones - can't wait to get on the plane."
Australia reached the final of the 2015 World Cup in England, where they lost to New Zealand 34-17.
(Editing by Peter Rutherford)
