Japan's preparations were dealt a blow last month when Prime Minister Shinzo Abe ordered plans for Tokyo's National Stadium re-drawn, robbing the tournament of its centrepiece venue.
The National Stadium, which is being re-built for the 2020 Olympic Games, was to host the final and key games in rugby's global showcase but will no longer be finished in time.
World Rugby issued a media release last week demanding Japan submit a new venues plan and an updated budget by September to provide assurances that the tournament would go ahead.
"I've read the World Rugby press release regarding the assurances they are seeking from Japan and it concerns me that this ultimatum may well be a precursor to World Rugby stripping Japan of the hosting rights," O'Neill said in comments published by The Australian newspaper on Monday.
"And indeed there has been some sentiment out of the UK in particular that the decision to award Japan the hosting rights in the first place was one that some would like to reverse."
Organisers have earmarked the Tokyo Stadium as a replacement but the 50,000-capacity venue some 20 km deeper into the city's western suburban sprawl may not be World Rugby's ideal choice.
O'Neill, who stepped down as ARU chief in 2013 after a second stint in charge, backed Japan's bid after pulling Australia out of the bidding process in 2009.
"The fact is the ARU, New Zealand and other forward-thinking nations worked very hard to ensure Japan was awarded the rights to host 2019, being the first non-traditional territory to do so," O'Neill added.
"And the strategic imperative is blindingly obvious that if World Rugby wants to live up to its objective of being a truly world game, then every effort and every ounce of energy must be directed into ensuring Japan does host 2019.
"If you look at the composition of the organising committee that has been assembled, it is a very large collection of captains of industry in Japan, chairmen and CEOs of incredibly large and important companies across a range of industries.
"I have no doubt that these gentlemen will rise to the challenge as will the Japanese government."
(Writing by Ian Ransom; Editing by John O'Brien)
