Nationalist Prime Minister Shinzo Abe angered his neighbours and worried the US by visiting the Yasukuni war shrine.
China said the visit on Thursday was an attempt to whitewash a history of warmongering.
South Korea said the move was "anachronistic".
And Tokyo's chief ally, the US, said it was disappointed with an act that would "exacerbate tensions with Japan's neighbours".
Abe described his visit, which came days after he gave Japan's military its second consecutive annual budget boost, as a pledge against war and said it was not aimed at hurting feelings in China or South Korea.
Yasukuni Shrine is the believed repository of about 2.5 million souls of Japan's war dead, most of them common soldiers, but also including several high-level officials executed for war crimes after World War II.
"Some people criticise the visit to Yasukuni as paying homage to war criminals, but the purpose of my visit today ... is ... to renew the pledge that Japan must never wage a war again," Abe said in a statement.
"It is not my intention at all to hurt the feelings of the Chinese and Korean people. It is my wish to respect each other's character, protect freedom and democracy, and build friendship with China and Korea with respect."
The visit came 12 months after Abe took power, a period in which he has not formally met China's President Xi Jinping or Korea's President Park Geun-Hye.
Beijing slammed Abe's move, which came on the day Xi and other senior Chinese leaders visited the mausoleum of Mao Zedong, the hardline communist who led China through the disastrous Great Leap Forward, in which up to 45 million people died.
"The essence of Japanese leaders' visits to the Yasukuni Shrine is to beautify Japan's history of militaristic aggression and colonial rule," foreign ministry spokesman Qin Gang said.
Chinese foreign ministry official Luo Zhaohui called the visit "absolutely unacceptable to the Chinese people" and cautioned Japan "must bear the consequences arising from this".
The last incumbent Japanese prime minister to visit the shrine was Junichiro Koizumi in 2006. His repeated pilgrimages soured relations with China, despite the important economic and trade ties that bind the two countries.
The foreign ministry in Tokyo said it wanted to stress Abe "visited Yasukuni Shrine in a purely personal capacity (and) ... not ... to pay homage to war criminals".
However, China and South Korea, victims of Japan's 20th-century aggression, say no distinction exists.
"We can't help deploring and expressing anger at the prime minister's visit to the Yasukuni Shrine ... despite concerns and warnings by neighbouring countries," Seoul's culture minister Yoo Jin-ryong told reporters.
"The visit ... is an anachronistic behaviour that fundamentally damages not only relations between the South and Japan but also the stability and co-operation of northeast Asia," he said.
Washington, which has to tread a careful line between supporting its chief regional ally in the face of China's rise to becoming a global superpower and emboldening a prime minister many observers see as a hot-headed trouble-maker, offered qualified criticism.
"Japan is a valued ally and friend.
"Nevertheless, the United States is disappointed that Japan's leadership has taken an action that will exacerbate tensions with Japan's neighbours," a written statement said.
