At least two infected refugees have gone missing since testing positive for the virus after the first COVID-19 death was reported Tuesday, they said.
About one million Rohingya, most of whom fled a military crackdown in Myanmar in 2017, are packed into camps along the Bangladesh border, and the coronavirus has become the latest cause of misery.
Aid agencies have long warned that the virus could cause chaos in the overcrowded camps, where social distancing is virtually impossible.
So far only 29 infections have been detected, although 16,000 Rohingya are in quarantine zones within the camps.
It was not immediately clear how many tests have been conducted in the camps, but a senior health official said two people who proved positive had "fled the isolation hospital".
He added that only 20 refugees agreed to be tested in the past two days because they believe those infected will be sent to Bhashan Char island in the Bay of Bengal.
"It has created mass panic," Nurul Islam, a community leader, told AFP.
Bangladesh authorities have long wanted to establish a camp for 100,000 people on the isolated island, and have already sent 306 Rohingya there.
"The Rohingya are petrified," the health official told AFP on condition of anonymity.
"We have told them they won't be sent anywhere."
Some 500 isolation beds have been prepared in the camps, but most are empty because so few confirmed cases have been found, according to the official.
Rohingya refugees walk through one of the arterial roads at the Kutupalong refugee camp in Cox's Bazar, Bangladesh. Source: AP
The first Rohingya fatality from the coronavirus was announced only Tuesday, and health officials say they desperately need to increase testing to see how widespread the virus may be.
But Khalilur Rahman Khan, the government administrator of one camp block, said doctors told him refugees were reluctant to participate.
Several Rohingya leaders said the transfer of the 306 refugees to Bhashan Char had sparked rumours that anyone with coronavirus would be sent to join them.
"People are scared to go for virus tests," said Abu Zaman, a community leader.
Mohammad Shafi, a camp neighbour of the refugee whose death was announced Tuesday, said people who had coronavirus symptoms such as fever and aches insisted they only had seasonal flu.
"I tried to reassure them that this a curable disease and most people will recover, but many don't believe it," he said.
People in Australia must stay at least 1.5 metres away from others. Check your state’s restrictions on gathering limits. Testing for coronavirus is now widely available across Australia.
If you are experiencing cold or flu symptoms, arrange a test by calling your doctor or contact the Coronavirus Health Information Hotline on 1800 020 080. The federal government's coronavirus tracing app COVIDSafe is available for download from your phone's app store.
SBS is committed to informing Australia’s diverse communities about the latest COVID-19 developments. News and information is available in 63 languages at .