They are now caught on the sidelines of fighting between Myanmar troops and the Arakan Army, a militant group seeking more autonomy for western Rakhine state's Buddhist-majority population.
"Heavy fighting is going on between the government troops and Arakan Army inside Myanmar," Rohingya leader Dil Mohammad told AFP.
"The situation is very tense," he said, adding the security build-up and daily gunfire had created "panic".
Myanmar troops last week set up security camps and bunkers along the border after fighting saw 13 police killed.
Some of the fortifications are directly adjacent to a border fence running alongside a stream, and overlook shacks erected by an estimated 4,500 displaced Muslims living in the narrow strip of land.
Refugee community leader Nur Alam said gunfire could frequently be heard after dark on the other side of the border.
"Every night it is close by. The Myanmar border guard have set up 10 new posts near our camp. It's very intimidating," he told AFP.
In a statement issued Wednesday, the United Nations said it was "deeply concerned" about the situation in the area.
A Bangladesh official said they were aware of the border tensions.
"We will talk to the relevant authorities to discuss what to do," local administrator Kamal Hossain said.
Rohingya in Buddhist-majority Myanmar have suffered decades of persecution. Impoverished western Rakhine state in particular is scarred by deep ethnic and religious hatred.
Refugees pouring into Bangladesh have detailed mass killings, arson and rape at the hands of Myanmar troops and Buddhist mobs.
Myanmar has denied any wrongdoing, saying it was defending itself against Rohingya militants who attacked police posts.
The United Nations has called for a genocide investigation into the crackdown.