The gathering on a busy promenade beside the Caspian Sea in the glitzy capital was called by the Public Chamber opposition alliance which brings together the main opposition parties in the energy-rich country.
But police grabbed about 30 protesters from the crowd of several dozen people as they shouted "Freedom!", putting them in vans, an AFP correspondent reported.
One young woman screamed as she was shoved into a vehicle.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The busy public place filled with journalists and observers, including some representatives of Europe's security body OSCE, as police officers struggled to identify more protesters in the crowd.
"I must say I did not expect them to crack down like this," said one of the observers Anders Nielsen of the Norwegian Helsinki Committee, who came on a visit to Baku during Eurovision.
"What we are truly seeing here is a totalitarian regime that cracks down on anything that looks like opposition," he told AFP.
Azerbaijan has pulled out all the stops to put on the international Eurovision song contest decorating its Caspian capital with contest insignia and constructing the glamorous Crystal Hall venue.
But the festivities have been overshadowed this week by arrests of dozens of opposition supporters who have seized on the opportunity to draw attention to the country's rights record and issues like political prisoners.
