The hostages, human rights worker Kate Burton and her parents, were freed on Friday after two days in captivity in an abduction claimed by a previously unknown group that called itself the Brigades of the Mujahideen-Jerusalem.
The man, a local member of the al-Aqsa Martyrs Brigades, an armed wing of the mainstream Fatah movement, was whisked into an unmarked car in the town of Rafah.
An official said: "He was arrested for interrogation about involvement in the kidnapping."
Angry gunmen fired into the air outside a security headquarters in Rafah, a Gaza town on the border with Egypt, to protest the arrest.
About 20 Palestinian gunmen later stormed into a government office in the Gaza Strip to protest the arrest of the kidnap suspect.
Members of the al-Aqsa Martyrs Brigades, an armed wing of the mainstream Fatah movement, forced their way inside the Interior Ministry in Rafah, demanding the release of their colleague, after burning tyres and shooting.
The office was empty of personnel, witnesses said.
Palestinian sources said officials were in contact with the gunmen by telephone and trying to persuade them to leave.
All were armed with assault rifles and some brandished rocket-propelled grenade launchers which they threatened to shoot.
Gaza has seen a rash of kidnappings and other violence since Israel withdrew from the coastal strip in September after 38 years of military rule.
The violence has threatened to delay a Palestinian January 25 parliamentary election.