The 27-year-old man was pinned to the ground in the middle of the road close to the Treasury and entrances to Westminster station on the London Underground train network.
The area has been on a state of heightened alert since a deadly attack on March 22 perpetrated by Khalid Masood, a 52-year-old convert to Islam.
Masood drove into pedestrians, killing four, and stabbed a police officer to death at the gates of parliament before being shot dead by an armed officer.
London's Scotland Yard police headquarters said nobody was injured in Thursday's incident.
They said the man was arrested at 2:22pm (1322 GMT) following a "stop and search" procedure as part of an ongoing operation.
The man "was arrested on suspicion of possession of an offensive weapon and on suspicion of the commission, preparation and instigation of acts of terrorism", Scotland Yard said.
"Knives have been recovered from him.
"He is being detained under the Terrorism Act and is in custody in a south London police station.
"Detectives from the Counter Terrorism Command are continuing their investigation, and as a result of this arrest there is no immediate known threat."
An AFP photographer saw firearms officers surrounding the man, who was dressed in dark clothing, and pinning him to the ground at a pedestrian island.
The bearded man, with close-cropped hair, was seen being detained closed to a telephone booth used by countless tourists posing for photographs with the Houses of Parliament in the background.
David Wisniowski, who was working on a building site right next to the incident, told AFP he could see "three knives on the floor, one big one and two small".
WATCH: Eyewitness describes arrest of suspect
Gabrielle Hennessy, 20, a US tourist, described seeing police rush at a man.
"We were walking along Parliament Square and we saw police rush at him. And then he was on the ground surrounded by police. They stood him up and that's when I saw the knife on the ground," she told AFP.
Police taped off the area in question.
An officer in forensic overalls could be seen photographing and picking up the knives, and looking inside a bag on the ground.
The incident happened close to Prime Minister Theresa May's Downing Street office.
May was outside London on a general election campaign visit at the time, a spokeswoman said.
"This is an ongoing police operation and we are monitoring it," she said.
"The prime minister has been informed by officials and she will be updated, as usual for incidents like this."
