Australian Federal Police, NSW Police and the domestic spy agency ASIO jointly carried out the major counter terrorism operation on Saturday afternoon in Surry Hills, Lakemba, Wiley Park and Punchbowl.
"Four men have been taken into custody and are assisting police with their enquiries," the AFP and NSW Police said in a joint statement following the raids.
Fairfax Media reports the raids were in relation to a terrorist cell "and a bomb plot involving aircraft".
The Seven Network reports 40 riots squad officers stormed the Surry Hills terrace before the explosives team found a suspicious device.
TV footage shows a man with a bandage on his head and draped in a blanket being led away by authorities.
Prime Minister Malcolm Turnbull has been briefed on the progress of the operation by security agency heads.
"These operations are designed to disrupt and prevent plans to undertake terrorist attacks in Australia," the prime minister said in a statement.
"My number one priority, and that of my government, is the safety and security of all Australians, and the public should be reassured that our security and intelligence agencies are working tirelessly to keep us safe."
The family who lived in the raided house on Cleveland Street in Surry Hills has been described as "perfectly nice and normal people" by a neighbour whose property backs onto theirs.
"We knew them to say hello to and they seemed nice," the woman in her early 30s, who didn't want to be identified, told AAP.
An elderly couple lived in the home, the neighbour said, and they had adult children.
The neighbour came home while the raid was underway and said there were "heaps" of police at the scene.
Cleveland Street remains closed in both directions in Surry Hills between Elizabeth and Crown streets.
"Motorists are advised to avoid the area or allow plenty of extra travel time," a Transport Management Centre spokesman said in a statement.
