Liverpool moved on to 17 points from eight matches and will stay top at least until Arsenal play Norwich City at the Emirates later on Saturday (1400 GMT).
Newcastle had to play the whole of the second half at St James' Park with 10 men after the 42nd-minute sending-off of defender Mapou Yanga-Mbiwa.
They had gone ahead in the 23rd minute when Yohan Cabye scored with an unstoppable, swerving, 25-metre shot that gave Liverpool goalkeeper Simon Mignolet no chance.
The last time this fixture ended goalless was in 1974 and the scoring continued when Steven Gerrard pulled Liverpool level from the penalty spot after France international Yanga-Mbiwa had been sent off in the 40th minute for hauling down Luis Suarez.
The Uruguayan striker was in with a scoring chance when the Frenchman grabbed his shoulder and pulled him to the ground, earning an immediate red card from referee Andre Marriner.
Gerrard became the 24th player in the Premier League era to score 100 goals and the third from Liverpool after Michael Owen and Robbie Fowler.
Newcastle went back in front when substitute Paul Dummett, making only his second appearance for the Magpies and brought on to shore up the defence after Yanga-Mbiwa's dismissal, sidefooted a volley home at the back post for his first Newcastle goal after a Cabaye free kick in the 57th minute.
Liverpool, who won 6-0 at Newcastle last season, rallied again when Suarez and Daniel Sturridge combined, with Sturridge heading in to make it 2-2 18 minutes from time.
The match ended with Tim Krul making a flying leap to keep out a Suarez free kick in the seventh minute of stoppage time.
As well as Arsenal's match with Norwich, six other Premier League games were being played on Saturday as the domestic programme resumed following the international break.
(Editing by Clare Fallon)
