The result left Manchester City two points clear of them at the top of the table.
The fiery Spain striker scored in the 17th minute as Antonio Conte's team took control of the game before conceding twice in quick succession.
In the 59th minute goalkeeper Thibaut Courtois gave away a penalty by fouling Gylfi Sigurdsson after a Swansea counter-attack and the Iceland international got up to convert it.
Then Gary Cahill was dispossessed by Leroy Fer and apparently fouled, but Fer was allowed to go on and score.
Nine minutes from the end Costa, who also received his third yellow card in four games, equalised with an overhead scissors kick that might have been penalised for dangerous play.
It all added up to one of the most dramatic Premier League games of the season so far, which Chelsea, who had 28 attempts on goal, dominated for almost an hour.
They should have been further ahead at the interval.
Lukasz Fabianski made saves from Willian, Eden Hazard and Cesar Azpilicueta, before Costa was guilty of an extraordinary miss.
As John Terry flicked on a corner the Spanish international was unmarked two metres from goal but directed the ball wide.
Fortunately for the Londoners, Costa had been sharper earlier on, guiding the ball perfectly just inside a post as Oscar laid it square to him.
It was poor Swansea defending, however, as Federico Fernandez twice failed to clear.
Manager Francesco Guidolin changed tactics from a misfiring 3-5-1-1 system before halftime, withdrawing the disgruntled Welsh international Neil Taylor for Gambian winger Modou Barrow.
He could hardly have envisaged such a dramatic turnaround, helped as it was by mistakes from Courtois and Cahill.
Costa continued to be involved in controversy right to the finish.
Booked for a foul, he could have been sent off after an apparent dive, but had the last word with an acrobatic effort that may have made slight contact with defender Kyle Naughton's head and went in off Federico Fernandez's face.
Conte felt Swansea's second goal should not have been allowed because of Fer's challenge on Cahill.
"I think everyone can see it was a bad decision," he told Sky Sports.
"I'm very happy for (Costa's) passion but he took a lot of kicks and it's incredible the defender finished the match."
Chelsea left David Luiz, re-signed from Paris St Germain, on the substitute's bench, but he may be needed for Friday's televised game at home to Liverpool after Terry left the stadium on crutches with an ankle injury.
(Reporting by Steve Tongue; editing by Clare Lovell)
