The result left Sweden, who were unbeaten in their previous 10 competitive games against Bulgaria dating back to 1967, second with 13 points from seven games, three behind France, who thrashed Netherlands 4-0.
Bulgaria, who stayed on course to reach the World Cup finals for the first time since 1998, climbed to third on 12 points, followed by Netherlands with 10.
The hosts took the lead when Stanislav Manolev headed a precise left-wing cross by Georgi Milanov past goalkeeper Robin Olsen in the 12th minute for Bulgaria's first goal against the Swedes since 1993, during which time they have suffered seven consecutive defeats.
Four minutes later Bulgaria keeper Plamen Iliev, making his competitive debut, saved an Emil Forsberg penalty, awarded for Georgi Kostadinov’s foul on Sweden captain Andreas Granqvist.
The visitors levelled when right back Mikael Lustig scored with a header from a Ludwig Augustinsson cross in the 29th minute but Bulgaria restored their lead soon after, thanks to Georgi Kostadinov, who netted for a second successive game.
Markus Berg made it 2-2 two minutes before the break, giving Iliev no chance with a glancing header from Lustig’s right-wing cross.
In an eventful first half Bulgaria coach Petar Hubchev was sent to the stands by Italian referee Paolo Tagliavento after protesting against a decision.
Bulgaria held on in the second half as a re-energised Sweden created several chances, with Iliev making a string of good saves, and Chochev scored the winner with an 18-metre shot to delight the sparse crowd at the Vasil Levski national stadium.
Bulgaria visit Netherlands on Sunday in their next game in the group while Sweden are away to Belarus.
(Reporting by Angel Krasimirov,; Editing by Neville Dalton)
