PSG, who drew their opening three games last season, have two points from two matches, four less than big-spenders Monaco who thrashed Montpellier 4-1 thanks to Emmanuel Riviere's hat-trick and a Radamel Falcao penalty.
Uruguay striker Cavani fired PSG's 32nd attempt on goal into the net with four minutes left to cancel Benoit Pedretti's splendid curling ninth minute opener.
PSG had 78 percent of possession and were left frustrated in front of a crowd of 43,000 by their visitors' heroic defensive performance and multiple saves by goalkeeper Guillermo Ochoa.
They also lacked some luck after captain Thiago Silva saw both a powerful 81st minute header and a stoppage-time strike hit the crossbar.
Monaco shone at a sun drenched Stade Louis II with Riviere finding himself in the side following injuries to summer signings Joao Moutinho and James Rodriguez.
Falcao put Monaco ahead in the 18th minute following Siaka Tiene's foul on winger Lucas Ocampos in the box, but his fellow Colombian Victor Montano levelled for Montpellier six minutes later with a powerful header.
The hosts then took a decisive advantage thanks to Riviere's first two goals which came either side of halftime.
Montpellier were reduced to 10 men after Daniel Congre was dismissed and quickly surrendered as 23-year-old Riviere, who has never scored more than eight goals in a league season, completed his hat-trick with eight minutes left.
Monaco are one of four teams to maintain a perfect start to the new season along with Olympique Lyon, Olympique Marseille and St Etienne.
(Reporting by Gregory Blachier; Editing by Martyn Herman and Mark Pangallo)
