Genoa and Udinese squandered the chance to move level on points with Juventus after defeats by Sassuolo and Chievo Verona respectively.
Stephan Lichsteiner had knocked in Alex Sandro's low cross to give Juve the lead in the 66th minute in a typically tense encounter but Inter captain Mauro Icardi levelled two minutes later by heading home from a corner.
Icardi then provided a stylish cross with the outside of his boot for Perisic to score in the 78th, nine minutes after the Croatia international had come off the bench.
Inter had midfielder Ever Banega sent off in the 90th minute for a second yellow card but they clung on to inflict a first league defeat on Juve since May 8 and end their 100 percent start to the season.
“This win is a great moment for the fans, the club and the team and will start to grow our confidence,” Inter coach Frank de Boer told Sky Sport Italia.
“Juventus of course give you extra motivation but we said before the game we had to play as a team from the first to the 90th minute.”
The win will give De Boer breathing space after a difficult start in the Inter dugout, taking four points from his first three league games and suffering an embarrassing home defeat to Israeli side Hapoel Beer-Sheva in the Europa League.
Juventus coach Massimiliano Allegri surprisingly left Serie A record signing Gonzalo Higuain out of the starting line up and was forced to take injured defender Medhi Benatia off midway through the first half.
Juve midfielder Sami Khedira should have given the visiting side the lead but failed to get enough power on a free header, and immediately down the other end Icardi battled past Giorgio Chiellini to curl a shot off the far post.
Inter took the game to Juve after the break but were pegged back by Lichsteiner's close range strike.
Juve's joy at going ahead did not last long, however, and Inter roared back into the game thanks to Icardi and Perisic's headers to claim their first Serie A win over their rivals since 2012.
"It's not a tragedy. I was worried about such a strong series of early fixtures, but the players did well overall," said Allegri.
"I think every now and then a defeat like this can do us good, if we are smart and humble enough to learn from it.”
Juve stay second on nine points, one behind leaders Napoli who beat Bologna 3-1 on Saturday, while Inter move up to sixth on seven points.
Croatia international Badelj settled a tense game in Florence with a low strike in off the post from the edge of the area, although Roma complained that two offside Fiorentina players were obstructing goalkeeper Wojciech Szczesny's view.
Sassuolo beat Genoa 2-0 at home while Udinese threw away the lead as they lost 2-1 at home to Chievo after conceding a goal in the fifth minute of added time.
England international Joe Hart kept a clean sheet in his home bow for Torino, who drew 0-0 with Empoli, while Cagliari beat Atlanta 3-0 and Crotone picked up their first ever Serie A point by drawing 1-1 at home to Palermo.
(Reporting by Richard Martin, editing by Alan Baldwin/Ken Ferris/Pritha Sarkar)
