The Argentina striker converted a controversial 90th-minute penalty after Ricardo Rodriguez was adjudged to have pulled back Danilo D'Ambrosio to snatch victory at the San Siro.
Icardi put the home side ahead with a deft 28th-minute finish and then after Suso had levelled with an equally adept finish, dispatched an improvised 63rd-minute volley.
Montella's men thought they had claimed a point when goalkeeper Samir Handanovic could only carry Giacomo Bonaventura's 81st-minute shot into his own net, but Icardi's late heroics condemned them to a fourth Serie A defeat of the new campaign which leaves them 10 points adrift of their city rivals.
Inter enjoyed the better start and Antonio Candreva was unfortunate not to open the scoring with 13 minutes gone when he ran on to Borja Valero's pass and stabbed a shot against the crossbar with keeper Gianluigi Donnarumma beaten.
Milan gradually worked their way into the game as Inter central defender Miranda found himself under pressure, although it was he who went close at the other end with a header from Ivan Perisic's cross which dropped just wide after Valero's 23rd-minute free-kick had been only half-cleared.
But the home side forced their way in front five minutes later when Icardi got ahead of defender Mateo Musacchio to guide Candreva's cross inside the far post.
Donnarumma was relieved to grasp a skidding Icardi long-range effort to his chest nine minutes before the break, but opposite number Handanovic had to get down smartly to block Fabio Borini's shot at his near post in the final minute of the half.
Milan thought they were back in it within four minutes of the restart when Musacchio rifled the ball into the roof of the net after Andre Silva's curling left-foot effort had come back off the post, only for an offside flag to cut their celebrations short.
Handanovic had to save from former Liverpool strikers Suso and Borini in quick succession and half-time substitute Patrick Cutrone guided Rodriguez's whipped cross into the side-netting as the visitors sensed an opportunity.
They were back in it with 56 minutes gone when Suso curled a fine left-foot shot from distance into the bottom corner to level.
Matias Vecino should have restored Inter's lead on the hour when he steered another inviting Candreva cross inches wide, but after a fine save by Handanovic denied Bonaventura at the other end, they did force their way in front once again.
Icardi had to adjust in mid-air as he met Perisic's cross, but he did so perfectly to volley into the far corner, although the drama continued when Handanovic could only carry the ball over his own line as he attempted to claw away Bonaventura's shot.
But there was still more to come at the death when Rodriguez impeded D'Ambrosio in the area and Icardi stepped up to score from the spot.