Milan, who have angered new coach Sinisa Mihajlovic with their performances, have three points, although they livened up after Mario Balotelli came on for his debut in the second half as they maverick striker hit the post.
Atalanta forward Mauricio Pinilla scored two goals, the first with one of his trademark bicycle kicks, before he was sent off in a pulsating 2-2 draw at Sassuolo in which both teams missed penalties.
Three other matches also ended 2-2, with Carpi denied a first-ever top flight win after conceding a late goal at Palermo and Napoli still winless this season after being held at Empoli.
Sampdoria's match at home to Bologna was postponed because of a flood alert in Genoa and Lazio beat Udinese 2-0 with two goals from Alessandro Matri.
Inter are followed by Chievo, Sassuolo, AS Roma, Palermo and Torino on seven. Titleholders Juventus have only one point after drawing 1-1 at home to Chievo on Saturday.
Milan forward Luiz Adriano got clear of the Inter defence three times in the first half, but his first effort was saved by Samir Handanovic, the second went over the bar and the third was blocked by a defender.
Guarin broke the deadlock just before the hour, the Colombian forward powering towards the area and unleashing a left-foot shot which left Diego Lopez floundering.
Balotelli came on to a deafening chorus of jeers from Inter fans but made an impact as he provoked Felipe Melo into being booked, hit the post with a 25-metre effort and was close again with a long-range free kick.
The drama at Sassuolo began when forward Nicola Sansone saw an early penalty saved by Marco Sportiello and Pinilla quickly made them pay.
A cross was floated over from the left and the Chile forward quickly got into position and met it with a perfectly executed overhead kick from 12 metres.
Francesco Magnanelli equalised with a 35-metre rocket, Pinilla pounced on a dreadful back pass to put Atalanta back in front and Antonio Floro Flores made it 2-2 with a spectacular half-volley, all before halftime.
The drama continued as Pinilla was dismissed early in the second half for a reckless and pointless sliding tackle, then Sassuolo's Sime Vrsaljko was sent off and conceded a penalty for a clumsy challenge.
Atalanta forward Maxi Moralez saw his effort saved by Andrea Consigli.
Napoli, who have only two points from three games, twice came from behind for a 2-2 draw at Empoli with Lorenzo Insigne and Allan scoring to cancel out goals by Riccardo Saponara and Manuel Pucciarelli.
Serie A debutants Carpi led 2-1 at Palermo until Uros Djurdjevic headed home with two minutes left for the Sicilians, although the visitors still took home their first Serie A point.
Former AC Milan, AS Roma and Juventus forward Marco Borriello had given Carpi the lead with his first goal for the club.
The 38-year-old Luca Toni, who was joint topscorer last season with 22 goals, scored his first this term when he converted a penalty in Verona's draw at home to Torino.
(Writing by Brian Homewood in Berne; editing by Clare Fallon and Pritha Sarkar)
Share
