Second-half goals from teenager Emanuel Vignato and Perparim Hetemaj gave the Flying Donkeys, condemned to Serie B last week with six games still to play, only their second win of the season and the first this year.
The defeat meant that Lazio dropped below Torino into eighth place with 52 points, four adrift of the top four places where they need to finish to qualify for the Champions League.
Lazio were in trouble from the moment that Milinkovic-Savic was given his marching orders in the 34th minute for lashing out after being fouled by Mariusz Stepinski.
Vignato, 18, curled in his first Serie A goal in the 49th minute and Hetemaj headed the second two minutes later.
Felipe Caicedo replied for Lazio and Joaquin Correa hit the post in stoppage time before the hosts had Luis Alberto sent off on the final whistle for dissent.
(Writing by Brian Homewood; editing by Clare Fallon)
