It was the third time they had led against a Newcastle side, who are now six points behind their opponents despite substitute Aleksandar Mitrovic twice bringing the visitors level.
Norwich went ahead before the break when Swiss defender Timm Klose headed past Newcastle's third-choice keeper Karl Darlow but Mitrovic levelled in the 71st minute with a glancing header before Dieumerci Mbokani put the hosts back in front.
The Congolese international cut in from the left and smashed the ball into the top corner with his right giving Darlow no chance after 74 minutes, but Serbian Mitrovic then scored again, this time with a penalty, in the 86th after Gary O'Neil handled.
Norwich were not finished though as Olsson raced through and hit the winner with a low drive three minutes into added time.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
(Reporting by Ken Ferris; Editing by Pritha Sarkar)
