Algeria led thanks to William Troost-Ekong's own goal but Nigeria hit back through Odion Ighalo's penalty, awarded following a VAR check.
Extra time was looming until Algeria, who have not been to the final since winning the tournament on home soil in 1990, won a free-kick on the edge of the box.
And Mahrez, star man for the Desert Foxes, bent the ball around the wall to beat Nigeria goalkeeper Daniel Akpeyi and seal his country a place against Senegal in Friday's final.
Baghdad Bounedjah should have scored an early opener in Cairo but somehow bundled wide of the post from close range, then Ramy Bensebaini wasted a free header in the Nigeria box.
Bounedjah, without a goal since Algeria's first AFCON game, fluffed his lines again in the 29th minute, hustling Kenneth Omeruo out of possession but firing straight at Akpeyi.
Ighalo slotted Alex Iwobi's pass just wide and Algeria then opened the scoring just before half-time, Mahrez's cross deflecting off left-back Jamilu Collins before bouncing in off the helpless Troost-Ekong.
Nigeria showed a huge improvement after the break but they needed a slice of luck to equalise in the 72nd minute.
Aissa Mandi blocked a long-range Peter Etebo drive with his arm, referee Bakary Gassama awarding a penalty after consulting VAR, with Ighalo coolly converting despite a long delay.
Algeria midfielder Ismael Bennacer clipped the crossbar with a 25-yard drive but Mahrez then brilliantly settled matters after Wilfred Ndidi gave away a free-kick in a perfect position.
Senegal, who beat Tunisia 1-0 in extra time thanks to an own goal, await in Friday's final.
