The victory came as a minor relief for champions Leicester who are one point above the Premier League relegation zone after failing to win a league game this year.
Ndidi, a January signing from Racing Genk, put Leicester 2-1 ahead with a bullet finish and Gray slalomed through the Derby defence to rubber-stamp Leicester's progress.
Leicester had taken the lead a minute after halftime when Andy King headed home from close range, but Derby clawed their way back when Abdoul Camara's free kick took a wicked deflection on its way into the net.
Leicester, who along with Derby fielded a largely second-string side, visit third-tier Millwall in the last 16 on Feb. 18.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
With Leicester's Premier League future looking increasingly uncertain after a run of four straight defeats, manager Claudio Ranieri made 10 changes to the side that lost 3-0 to Manchester United on Sunday.
Leicester went ahead a minute after the restart when Gray's superb cross to the far post was headed back by Marc Albrighton for King to head into the empty net from close range.
The lead lasted 15 minutes, Derby drawing level as Camara's free kick took a huge deflection off Ben Chilwell in the wall and skidded past Leicester keeper Ron-Robert Zieler.
Ndidi was introduced by Ranieri for extra time and the Nigerian picked up the ball and drove towards the Derby goal before letting fly with an unstoppable effort that hit the inside of the post on its way into the net.
With seven minutes of extra time remaining, Gray set off on a jinking run, leaving several defenders for dead before firing high into the net.
(Reporting by Toby Davis; editing by Ed Osmond)
