With no extra time in this season's competition, the contest went straight to spot kicks and Frank Lampard's Derby won 8-7 with Phil Jones's effort saved by visiting keeper Scott Carson.
United manager Jose Mourinho said he felt his team were in difficulty once the shootout got down to his defenders.
"I knew that we were going to be in trouble, with Jones with Eric (Bailly) I knew that we were going to be in trouble then," he said.
For former England midfielder Lampard, who played under Mourinho at Chelsea, it was just reward for playing a full-strength side and approaching the game in positive fashion.
"I wanted to bring a team who could really compete. I am so proud of them," said Lampard, in his first season as a manager.
"I am trying to build a group to be successful and nights like this are incredible. To go against world-class players and play that way, it's right up there," he added.
Derby had been moments away from winning 2-1 in normal time after United keeper Sergio Romero was sent off in the 67th minute but Marouane Fellaini got a stoppage-time equaliser.
United's rivals and League Cup holders Manchester City had no such trouble as they won 3-0 at Oxford United but top flight Burnley went down 2-1 at third-tier Burton Albion.
United had taken the lead against Derby in the third minute with a well-placed shot from Juan Mata after Anthony Martial had burst down the left flank.
However, the visitors drew level in spectacular fashion after 59 minutes with a brilliant bending free-kick from Harry Wilson, the young Welsh winger on loan from Liverpool.
Eight minutes later, Romero was dismissed for handling outside the box and Derby grabbed what they thought was the winner when United's substitute keeper Lee Grant parried a shot from Mason Mount and Jack Marriott nodded in the loose ball.
But in the fifth minute of added time, substitute Fellaini headed in a Diogo Dalot cross to force the shootout.
BURNLEY STUNNED
Manchester City's Gabriel Jesus was one of several big names in their lineup and scored after 36 minutes with a header before Riyad Mahrez and Phil Foden wrapped up proceedings late on.
Burnley picked up their first league win of the season at the weekend but came crashing to earth at League One Burton.
Goals from the home side's Liam Boyce and Jamie Allen cancelled out a first-half strike from Burnley's Kevin Long, and made sure Burton reached the fourth round for the first time.
They were joined by fellow third-tier side Blackpool who piled misery on second-tier Queens Park Rangers with a 2-0 win.
Fulham's Cyrus Christie scored with a stunning long-range strike as they ran out 3-1 winners at Millwall, while Andros Townsend grabbed two goals for Crystal Palace as they triumphed 3-0 at former Premier League side West Bromwich Albion.
Bournemouth let a two-goal lead slip at home to Blackburn Rovers before Callum Wilson rescued a 3-2 win in stoppage time.
Gareth Ainsworth's Wycombe Wanderers almost fought back from 4-1 down following a Jordan Rhodes hat-trick for visitors Norwich City but despite their valiant efforts they lost 4-3.
Middlesbrough won a shootout 4-3 at Preston North End after the game finished 2-2, while Leicester City beat Wolverhampton Wanderers 3-1 on penalties after a 0-0 Midlands derby draw.
(Additional reporting by Christian Radnedge; editing by Ken Ferris)
Share
