Martyn Waghorn scored the only goal on 78 minutes, scrambling the ball home after the home side had failed to clear a short corner.
Accrington had already been reduced to 10 men when midfielder Daniel Barlaser, on loan from Newcastle United, was sent off on 59 minutes for a second yellow card after a rash challenge on Mason Bennett.
That tipped the game's balance in favour of the visitors, but Derby also finished with 10 men when Jayden Bogle cynically brought down Paul Smyth when he was free on goal.
That ensured a thrilling final few minutes with Derby keeper Kelle Roos saving superbly from Billy Kee.
Derby won the world's oldest cup competition for the only time in 1946 and are managed by Frank Lampard, who won it four times as a player with Chelsea.
Ten other ties take place later on Saturday.
(Reporting by Neil Robinson; Editing by Toby Davis)
