Hiddink won the competition during his first caretaker spell in 2009 and, although he did not see last season's fourth-round loss to Bradford, the Dutchman knows the Blues must prevail against their League One visitors.
"I didn't see (the Bradford game), but I know you must not stumble again," Hiddink said.
Chelsea are unbeaten in four games since Jose Mourinho's departure last month, with Hiddink succeeding him.
"I'm happy with the restart of the team," Hiddink said.
"For me the next step is very important. That's why (after) a win against a very difficult team, as Crystal Palace are, you have to follow up.
"In the upcoming games we have to show what we're capable of."
Chelsea must do so without Eden Hazard, whose groin injury will see him miss Thursday's (AEDT) English Premier League clash with West Brom and could also rule him out of the after the weekend fixture with Everton.
The Blues also must manage Diego Costa carefully as the Brazil-born Spain striker is Chelsea's only fit frontman, with Radamel Falcao (thigh) and Loic Remy (calf) out.
"We have to be very cautious, careful about our (one) striker, because that's Diego at the moment," Hiddink added.
"Not having Remy and Falcao it's a little bit worrying."