Spaniard Alonso broke the deadlock for the west London side in the 86th minute when he sneaked behind two defenders to connect with Willian's cross and score what proved to be the winner in the first competitive meeting between the two sides.
"I'm very happy with the result and also with the performance, because at this moment we are able to suffer, in the past we were not," Chelsea manager Maurizio Sarri told reporters.
"In the past, when we were in trouble, we conceded immediately a goal, now we did not concede anything."
Sarri made seven changes from the team that beat West Ham United 2-0 in the Premier League on Monday, with talisman Eden Hazard coming off the bench for the last half an hour.
Willian's fierce shot hit the bar in the first half before the hosts had a chance to open the scoring on the counter in the 69th minute when Ibrahim Traore drive into the Chelsea box and struck a shot that was superbly punched away by Kepa.
The Spanish goalkeeper then kept out left-back Jan Boril's close-range shot in the 80th minute.
Alonso's goal gave the Blues a vital away goal as they put behind them bitter memories of the stadium where they lost the 2013 Super Cup to Bayern Munich in a penalty shootout.
Slavia defender Simon Deli said: “It’s disappointing because we did very well and in the last minutes we conceded this kind of goal. Of course, it’s difficult for us, but it’s football.
"Now we’ll prepare for our next game - the derby - and then against Chelsea in London,” he added referring to the Czech league leaders' game against third-placed Sparta on Sunday.
(Reporting by Robert Muller; Editing by Ken Ferris)
