Liverpool have kept two clean sheets in nine matches so far this season and are winless in four games in all competitions.
Leicester substitute Shinji Okazaki took advantage in the 65th minute when Liverpool failed to clear a corner and the Japan forward's deflected shot crept into the net.
Liverpool's defence was also culpable in the 78th minute, switching off from a throw-in, although Islam Slimani's left-footed finish into the top corner was sublime.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"We played really good football in the first half, we should have decided the game early. We had big chances, strange situations but it's football, it's at least a 90-minute game," said Klopp, whose side were held to a 1-1 draw at home by Burnley in the Premier League on Saturday.
"It's second ball after a corner, I think, and a throw-in again. I'm sick of goals like this to be honest, you cannot play football and do that.
NOT CLINICAL
"Leicester somehow survived in the game because we are not clinical enough. That's sometimes okay but then you need to stay concentrated until the end and at least you don't allow them to score... and yeah, they scored too easy," he added.
"I can say it one time or 1,000 times or whatever -- it's not nice to have to concede these goals."
Klopp said playing well for 45 minutes was irrelevant.
"Nobody was interested in who played the better first half, everyone was interested (in) who will go through and we are not the team that will be in the next round for some good reasons." he said. "We wanted to go to the next round."
Liverpool midfielder Philippe Coutinho was substituted at halftime, but Klopp said it was not through injury.
"It was clear that 45 is the limit, we need to make him fit but we cannot push him through. That's all," he said.
Liverpool visit Leicester's King Power Stadium again on Saturday in the Premier League, hoping to get back on track.
"Will I still feel sick when I come back here on Saturday? I lost more games in my life unfortunately and, believe me or not, tomorrow morning I am fine," he said.
(Reporting by Martyn Herman; Editing by Ken Ferris)
