Kompany returned from injury as a substitute on Boxing Day but lasted only nine minutes before limping off again, and scans have revealed he has aggravated the same calf but with a different problem.
The Belgium international, 29, missed the goalless draw against Leicester on Tuesday night and Pellegrini revealed he will be out for at least another three to four weeks.
"Vincent has an injury, a second degree tear of the calf," said Pellegrini.
"We will see how long he will be out, but I don't think it will be less than three to four weeks.
"I am not a doctor, but with a muscle injury, it will be very difficult for him to be back in less than four weeks. But without him, we must continue to defend as did tonight."
Kompany has started just eight Premier League games all season for City and the stalemate at the King Power Stadium was the first time Pellegrini's men have stopped the opposition scoring in his absence.
With Nicolas Otamendi and Eliaquim Mangala yet to keep a clean sheet prior to Tuesday, some City fans may have feared the worst facing the league's joint-highest scorers without their skipper.
However, the expensive and often criticised central defensive pairing produced an accomplished display in the east midlands, keeping Leicester's leading scorer Jamie Vardy quiet for much of the game, while the Foxes' other danger man Riyad Mahrez was also reduced to a bit-part role.
Pellegrini refuted claims City had come for a draw, though.
"I think we are returning to a very solid and consistent team, which is important," Pellegrini said.
"One of the good things about tonight is that we kept a clean sheet after many games without. But I am not happy because we came here to try to win the game.
"We went for the three points and I think that we had more possession than Leicester and more attempts and did more things to win the game.
"But of course if you can't win then it is very important you don't lose and we were a very solid and consistent team defending against a very difficult team like Leicester who have players in a very good moment and that always scores goals.
"I am not happy but not disappointed also because I think we played well but we couldn't score."
Leicester lost their 10-game unbeaten record with a 1-0 defeat to Liverpool on Boxing Day, leaving some observers to wonder if their challenge at the top was about to falter.
But Claudio Ranieri's team silenced their critics with another superb performance as they more than held their own against Pellegrini's men.
"I think we made a very good match. It was difficult but we played well, we created some chances," Ranieri said.
"I wanted to see how my players responded after the Liverpool defeat and they responded well.
"It's not easy for us but we want to fight with everybody. It's a miracle what we're doing
"I'd love this to continue but it's not easy. I'd like my players to continue to fight like today. It's fantastic."