Kompany was making his first Premier League start for seven months but lasted just 37 minutes before having to come off following a collision with his own goalkeeper Claudio Bravo.
"He has a problem with his knee. He could be out for weeks," Guardiola told reporters ahead of City's Champions League Group C match at Borussia Moenchengladbach on Wednesday.
"The doctor told me weeks but did not explain how many."
The 30-year-old Belgium international has suffered an injury-plagued start to the season after his thigh injury in City's 1-0 Champions League semi-final defeat by Real Madrid in May.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
He returned to action in their League Cup win against Swansea City in September but again was forced off before the end with a groin injury.
"He is sad but he is in the best hands. He is travelling to a hospital in Spain so he can recover as soon as possible," Guardiola said.
Asked whether Kompany would be fit for the knockout stage of the Champions League early next year, Guardiola said: "Now is not the moment for this. First we have to qualify."
City, in second place of Group C on seven points, will secure a spot in the knockout stage with a win over Gladbach, who are third on four. Barcelona, beaten by Manchester City in the previous matchday, are top with nine points.
"Gladbach are a really good team and they have a good mentality. We will see tomorrow whether we will win," Guardiola said. "We were so happy with beating Barcelona but it was just one game. We cannot be considered favourites to win the tournament."
(Reporting by Karolos Grohmann; editing by Ken Ferris and Pritha Sarkar)
