The two athletes were the first Russians to be sanctioned for anti-doping rule violations following an IOC investigation into allegations of widespread doping among Russians and sample tampering by lab and security officials to give them the edge.
Legkov won gold in the 50km and a silver medal in the 4x10km relay event. The IOC said all of Russia's 4x10km team would be disqualified given Legkov's violation.
Belov, who competed in the men's skiathlon 15+15km mass start event and the 15km classic country skiing event, did not medal at the Sochi Games.
Legkov could not be immediately reached by Reuters for a comment.
The IOC did not specify what kind of anti-doping rule violation the skiers had committed but said both of them were "ineligible to be accredited in any capacity for all editions of the Games... subsequent to the Sochi Olympic winter Games."
It said more decisions would follow in the coming days as the disciplinary commission led by IOC member Denis Oswald completes its investigation into Russian competitors in Sochi.
Russia's cross country skiing federation said it was preparing an appeal.
"The federation has already begun preparing documents for an appeal to the Court of Arbitration for Sport (CAS)," it said in a statement. "Federation president Elena Valbe and other employees of the federation will not give further comments."
The IOC is re-testing all Russian athletes' samples from the 2014 Games following revelations by Grigory Rodchenkov, the former head of Moscow's discredited anti-doping laboratory, of a scheme to cover up home competitors' positive samples.
The Sochi scandal is part of a wider doping affair in Russia that has led to the suspension of the country's anti-doping agency and an international ban for track and field athletes.
The IOC has set up two commissions to investigate Sochi.
The one led by Oswald is in charge of looking into alleged doping violations by individual Russian athletes.
The second investigation is chaired by former Switzerland President Samuel Schmid and is looking into allegations of institutionalised doping in Russia.
The IOC has said it will decide during its executive board meeting being held from Dec. 5-7 on the participation of Russian competitors at the Pyeongchang winter Olympics in February.
Calls from some athletes and anti-doping agencies for a blanket ban of Russians in Pyeongchang have been growing louder in recent weeks.
Russian track and field athletes were banned from the Rio de Janeiro summer Games following the doping affair.
(Reporting by Karolos Grohmann, Additional reporting by Gabrielle Tetrault-Farber in Moscow; Editing by Catherine Evans and Ken Ferris)
