Mr Rugova died of lung cancer on Saturday at the age of 61.
The independence leader was the driving force behind a non-violent campaign by Kosovo Albanians to end Serbian rule.
At least 50,000 people braved freezing temperatures to farewell him, lining the narrow streets of the main town, Pristina, as his casket passed.
Police officers were deployed one metre apart along the route of the procession.
Followed by a solemn crowd, the coffin was carried to a hall where the dignatories attended a ceremony in tribute to Mr Rugova.
They included EU foreign policy chief, Javier Solana and the chief of the UN mission in Kosovo, Soren Jessen-Petersen.
During his eulogy, Mr Jessen-Petersen said the late president had never waivered in setting Kosovo on a path to make it "democratic, multi-ethnic and free".
"President Rugova has left a void behind him -- but he has also left a vision to guide Kosovo forward."
Prime minister Bajram Kosumi used his speech to call for unity among Kosovo Albanians in their independence bid after the passing of their iconic leader.
"President Ibrahim Rugova is an unavoidable part of independent Kosovo, who became our image in the world. His death imposed our need for unity in order to crown the success of a modern and independent Kosovo," said Kosumi.
Draped in the blue and red flag he conceived for an independent Kosovo, his coffin was later taken for burial at the Tomb of the Martyrs.
A gun salute was fired as his coffin was lowered into a specially built tomb atop a hill overlooking Pristina.
Sixty-one trees, one for each year of his life, were planted within the complex.
His death has left a political void that forced the United Nations to
postpone talks between Kosovo Albanian leaders and Serbia on the province's future status.
Kosovo was placed under UN administration in 1999 after NATO's air war forced Serbian troops loyal to former Serbian leader Slobodan Milosevic to abandon a crackdown on separatist Albanian rebels.
Tensions remain high in the province, as Albanians continue to seek independence from Serbia in the UN-backed talks now due to begin next month.
Belgrade is opposed to losing the province it sees as the birthplace of Serbian civilisation.