Vajpayee was admitted to the All India Institute of Medical Sciences hospital in June and was being treated for a kidney tract infection.
Prime Minister Narendra Modi visited the ailing former leader on Wednesday evening.
In a statement the hospital said: "Despite the best of efforts, we have lost him today. We join the nation in deeply mourning this great loss."
Vajpayee was among India's most popular politicians and a founding leader of the ruling Hindu nationalist Bharatiya Janata Party. He retired from active politics in 2005.
Vajpayee was twice elected prime minister, first for 13 days in 1996 and then from 1998 to 2004.
The BJP patriarch had been suffering from ill health in the past years and remained confined to his home. He suffered a stroke in 2009 and was also hospitalised that year with a serious respiratory infection.
Modi led the tributes, tweeting: "His passing away marks the end of an era. He lived for the nation and served it assiduously for decades. My thoughts are with his family, BJP Karyakartas and millions of admirers in this hour of sadness. Om Shanti."
Vajpayee's supporters saw him as a skilled politician who managed to avoid fanaticism, a man who refused to see the world in black and white.
But his critics considered him the leader of a fanatic movement - a movement partially rooted in European fascism - that sought power by stoking public fears of India's large Muslim minority.
The one thing both sides could agree on was his honesty. Vajpayee was that rare thing in Indian politics: a man untainted by corruption scandals.
One of seven children of a schoolteacher in central India, Vajpayee joined India's Hindu revivalist political movement in his late 20s.
Elected to parliament in 1957, he became the best-known figure in its moderate wing, and helped the Bharatiya Janata Party become one of India's few national political parties.
