Openers Prithvi Shaw and Lokesh Rahul, both of whom were not out on 33, completed the formalities in 16.1 overs to inflict on West Indies their second successive defeat inside three days.
Umesh, who was twice on a hat-trick in the match, claimed 4-45 to become only the third Indian paceman -- after Kapil Dev and Javagal Srinath -- to claim 10 wickets in a home test.
"I felt really happy for him," India coach Ravi Shastri said of Umesh, who has often had to warm the bench because of the competition in the pace bowling department.
"It can be a frustrating profession when you sit on the reserve bench like he did in four test matches in England.
"He got his opportunity here and he grabbed it with both hands... He's given us a (selection) headache and it's a very good headache."
After India resumed day three on 308-4, Rishabh Pant fell on 92 for the second successive innings and Ajinkya Rahane also missed his century as Holder and Shannon Gabriel restricted India to 367.
Holder surprised Rahane, who made 80, with extra bounce and two balls later trapped a scoreless Ravindra Jadeja lbw.
Rahane featured in a 152-run stand with Pant, who could not reach triple figures thanks to a brilliant catch by Shimron Hetmyer.
The West Indies batsmen, however, could not complement the lion-hearted effort by their bowlers.
The tourists got off to a nightmarish start with opener Kraigg Brathwaite and Kieran Powell both departing for ducks.
Brathwaite blocked the first delivery from Umesh to deny him a hat-trick spread across two innings though fell caught behind from the very next ball.
Umesh dismissed Roston Chase and Shane Dowrich with the last ball of his eighth over and the first of his ninth but a hat-trick eluded him for the second time in the match.
Sunil Ambris's 38 was the top score in a poor batting display by his team.
"Definitely a little disappointed with the batting," Holder said.
"We fought our way back nicely after the first Test. Credit to the way India play cricket."
India's spin trio, led by Ravindra Jadeja (3-12), shared six wickets among them, but were, for once, content playing second fiddle to paceman Umesh, who was the obvious choice for the man-of-the-match award.
(Reporting by Amlan Chakraborty in New Delhi; editing by Peter Rutherford and Toby Davis)
