After losing the first game, Lin regained his composure to draw level. With the decider locked at 16-16, he looked in trouble but relied on experience to oust world number 39 Holst.
The inspired display kept Lin's hopes of winning a sixth world title alive and the 33-year-old said he fancied his chances after Lee's exit had opened up the competition.
"It is more exciting. Every player now should try their best. My opponent today did well in the first game. After that, I adjusted my strategy and performed better," he said.
Lin next faces England's 16th seed Rajiv Ouseph, who beat India's Sameer Verma 22-20 21-9.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Fifth seed Chen Long advanced in a convincing manner, knocking out unseeded Indonesian Sony Dwi Kuncoro 21-13 21-9. The defending champion next plays India's Ajay Jayaram.
Another Indian, Kidambi Srikanth also progressed, seeing off France's Lucas Corvee 21-9 21-17 while Brice Leverdez beat Croatia's Zvonimir Durkinjak 16-21 21-19 22-20.
Top-seeded South Korean Son Wan-ho made short work of Hungary's Gergely Krausz with a 21-13 21-9 win. China's fourth seed Shi Yuqi edged Brazil's Ygor Coelho 21-17 21-23 21-10.
Among the women, South Korea's second-seed Sung Ji-hyun beat India's Tanvi Lad 21-9 21-19 while 12th seed Saina Nehwal powered past Switzerland's Sabrina Jacquet 21-11 21-12.
(Reporting by Shrivathsa Sridhar in Bengaluru, editing by Pritha Sarkar)
