The touring side reached the close on 102 for four in their second innings, a lead of 334 runs, after Herath, who opened the bowling, bagged his 27th test five-wicket haul with figures of 5-89.
Dimuth Karunaratne (54 not out) and Asela Gunaratne (six not out), who scored a maiden test century in the first innings, will resume on Wednesday as Sri Lanka seek to set a massive target after bowling Zimbabwe out for 272.
Seamer Carl Mumba (3-31) was the pick of the home bowlers supported by leg-spinner Graeme Cremer (1-21).
Zimbabwe had resumed on the third morning on 126 for two, but lost regular wickets through the day.
Seamer Suranga Lakmal ended a 117-run third-wicket stand between Craig Ervine (64) and Brian Chari (80) as the former was caught at first slip by Karunaratne.
After Chari, who had posted his best test score, was bowled by Herath it became something of a procession for the home batsmen.
Only Sean Williams (58) offered any resistance before he was trapped lbw by spinner Dilruwan Perera (3-51).
Sri Lanka won the first test by 225 runs.
(Reporting by Nick Said, editing by Ed Osmond)
