The decision came the day after a 3-0 home defeat by Hamburg SV, their 10th loss in 11 games since Schaaf took over, left them 10 points adrift at the bottom of the table.
"We have come to the conclusion that we don't want to end the season in this way and will try to give the team a new impulse with a change of coach," said sporting director Martin Bader in a club statement.
"We know that avoiding relegation with a ten-point gap is only really possible in theory.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"It was not an easy decision," he added. "Thomas went about his work very meticulously. He has, together with his team, exhausted all the possibilities to get positive results..."
The club said under-19 team coach Daniel Stendel would take charge of training on Monday.
Hanover have been in the top flight since winning promotion in 2002, generally finishing in mid-table although they managed a best-ever fourth place in 2011.
They have taken only 17 points from 28 games, 10 behind Eintracht Frankfurt and also Augsburg, who are in the relegation playoff spot.
(Writing by Brian Homewood; Editing by Ken Ferris)
