Heldt, who has held the post since 2010, and Schalke agreed to a "seamless transition", it said.
"This seamless transition is something we jointly worked on for months," said Schalke boss Clemens Toennies. "I am happy Christian Heidel and Horst Heldt accompanied me in this process in a constructive way."
Heidel, a long-time sports director at Mainz 05, will receive a four-year deal.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Schalke have had a disappointing season and could miss out altogether on European football next season.
Currently lying in seventh place, the Royal Blues need to beat Hoffenheim in their last match and hope for a combination of other results to secure a top-six finish and a place in European club competitions.
(Reporting by Karolos Grohmann; Editing by Catherine Evans)
