The Dutchman was clearly struggling, not racing to the usual level fans are used to in recent years and had been contemplating leaving the race for several days.
But there seemed to be fresh life in his legs in the early stages of the second day of racing following the rest day, as Van der Poel attacked from the peloton with Wout van Aert (Jumbo-Visma).
That was short-lived though, as once the proper breakaway was established he saw himself dropped on the first climb of the day, the Lacets de Montvernier before the stage had reached the 50 kilometre mark.
Van der Poel rejoined the peloton to attempt to make it over the next series of tough climbs, but was dropped out the back again on the Télégraphe and made the decision there to walk away from the race.
His second participation in the Tour de France has been a stark contrast to last year, where he won Stage 2 and wore the yellow jersey for six days before leaving early to prepare for the Tokyo Olympics.
After appearing at the Giro d'Italia earlier this year where he looked in great form, winning the opening stage and finishing the event, it may have been too much of a workload for the 27-year-old.
A tall task not all riders take on, Van der Poel said after Stage 6 that two Grand Tours in quick succession was taking it's toll on him physically and mentally.
“I knew it wasn’t going to be an easy combination," he said.
"But I didn’t expect to be so disappointing myself.
"For me, three weeks of racing is different from racing one race – that also applies to me mentally.
“In the Giro, there was that chance of pink and I wanted to finish the whole race anyway. But now here we are. It’s not fun like this.”