Yates was dropped by the surge of Tadej Pogačar (UAE Team Emirates) on the Col de Marie Blanque, and couldn't quite reach the second group of general classification riders either, eventually finishing 54 seconds behind the stage winner.
Yates now sits eighth on the general classification, a minute and two seconds down after new yellow jersey holder Primož Roglič (Jumbo-Visma) also took some bonus seconds out on course to pad his advantage.
"I did my best. I knew coming into the race I wasn't 100 per cent," Yates said after the finish. "I gave everything I could to hang on. I think we can be proud of what we did. I'll freshen up now, have a rest day and then go after some stages.
"We'll see what happens with the rest of the race. The next couple of days are flat - there might be crosswinds so it could be pretty interesting. I said from the very beginning there were a lot of stages in the last week that suit me quite well so we'll have a go and see what we can do.
"It's a big honour to ride in yellow. It was good fun while it lasted."
Mitchelton-Scott directeur sportif Matt White said that stage wins were always the objective for the team and they'll go back to that now that they aren't holding yellow.
"We came into this race with the objective to go hard in the first week to look for stages," he said. "That turned into the yellow jersey, we kept it for a few days, and now we can go back to the original plan to target some stages."
There might be a wait until Yates can challenge for wins again, though, with his GC position still too dangerous for him to get free reign to attack from the peloton. It will mean that Yates' Mitchelton-Scott teammates aren't on yellow jersey defence duty, with the likes of Daryl Impey to chase stage wins likely to chase wins in breakaways.
"I think we'll have to wait and see," said White. "We've got a rest day, a couple of easier days and then the next real test for the GC guys doesn't come until next Friday.
"But there are some good stages for the intermediate rider – stages 12 and 13 – but I think with Adam's position they probably won't be so viable until we get back into the mountains."
The Tour de France continues after the rest day with a pancake flat stage on the western coastline of France from Île d’Oléron to Île de Ré. Watch the action on SBS, with the race starting on the SBS ŠKODA Tour Tracker at 9.20 AEST and the broadcast beginning at 9.30 AEST on SBS VICELAND, switching to SBS HD at 10.30pm.