Wiggins, who said earlier this month he expects to miss this year's Tour with 2013 champion Chris Froome fronting the squad, was brought down after 28km on Tuesday, suffering bruising to his right thigh.
Wiggins, who in 2012 became the first Briton to win the Tour de France, said he hoped to recover in time for the British national Time Trial Championships on June 26, but did not mention of the Tour de France.
"I've got quite a lot of swelling on the left-hand side of my right knee," he told the Team Sky website.
"It's one of those things, I've been pretty fortunate this year with crashes. I probably could have started today but it's best to play it safe and come home now.
"I really want to try to be ready for the Nationals next Thursday so I didn't want to jeopardise that for the sake of pushing on and potentially doing more damage."
Fellow Briton Mark Cavendish (Omega Pharma Quick-Step) won the fourth stage while Italian Sacha Modolo won the 183km stage on Wednesday. Germany's Tony Martin (Omega Pharma) is the overall leader with four stages remaining.
(Editing by Martyn Herman)
Share
