The Trek Factory Racing rider was one of eight riders in the early break when he was sideswiped by a Shimano neutral service vehicle which swerved to avoid a curb.
"I don't even really know what happened," explained Sergent from the hospital.
"I was taking a pull and the road curved to the left and a car was coming and the next moment I was on the ground. Straight away I knew that I wasn't going to be getting back on the bike. I am pretty disappointed.
"Once I got up I realised it was the neutral support car trying to pass on the inside, I am not sure why, but it hit my handlebars and my arm or something and I had no chance to control the bike. It was also on a bit of a downhill section so we were going quite fast."
X-rays revealed a dislocated fracture of the left collarbone and Sergent will undergo surgery Monday to insert a plate and screws to facilitate the healing.
Sergent was not the only rider to suffer at the hands of reckless driving. FDJ's Sebastian Chavanel was another unintentional victim after a different neutral support vehicle rear ended his team car as he was receiving support.
The team later announced that Chavanel would undergo further examinations and that he has a significant hematoma on his thigh. Chavanel tweeted that despite the incident he was in good spirits.
Subsequent to the race Shimano, who run the neutral support vehicles, released a statement apologising for the accidents.
"At Shimano we apologise for both incidents the Shimano sponsored neutral caused today at the Tour of Flanders," the statement read.
"We especially apologize to the riders and teams involved, Jesse Sergent of TREK Factory Racing and SÃ©bastien Chavanel of FranÃ§aise des Jeux. We wish them a speedy recovery.
"The drivers of the neutral cars are professionals with many years of experience in professional cycling races. We will investigate both incidents deeply and take appropriate action."