Dowsett had started stage six one minute and 25 seconds behind Omega Pharma-QuickStep's Michal Kwiatkowski in the overall standings, yet he ended the day's racing in possession of the yellow jersey thanks to a powerful finish.
At the start of the day Dowsett, in 21st position overall at 1min 25sec behind, was seemingly out of contention for the general classification despite his chances in the coming time-trial.
The 25-year-old put himself in the break, along with Brandle (IAM) and Tom Stewart of Madison Genesis, and was able to change the shape of the Tour from that position.
Capitalising on the time trialling skills of both Brandle and Dowsett, they built a lead of more than nine minutes on the road, and the peloton rarely looked capable of bringing it back despite some frantic efforts from Omega Pharma-QuickStep and Team Sky in the closing stages of the 205.6km run from Bath to Hemel Hempstead.
Omega Pharma QuickStep drove the pace hard riding for Kwiatkowski with Mark Cavendish and Niki Terpstra burying themselves on the Chiltern climbs in an attempt to bring back the break.
Team Sky also sent Ben Swift up the front to try and claw back some of the time, and eventually Bardiani CSF, the team of second place Edoardo Zardini, also contributed. But it was all too late.
"It was the hardest day of my life," said Dowsett.
"I just thought they were letting us out to bring us back and I was a bit hacked off with myself because I thought I could have a good time trial but I was leaving a lot out on the road and I might have given too much.
"Once the gap went up to seven minutes I thought, that's quite big, then it was eight, nine, and I really started digging in.
"I spoke to Matthias and we made an agreement he would go for the stage because I knew there was a good chance I would take yellow and it was a risk I was willing to take."
Dowsett now leads Kwiatkowski by 34 seconds, with Eduardo Zardini of Bardiani 40 seconds behind in third and Tinkoff-Saxo's Nicholas Roche 50 seconds back in fourth.
"What can I say, it's part of the sport," said Kwiatkowski. "Sometimes you win, sometimes you lose. Today, chapeau to the guys in the front. They went hard. We tried to chase them the whole race, but it wasn't possible.
"It's not over. Even when we lose, we don't give up. I was even trying to chase in the last kilometers. I was trying to minimise the gap. We all gave everything we had and that's all we can ask for."
Stage 6: 205.7km, Bath to Hemel Hempstead
1 Matthias Brandle (AUT) IAM 4hr 44min 49sec
2 Alex Dowsett (GBR) Movistar 0:00:01
3 Thomas Stewart (GBR) Madison Genesis
4 Sonny Colbrelli (ITA) Bardiani CSF 0:01:51
5 Nicola Ruffoni (ITA) Bardiani CSF
6 Ben Swift (GBR) Sky
7 Martin Kohler (SUI) BMC
8 Heinrich Haussler (AUS) IAM
9 Rick Zabel (GER) BMC
10 Tom Veelers (NED) Giant-Shimano
1 Alex Dowsett (GBR) Movistar 25hr 07min 53sec
2 Michal Kwiatkowski (POL) Omega Pharma-QuickStep 0:00:34
3 Edoardo Zardini (ITA) Bardiani CSF 0:00:40
4 Nicolas Roche (IRL) Tinkoff-Saxo 0:00:50
5 Dylan Teuns (BEL) BMC 0:00:51
6 Jon Izagirre (ESP) Movistar 0:01:00
7 Bradley Wiggins (GBR) Sky 0:01:02
8 David Lopez (ESP) Sky 0:01:04
9 Leopold KÃ¶nig (CZE) NetApp-Endura 0:01:06
10 Sebastien Reichenbach (SUI) IAM