Brandle of IAM Cycling held his nerve to emerge victorious from a four-man breakaway alongside MTN-Qhubeka's Andreas Stauff, Belkin's Maarten Wynants and Shane Archbold of An Post-Chainreaction.
The decisive moment of the 177.3-kilometre route from Exmouth came on the drag into Exeter up Stoke Hill, with Brandle turning on the power to win by eight seconds from second-placed Archbold, who beat Wynants in a two-up sprint, while Bardiani CSF's Sonny Colbrelli led the peloton in 14secs behind in fourth.
Team Sky's Ben Swift was fifth, with Kwiatkowski (Omega Pharma-QuickStep) seventh and defending champion Bradley Wiggins (Team Sky) 36th.
Wiggins remained sixth overall, 27secs behind Kwiatkowski, ahead of Friday's sixth stage, the 205.6km from Bath to Hemel Hempstead.
"We were trying to control that gap to around three minutes, we knew that Brandle had three minutes in the GC so was not a dangerous break away for us," said Kwiatkowski.
"We knew that if someone from another team wants to win the stage they have to take control and chase down the break away, Garmin did it and I am really thankful to my team mates today they put me out of pressure.
"Out of the four stages so far this was the first one that I could really relax, it of course was tough, hard with the one kilometre climb and the fast decent into the finish."
The day's racing included an ascent of Haytor on Dartmoor, but it was the last moments that provided the most drama.
Garmin-Sharp initially led the pursuit on the run-in into Exeter, before the other teams with overall contenders shared the workload and set the stage for Brandle's charge to the finish line.
"We had to chase yesterday, because it is my job I had to go really deep to chase the breakaway back," said Brandle.
"Today I got in the breakaway and thought maybe it was possible for a stage win, that's what I tried to do. And finally its happens.
"No one really believed, but then they give us a little bit more and then we chatted some more and now we go full speed, we started to try and then it was 230- 2'40 and it was impossible for them to get us back.
"Finally we really worked well together and it was me who could take the win I had the best legs on the last climb, I tried to push as hard as I could, I managed to break them down and so it was a really nice victory for me but also to win a stage at the Tour of Britain, they are really good guys here and I'm really happy that I could take the win for my team."
Stage 5: 177.3km, Exmouth to Exeter
1 Matthias Brandle (AUT) IAM 4hr 32min 03sec
2 Shane Archbold (NZL) An Post-Chainreaction 0:00:08
3 Maarten Wynants (BEL) Belkin
4 Sonny Colbrelli (ITA) Bardiani CSF 0:00:14
5 Ben Swift (GBR) Sky
6 Rick Zabel (GER) BMC
7 Michal Kwiatkowski (POL) Omega Pharma-QuickStep
8 Kevin Ista (BEL) IAM
9 Nicolas Roche (IRL) Tinkoff-Saxo
10 Jack Bauer (NZL) Garmin-Sharp
1 Michal Kwiatkowski (POL) Omega Pharma-QuickStep 20hr 21min 50sec
2 Edoardo Zardini (ITA) Bardiani CSF 0:00:03
3 Dylan Teuns (BEL) BMC 0:00:14
4 Nicolas Roche (IRL) Tinkoff-Saxo
5 Jon Izagirre (ESP) Movistar 0:00:23
6 Bradley Wiggins (GBR) Sky 0:00:27
7 David Lopez (ESP) Sky
8 Leopold Konig (CZE) NetApp-Endura 0:00:29
9 Sebastien Reichenbach (SUI) IAM
10 Giovanni Visconti (ITA) Movistar 0:00:40