Wiggins trailed Armstrong in the overall standings 11 years ago and while the record books have since removed the American's efforts, he remains resigned to fourth.
Armstrong's history of doping led to a lifetime ban and the removal of all results since August 1998 in what was a remarkable turn from deity to disgrace.
In an interview with Graham Willgoss, Wiggins discussed Armstrong's actions and the fallout from that infamous event.
"I did go toe-to-toe with him in 2009 and then it became apparent that the place got taken off him and I got third," the Brit said during the latest episode of Eurosport's The Bradley Wiggins Show.
"But in my head I’m forever fourth in the Tour de France because you miss out on the podium and it’s gone.
"And it doesn’t matter but again it was an added pressure that I had to deal with.
"I saw it as a pressure because there were lines to get right and there was interview after interview where the line of questioning was all Lance.
"Riders are expected to have all the answers. It's not that I don’t care or anything like that. The answer should be obvious.
"It's all well saying, 'you should have been more vocal' and things like that, but riders are more selfish; we've got enough to worry about.
"And that's not to diminish it or not pay attention to it, but that's what other people are there for.
"Maybe that’s the wrong attitude to adopt but it’s the attitude I adopted at the time. I think whatever I say on it, I risk there being a backlash on it.
"I see stuff from a human point of view as well. Some things I wasn’t happy about that I said just to over-egg the point.
"It’s s***, that time in cycling was s***, but this is all with cognitive bias, people got a lot of pleasure in those years, people benefited a lot those years…
"Now we’re picking up the pieces and we’re expected to have the answers to it.
"But we still haven’t got the answers to it and it’s still an open wound in the sport.
"But at that point in my life I had far greater concerns than worrying about, maybe wrongly – I’m sure people have views on it – being the voice for that whole scenario."
The 2020 Tour de France will return for Stage 16 tonight LIVE, FREE and EXCLUSIVE at 8:55pm (AEST) tonight, via the SBS ŠKODA Tour Tracker), and at 9:30pm (AEST) on SBS VICELAND and SBS On Demand.
