The teams faced a shootout after Colombia's Yerry Mina scored one of his signature headed goals in stoppage-time to cancel out England captain Harry Kane's 57th-minute penalty and the score was still level at 1-1 after extra-time.
England have a traditionally poor record with penalties - they were the team's undoing at the 1990, 1998 and 2006 FIFA World Cup, as well as the 1996, 2004 and 2012 European Championship. But coach Gareth Southgate's team prevailed 4-3.
England will face Sweden in the quarter-finals.
The game was rowdy, earning players a flurry of yellow cards, and Colombia vociferously protested the call that lead to Kane's goal, the England forward grappling with Carlos Sanchez before the Colombian pushed him to the ground.
The Andean nation's capital, Bogota, came to a standstill as people tuned in to watch the 'Cafeteros', with electrifying roars and loud gasps surging throughout the city each time either team came close to scoring.
Giant screens showed the match in many cities around the nation of nearly 50 million people and in downtown Bogota, where suited lawyers, policemen and street vendors cheered and then commiserated in front of a screen set amid the gray concrete streets, as office workers watched from the windows above.
"It was a good game - sad that we couldn't get through, but I'm proud of the team's effort," said 47-year-old shoe shiner Mauricio Sanchez, who spent much of the game standing atop his box of supplies, the better to see the screen.
Colombia's stand-out player from the tournament will be Mina, whose headers earned him the nickname 'Mina de Oro' or gold mine, after they carried Colombia to victories against Senegal and Poland in the group stage.
Mina's performance earned accolades from spectators despite the team's loss, while midfielder Mateus Uribe, who fluffed the first of Colombia's two missed penalties, earned sympathy.
"The best player was Mina," said lawyer Adriana Palma, 33. "I'm really sad for Uribe."
At a screening in an English-themed bar in northern Bogota, British expatriates and tourists celebrated England's victory.
"It was a good match, but (I'm) always disappointed when the game goes to penalties - it doesn't show the true potential of the team," said Emma Kinsey, 32, a yoga instructor from Lincoln who lives in Bogota.
Last week's game against Senegal saw Colombia's key playmaker James Rodriguez forced off due to a swelling in his right calf. He had to miss the England match, watching from the sidelines.
Colombia have lost to England at the FIFA World Cup before. At France 1998 the teams met in a group match and England won 2-0, thanks to goals by Darren Anderton and David Beckham, to advance to the last 16 where they lost to Argentina - on penalties.