A team who have earned a reputation for punching above their weight, Slovenia captain Jan Mursak tied the game with under two minutes to go by tapping home a rebound and scored the winner less than a minute into sudden-death overtime.
"I'm proud of my guys, I'm proud of the team," said Mursak, who played sporadically with the NHL's Detroit Red Wings between 2010 and 2012.
"Hopefully, we can start looking to the next game and surprise somebody else."
The U.S. struck first late in the opening period when Garrett Roe kept a loose puck in the Slovenian zone, carried it behind the net and fed Brian O'Neill who was streaking into the slot. O'Neill fired a wrist shot that beat Slovenia goaltender Gasper Kroselj on his stick side.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
In the second period Jordan Greenway, the first African American to play for the U.S. men's Olympic hockey team, tapped in a puck that popped loose from a scrum in front of Kroselj.
"It was exciting to get my first, but it would have been much better to get a win tonight," said Greenway, one of four American college players in the U.S. team.
Slovenia, ranked 15th by the International Ice Hockey Federation, got on the board in the third period when Blaz Gregorc threaded a wrist shot through a crowd in front of U.S. goalkeeper Ryan Zapolski.
A team who never quit, the Slovenians fought their way into the knockout round in Sochi four years ago and finished seventh.
The Olympic hockey tournament is being played without NHL players for the first time since the Lillehammer Games in 1994 after the league and International Olympic Committee failed to agree on players' travel and insurance costs.
That has left teams to build their rosters from emerging local talent or from players in other professional leagues, largely from Europe.
The Slovenians next face the Olympic Athletes from Russia, who lost to Slovakia 3-2 on Wednesday. The Americans play the Slovaks.
(Reporting by Dan Burns, editing by Ed Osmond)
