Long latched on to a deep cross from substitute goalkeeper Darren Randolph to drill past Germany stopper Manuel Neuer and take Ireland, who visit Poland on Sunday, level on 18 points with their next opponents.
"The whole team did magnificently. It was a wonder goal," Ireland coach Martin O'Neill told reporters after their first victory in a competitive game against the Germans.
"It's a historic win, a great win and the players deserve it."
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"Just a splendid effort by the team. We've still got a big struggle in a few days time," he said.
Group leaders Germany, who have 19 points and only needed a draw to qualify in Dublin, next host Georgia.
"It was an unexpected and completely unnecessary defeat," said Germany coach Joachim Loew. "I cannot accuse the team of not fighting but you have to stay cool and score those goals.
"We dominated throughout but made nothing out of that dominance."
The Germans had suffered a late blow in the build-up to the match with captain Bastian Schweinsteiger ruled out before the kickoff having picked up a groin injury in their final training session.
"What happened is what happens with teams like Ireland and that is that we had a lot of possession," Germany defender Mats Hummels said.
"We had our chances but they used their only one. Ireland made the most out of very little. Now we know it can get very tight in the last game and we have to be dominant against Georgia and not let anything happen."
But they still had no problem taking control of the game with some swift passing and lots of movement off the ball and wasted a couple of early chances, with the unmarked Jerome Boateng missing the target with a header and Ilkay Guendogan seeing his effort deflected wide.
The visitors gradually eased off, however, allowing Ireland more space and as the hosts grew in confidence they were almost caught on the break when Andre Schuerrle, a replacement for the injured Mario Goetze in the first half, fired over the bar.
Ireland continued to work hard and were rewarded when Randolph's long kick sent Long clear to beat Neuer with a powerful shot after he raced past two Germany defenders.
Germany striker Thomas Mueller should have levelled with a close-range effort seven minutes later but the hosts held on for a memorable win.
(Additional reporting by Padraic Halpin and Conor Humphries in Dublin, Writing by Karolos Grohmann, editing by Ed Osmond)
