Mr Abbott was visiting the Knox City Shopping Centre in Melbourne's east today.
When the man dressed in nothing but red budgie smugglers tried to get his attention by shouting: "Do you like the cut of my Speedos Tony?"
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The man was instantly surrounded by photographers and retreated into a nearby shop where he put on an overcoat and made a quick getaway.
Because of the media crush, the intruder didn't manage to get near Mr Abbott, whose choice of swimming attire has been the subject of national conjecture since he became opposition leader.
Mr Abbott was earlier addressing the Australia-Israel Chamber of Commerce lunch in Melbourne.
