Scotland's qualification hopes seemed set for a crushing setback when Fiodor Cernych fired the visitors ahead just before the hour mark but McArthur spared their blushes.
After a dismal first half Scotland were finally beginning to exert some sustained pressure on Lithuania's goal with Robert Snodgrass and Chris Martin both going close to an opener.
But Cernych silenced the home crowd with a well-worked goal in the 58th minute, playing a delightful one-two before spinning off and powering a shot past Scotland keeper David Marshall.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Cernych should have doubled Lithuania's lead but blazed wide after a swift counter-attack.
Scotland pushed forward in desperation and finally got their reward when substitute McArthur guided in a close-range header after Callum Paterson's cross was flicked on by Grant Hanley.
Lithuania survived some intense pressure in five minutes of stoppage time, meaning they, Scotland and Slovenia are all together on four points from two games, two behind leaders England who earlier beat Malta.
Scotland face pointless Slovakia away on Tuesday while Lithuania host Malta.
(Reporting by Martyn Herman, editing by Pritha Sarkar)
