The Nine Network's "worm" - a gauge based on opinions from an audience of swinging voters - declared Mr Rudd the winner of Tuesday's televised debate by a margin of 71 to 29.
Mr Abbott says he is not surprised the worm didn't like him.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"The worm has never liked Liberals," he told the Macquarie Radio. "It never liked John Howard.
"If the worm had had its way, Mark Latham would have been prime minister, so I'm not too worried about the worm.
Unauthorised arrivals
"What I'm worried about is a health plan that involves a massive shake-up to no good purpose, being run by a prime minister who is a proven incompetent at program delivery."
He criticised the government's handling of the home insulation program, as well as flaws in the school building program to emphasise his point, before the matter of another two unauthorised boats arriving in Australia was raised.
The 79 people on board have been taken to Christmas Island, where facilities are under strain with 2008 people already in detention.
"It's all very well for him (Mr Rudd) to say to his local MPs, 'let's go out and challenge every Liberal to a debate on healthcare', well I think the public are interested in a range of subjects," he said.
"Border protection is a very important one, and if you can't trust the prime minister to protect our borders, you can't trust him with pink batts, you can't trust him to build school halls successfully, why should you trust him with public hospitals?"
