Unions have demanded Prime Minister Tony Abbott act immediately and shut down the unions royal commission.
ACTU secretary Dave Oliver said Commissioner Dyson Heydon's delay in deciding whether he will stand aside did not stop Mr Abbott from taking action.
"The ACTU has always maintained that the prime minister must shut down his political witch-hunt and stop wasting millions of dollars in taxpayers' money," he said in a statement.
"The ACTU again calls on Tony Abbott to act immediately."
Asked if he would shut down the inquiry, Mr Abbott replied: "Absolutely not."
He said Mr Heydon was a man of utmost integrity and would carefully consider submissions made to him last Friday.
"I can't imagine it is going to be delayed too long because these are things which are inherently best decided quickly," he told ABC radio.
Mr Abbott said many serving judges had attended Labor events without attracting accusations of bias.
"There is nothing inherently wrong with going to a Liberal event," he said.
Mr Abbott said the attack was really on the royal commission not on Mr Heydon.
He said the union movement and Labor were trying to keep subterranean "the rorts, the rackets and ripoffs" uncovered by the royal commission.
Share

