Colleen McCullough's widower has agreed their marriage was in a terrible state and they were struggling financially less than a year before the famed author's death.
But Ric Robinson denies his wife feared for her safety after she refused him money and his behaviour changed.
Dr McCullough is said to have made the allegation to police when they briefly split in June 2014, the NSW Supreme Court has heard.
Under cross-examination by barrister Kim Morrissey on Monday, Mr Robinson further denied he raised his hand at his wife, or that he and his mistress entered her bedroom and abused her.
The widower is involved in a legal battle with Dr McCullough's executor and friend, Selwa Anthony, who claims the University of Oklahoma Foundation is the rightful beneficiary of the bestselling author's estate as per a July 2014 will.
Ms Anthony claims Mr Robinson took advantage of his wife's ill health to unduly influence her to leave him everything in the lead-up to her death on Norfolk Island on January 29, 2015.
Mr Robinson says he's the rightful sole beneficiary of the estate.
Ms Anthony previously told the court that at about the same time Dr McCullough created the Oklahoma will, she said her husband was having an affair and she'd kicked him out for good.
But within weeks they had reconciled, with Dr McCullough saying she couldn't afford a night carer and he needed the money.
Mr Robinson on Monday said he had an affair from about 2008 to August 2014.
When asked if the affair upset Dr McCullough, Mr Robinson said "it was her suggestion in the first place that I do have a friend".
The 65-year-old retiree denied suggestions by Mr Morrissey, who is acting for Ms Anthony, that he was largely responsible for the couple's financial woes.
He agreed he'd been financially dependent on Dr McCullough, and since her death he'd made some income by offering tours through their home in Norfolk Island.
Ms Anthony earlier on Monday told the court Dr McCullough "knew exactly what she was doing" when she left everything to the University of Oklahoma Foundation.
She said the author created the will without any pressure from anyone, when she was 100 per cent in the right state of mind.
"There is no other will as far as I'm concerned," Ms Anthony said.
The hearing continues before Justice Nigel Rein.
