In its apology, the newspaper described the cartoon as depicting an elderly man "with a large nose, sitting alone, with a remote control device in his hand, overseeing explosions in Gaza."
It originally appeared in a column written by Mike Carlton, which described the scale of destruction to civilian lives and property in the Palestinian territory as disproportionate to the threat posed by Hamas.
The unsigned apology, published today, acknowledged that the cartoon had sparked deep debate as well as accusations of racism.
"The Herald now appreciates that, in using the Star of David and the kippah in the cartoon, the newspaper invoked an inappropriate element of religion, rather than nationhood, and made a serious error of judgment," the apology reads.
"It was wrong to publish the cartoon in its original form."
On Sunday, several thousand people turned out in Sydney for two separate protests in support of Israelis and Palestinians, respectively.
Executive Council of Australian Jewry President Robert Goot addressed a crowd of Israeli supporters gathering at Bondi Beach just hours before the apology was issued by the paper.
"The cartoon in last Saturday’s SMH was unquestionably anti-Semitic," he said.
"Our letter of complaint was not even published."
The Attorney-General, George Brandis, has also criticised the publication of the cartoon, calling it "deplorable" and "overtly anti-Semitic," the Australian newspaper reported today.
Meanwhile some have come out in defence of the newspaper, taking to Twitter to say it shouldn’t have been forced to apologise.
What do you think? Tweet @SBSNews or comment below.