Using its power, it blocked key budget measures, forcing the Governor-General to intervene, and he promptly sacked Mr Whitlam.
This week marks the 40th anniversary of what is now famously referred to as "The Dismissal."
Gough Whitlam was the Labor leader who soared to power in 1972, ending two decades of conservative government in Australia.
But it was the way he left office that is remembered as one of the most divisive moments in modern Australian political history.
"Well may we say, 'God Save the Queen,' because nothing will save the Governor-General."
That was the voice of the late Gough Whitlam on the afternoon of his dismissal, 40 years ago.
The Governor-General of the time, Sir John Kerr, dismissed Prime Minister Whitlam after a period of instability when the Senate blocked the budget.
Journalist Paul Kelly has written a new book about the dismissal, with a focus on the relationship between Mr Kerr and then-opposition leader Malcolm Fraser.
Following the dismissal, the Governor-General appointed Mr Fraser as caretaker prime minister.
"He says to Kerr, 'Listen, if you don't intervene, if you don't solve this crisis, and if I lose, I'm going to have to go public, and I'll blame you. I will blame you as governor-general, and I'll say that you, as governor-general, refuse to accept your responsibilities, that you genuflected before Whitlam and you wouldn't accept the responsibility to sack him.'"
The subject of the dismissal still divides many Australians.
On the Labor side, it was viewed as an outrageous travesty that ended the term of a democratically elected government.
But many conservatives believed it was a dysfunctional government and the Governor-General acted correctly.
That interpretation is disputed now with what is known.
"The key here is that Kerr decided to sack Whitlam by surprise in a constitutional ambush because he feared that, if Whitlam had any warning or any counsel whatsoever, then Whitlam would move to sack him. So, in that sense, it is a very elemental shoot-out."
The governor-general is the Queen's representative in Australia, and Paul Kelly says, in London, the Royal Family was disappointed in Mr Kerr's actions on behalf of the Queen.
He says they thought he was what they termed "very greedy" and could potentially be embarrassing to them.
Mr Kelly says they helped push Mr Kerr towards an early retirement in 1977.
Meanwhile, in office for less than three years, Gough Whitlam had wasted no time getting work done.
He ended Australia's military involvement in Vietnam, as well as conscription.
He granted Australia's Indigenous people rights to their ancestral land and replaced "God Save the Queen" as the national anthem with "Advance Australia Fair."
Social reforms around free education, legal aid and universal health care remain in place today.
