Mr Chipp made the colourful promise to keep the government on the straight and narrow when he defected from the ruling Liberal Party, where he had held ministerial posts, to form the Australian Democrats in 1977.
Tributes poured in from all sides of the political spectrum after it was announced that Mr Chipp had died on Monday night after a long battle with Parkinson's disease.
Mr Howard said Mr Chipp was a "colourful figure in Australian politics" who had made a major impact on the country.
"To me, the most endearing thing about Don Chipp was, to the very end, he was passionate and committed to the causes in which he believed.
"Don Chipp has made a huge impact on Australian public life over a very long career and I know he will be greatly missed."
Opposition Labor Party leader Kim Beazley said Mr Chipp "made a very serious contribution to the character of Australian politics".
Australian Democrats leader Lyn Allison, one of four Democrats in the senate where the party has its only representatives in the federal parliament, said Chipp was "one of the greats in Australian politics".
"He propelled the Democrats into the federal political arena – inspiring hundreds of thousands of people to join him in developing a third force in Australian politics.
"He was a liberal thinker, a strong advocate of social justice and a man who voted according to his conscience - a legacy of which we can all be proud."
Australian Greens leader Bob Brown described Mr Chipp as a "rambunctious, outspoken and very deeply humanitarian character".
Mr Chipp entered federal politics in 1960 and, after serving as a cabinet minister in three governments, became disillusioned with party politics under the leadership of the then-prime minister, Malcolm Fraser.
Mr Fraser's maneuverings as opposition leader led to the 1975 dismissal of the left-leaning Gough Whitlam Labor government, still one of the most controversial episodes in Australian political history.
Mr Chipp resigned from the Liberals in March 1977 and founded the Democrats with the famous pledge the new party would "keep the bastards honest".
He won a Senate seat later that year and led the party until he retired in 1986.
He is survived by his second wife Idun Welz and six children.