The Pakistan-born 30-year-old said in a blog post that racial abuse was widespread on the playing field during his junior days and blamed it for curtailing the sporting ambitions of a number of his overseas-born friends.
"Getting sledged by opposition players and their parents was the norm," Khawaja wrote on the PlayersVoice website.
"Some of them said it just quietly enough for only me to hear. It still hurt, but I would never show it. Most of the time it was when I scored runs.
"Some parents take things too seriously.
"It is for this reason why so many of my friends, most of whom were born outside Australia, didn't support Australia in sporting contests. I didn't either.
"Especially in cricket. It was either West Indies, Pakistan, India, Sri Lanka. Anyone else."
The first Muslim to play test cricket for Australia, left-hander Khawaja has scored 1,728 runs in his 24 tests at an average of 45.47, and is likely to be selected at number three for the upcoming Ashes series against England.
He said the hard-edged Australia teams of his childhood held little appeal.
"In hindsight, the fact we didn't support Australia is disappointing," he added.
"Everything that was going on in our childhood and around us built up this resentment of the Australian cricket team.
"I mean, none of them looked like us.
"I was brought up to be respectful, humble and polite. But when I watched the Aussie team, I saw men who were hard-nosed, confident, almost brutish.
"The same type of men who would sledge me about my heritage growing up."
Khawaja credited his "strong-willed" family and his own competitive nature for his international breakthrough but said a number of aspiring players had been thwarted by racism.
"My point is this: it's no surprise it has taken Australia cricket so long for coloured players to come through the system," he said.
"There is no doubt racism and politics played a large role in selections in the past.
"I've heard a few stories from past Anglo-Saxon players where this seems to be the case. It would just be the times that they lived in. Certainly cricket and society has come a long way.
"Now sub-continental parents can see a future for their kids, at a younger age."
(Writing by Ian Ransom; Editing by John O'Brien)
