Mr Straw, who remains in Prime Minister Tony Blair's cabinet, reiterated that he was concerned the veil was a "visible demonstration of separateness," voicing disquiet over the development of "parallel communities" in Britain.
His remarks again provoked fury among many activists, with lawmaker and Respect party leader George Galloway calling for his resignation.
Mr Blair's spokesman insisted Mr Straw was expressing "personal views" which did not represent government policy, although he expressed support for discussion of the issue.
Discomfort
In his regular column Thursday in his local newspaper, the Lancashire Evening Telegraph, Mr Straw expressed discomfort at talking to someone he could not see.
But in a follow-up interview with BBC radio, he broadened his remarks to suggest that the veil had become a symbol of separateness between ethnic groups.
"I started this practice I think well over a year ago...because I had observed in the street that, although it's still a tiny minority, more women were wearing the veil and picked up quite considerable concerns about this being a rather visible demonstration of separateness," he said.
Asked if he would prefer that women did not wear veils, he said: "I'm not talking about being prescriptive but with all the caveats, yes, I would rather."
Mr Straw also suggested that some women were not fully aware of "the potential in terms of community relations" of wearing the veil.
Mr Galloway, formerly a colleague of Mr Straw before he was ousted from the Labour party over his vehement opposition to the Iraq war, said that Mr Straw was effectively asking women to "wear less".
"It is not women choosing to wear what they want that is sowing division in our society.
"It is poverty, racism and the despicable competition between the Tory and New Labour frontbenches over who can grab the headlines as the hammer of the Muslims," he said.
Mr Straw is currently Leader of the House of Commons, a government business management role, and has been widely tipped to stand as deputy leader of the ruling Labour Party when Mr Blair and his number two, John Prescott, step down within the next year.
Many of his Muslim constituents condemned his remarks -- one, 34-year-old Baksedha Khan, accused Mr Straw of "looking for publicity" ahead of a deputy leadership bid.
There are around 1.6 million Muslims living in Britain.