Several England players have also expressed concerns ahead of the World Cup, with defender Danny Rose saying that he had asked his family not to travel to Russia for fear of racism, but Boban has brushed off their worries.
"I don't believe we will see these idiots, but we have procedures. I believe we do enough at FIFA," Boban told The Times of London newspaper.
"We have to look after the players, but how far do you punish the game for a few idiots? To punish a whole city, a whole country when less than one percent of the people are guilty?"
Boban, a former AC Milan defender, says that FIFA's three-step protocol in the event of racist behaviour at Russia is a landmark plan that will deter perpetrators.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Kick It Out, an organisation dedicated to achieving equality and inclusion in soccer, said they were disappointed but not surprised with the relaxed approach that a high-ranking FIFA official had towards combating racism.
"While we welcome FIFA's commitment to the implementation of the three-step protocol at the tournament, it is laughable to suggest it has done enough to deter discriminatory behaviour up to this point," the organisation said in a statement.
"The racial abuse of French national team players by Russian supporters just three months ago is clear evidence of that."
Russia has pledged to crack down on racism ahead of the World Cup, which will be held from June 14 to July 15, with FIFA President Gianni Infantino saying he had no concerns of discrimination at matches earlier this month.
(Reporting by Aditi Prakash in Bengaluru; Editing by Christian Radnedge)
