Liverpool manager Klopp believes a walk off because of a minority of supporters would give "too much power for one or two idiots", but said the decision should be down to match officials if abuse escalated.
"If it's the whole stand I would do it, 100 percent," Klopp told reporters ahead of Sunday's Premier League clash against Tottenham Hotspur.
"But it's too much power for one or two idiots. We have to find ways to punish these incidents.
"It's not that the coaches should do that - match officials - because if I take the players off and they say it was not that serious, you lose the game, our job is to win football games."
Guardiola is also in favour of taking his Manchester City players off the pitch, adding football should try and set an example for action against racism.
"We could do that (walk off)," the Spaniard added. "Football is a strong weapon to defend the principles of humanity. People said you can't mix football and politics but that's not true.
"It's not a football problem, it's society itself. I think in Europe, not in football, it's getting worse."
The Football Association Board met on Thursday to discuss the incidents in Montenegro and called on a structured campaign within UEFA and FIFA to ensure that incidents such as this are dealt with appropriately.
The Board also agreed that there should be more transparency on FA charges going forward when clubs are found to be in breach of the rules following such incidents in the domestic game.
Chelsea manager Maurizio Sarri, meanwhile, wants the sport's governing bodies to introduce clear guidelines to stop matches in order to combat racist abuse.
"We need a rule, I think. A rule for the Premier League, but for all the championships across football. A rule that can permit us to stop the match," Sarri said.
(Reporting by Hardik Vyas in Bengaluru; Editing by Christian Radnedge)
