Colombian players have complained that the American referee Mark Geiger was biased towards England while their coach Jose Pekerman said football needed to be "protected" after suggesting English players had faked fouls.
But central defender Stones, who as a key player at set-pieces was close to the pushing and shoving at corners, had his own response.
"For me, it was one of those games that was so strange. They are probably the dirtiest team I have ever come up against," he said.
"They were pushing the referee, scuffing the (penalty) spot and there was off-the-ball stuff you have not even heard about," he added.
England won the game after a penalty shootout, having drawn 1-1 over 120 minutes, and face Sweden in Samara on Saturday in the quarter-finals.
Former Argentina World Cup winner Diego Maradona had also chimed in on the game labelling England's victory as a "monumental theft".
"The best thing for us was to beat them at football," said Stones.
"They are on a plane going back home now and we are not. That will hurt them the most".
The England defender said there were incidents that were not picked up that could have merited strong action from the referee.
"I'd rather not go into some of the stuff because I'll get into trouble.
"But I have not seen a game like that before and how they behaved. There could have been several red cards for off-the-ball stuff," he added.
The Manchester City player said that England had done well to avoid being caught up in incidents.
"I thought we did brilliant because we kept a cool head and we have not been dragged into it.
"That's a great thing for a team to have. It was a difficult situation to be in, but we have overcome it and we can be very proud of ourselves," he said.
One incident that was caught by the camera was the headbutt by Colombia's Wilmar Barrios on Jordan Henderson which earned him a yellow card that some observers felt should have been a red.
"If the referee had seen the headbutt it might have been a totally different game. It was so surprising to come across all that stuff happening," said Stones.
(Reporting by Simon Evans; Editing by Christian Radnedge)
