The Premier League side twice led with goals by Richarlison and Cenk Tosun but Millwall hit back quickly on both occasions before Murray Wallace struck in stoppage time to give Millwall a surprise 3-2 victory.
Everton felt aggrieved, however, that Millwall's second equaliser appeared to go in off the arm of Jake Cooper.
Referee Michael Oliver allowed the goal to stand; had it been referred to VAR it is almost certain he would have seen enough to disallow it.
Premier League grounds hosting FA Cup ties are using VAR this season but Millwall's New Den did not have the system installed -- a situation Silva said was unfair.
"It doesn't make sense," he told a news conference. "If we are to be fair with all the clubs you have to be fair with the competition and also you have to do it for all the games.
"Now we go home with big frustration. I am not here just to be angry with the referee but it is true."
To make matters worse the goal was shown on a big screen in the stadium immediately afterwards, with the handball obvious to everyone watching.
"It was a clear handball in the moment. The players they saw and went to speak with the assistant and afterwards thousands of people saw on the big screen," Silva said.
"I think they look also (at the screen) and the referee looks also. We have to be fair with all the clubs and put VAR in all the stadiums."
(Reporting by Martyn Herman; editing by Clare Fallon)
