Cricket Australia said after the visitors' recent 4-0 defeat that more than 865,000 people had attended the five-test series, the highest since the 1936-37 Ashes.
The day-night Adelaide test saw close to 200,000 spectators while 88,172 people squeezed into the MCG on Boxing Day.
Nevertheless, Moeen is worried.
"I feared in the Ashes actually.... The crowds were quite disappointing," Moeen said on his return to the England squad for a one-day series against New Zealand after being rested for the Twenty20 tri-series.
"There were a couple of big days, but even when they (Australia) won the Ashes there weren't that many celebrating. That's when I thought, 'Actually, we're struggling a bit'.
"It is a worry. Test cricket is the pinnacle. It is in my opinion where the best players play. You can really see who is the best. It's been a worry for a while, but Australia really opened my eyes."
Test cricket has been trying to lure in punters in a number of ways, including the introduction of day-night tests, four-day matches and a new championship.
"When the idea of four-day tests came up I was against it because it's always been five days. But maybe it has to change. I don't really like the pink-ball tests, but maybe if it works and brings crowds in it has to be done."
England take on New Zealand in the first of five one-day internationals in Hamilton on Sunday.
(Reporting by Shrivathsa Sridhar in Bengaluru)
