Villa have endured a woeful start to the season and have won just two of their 22 league games.
They are 10 points adrift of Swansea City, who are one place above the drop zone in 17th, with 16 matches remaining.
"One of the reasons this club has not done well over the last five seasons is that it has not had any stability," Hollis told Sky Sports television. "The last thing I will do as the incoming chairman is actually look to apportion blame.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"That is going to just put us in the exact same position we have seen over the last four or five years. My job is to build stability, build confidence in the leaders in the different parts of this business."
Garde was appointed in November following the sacking of Tim Sherwood but the Frenchman has struggled to turn around the fortunes of the Midlands club who have been in the top flight since 1988.
The former Arsenal midfielder, who previously coached Olympique Lyonnais, is Villa's sixth manager since 2010.
Hollis, appointed last week, also expressed frustration at Villa's recent record in the transfer market and said it was not through a lack of spending that the club found themselves in their current position.
The 1982 European champions spent around 50 million pounds in the close season following the departures of Christian Benteke to Liverpool and Fabian Delph to Manchester City, with Jordan Ayew, Rudy Gestede and Adama Traore among the arrivals.
When asked whether the money had been well spent, Hollis replied: "I think the results are the best indicator on answering that question".
($1 = 0.7036 pounds)
(Reporting by Toby Davis; Editing by Tony Jimenez)
