Despite the win, their first in the Premier League since the opening day of the season in August, Villa remain glued to the bottom of the standings, eight points from safety.
"But let's say this victory could give us a little bit more confidence. But it's not just the win," Frenchman Garde, who took over from Tim Sherwood in November, told reporters.
"I want to insist that the way we played tonight, I didn't see a bottom of the table side. We have to stay very humble. It's only three points. We are still far away from what we all want."
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Crystal Palace manager Alan Pardew described his side's performance as a "disjointed offensive display" after Wilfried Zaha's first minute strike against the post and Damien Delaney's first-half volley were the only chances of note for the London side.
"It's difficult to give my thoughts really. The lack of offensive power going forward is becoming a problem. It really was a disjointed offensive display," Pardew said.
"We have to acknowledge that our quality, our game plan didn't come to light in the first half and it didn't come to light in the second half, so we've got some work to do."
Palace, in seventh place, have not scored in their last four league games."
(Reporting by Shravanth Vijayakumar in Bengaluru; editing by Justin Palmer)
