The boss leaves after only taking charge in February, but has paid the price for the club's dismal start to the Barclays Premier League season.
Villa lost 2-1 at home to Swansea on Sunday (AEDT), a sixth straight defeat in the league, and it is four points from the safety line, having not won since the opening day.
Assistant manager Ray Wilkins, first-team coach Mark Robson and performance analyst Seamus Brady have also left, with Kevin MacDonald, who has been coaching the Under-21s, placed in temporary charge.
A club statement said: "Aston Villa Football Club has parted company with manager Tim Sherwood today.
"The board has monitored the performances closely all season and believes the results on the pitch were simply not good enough and that a change is imperative.
"However, the club would like to place on record its sincere thanks to Tim for all his efforts during a difficult period last season and for the many positive contributions he has made to the entire football set-up during his time with the club. We wish him well in the future.
"Assistant manager Ray Wilkins, first-team coach Mark Robson and performance analyst Seamus Brady have also left with immediate effect and do so with our best wishes.
"A search for a new manager has commenced and the squad will be managed in the interim by Kevin MacDonald."
Sherwood had vowed to fight for his job, but admitted last week he had been given no reassurances over his position by CEO Tom Fox or owner Randy Lerner.
And, after the loss to the Swans, he conceded he did not speak to chairman Lerner but used sporting director Hendrik Almstadt as a go-between.
Sherwood replaced Paul Lambert in February and guided Villa to safety last season, as well as taking it to the FA Cup final, where it lost 4-0 to Arsenal.
The ex-Tottenham boss leaves Villa having lost 16 of his 28 games in charge, winning 10 and drawing two.
MacDonald, who spent nine years at Villa as assistant manager between 2003 and 2012, became caretaker manager in 2010 after Martin O'Neill left and Sherwood only brought him back to the club in February after a coaching spell at Nottingham Forest.
His first game in charge will be the Capital One Cup fourth round trip to Southampton.
