No official reason was given for the resignation of Mr Ouyahia, made in a letter handed to Mr Bouteflika during a meeting between the two.
Mr Ouyahia is secretary general of the National Democratic Rally (RND), one of the parties in the governing coalition, which also includes the dominant National Liberation Front (FNL) and the Islamist Movement of a Peaceful Society (MSP).
Rumours had been circulating for a week that he was on his way out and intensified when he was not seen during a visit to Algiers this week by Turkish Prime Minister Recep Tayyip Erdogan.
His opposition to pay rises for public sector workers was attacked by Mr Belkhadem and by another senior minister, Bouguerra Soltani, head of the MSP.
He implemented an economic liberalisation plan agreed with the International Monetary Fund which was far from popular.
Mr Belkhadem and Mr Soltani called for the departure of Mr Ouyahia, blaming him for delays in implementing Mr Bouteflika's program, according to Wednesday's edition of the French-language daily newspaper El Watan.
Mr Belkhadem, foreign minister from August 2000 to May 2005, had also opposed Mr Ouyahia's plan to make a declaration of general policy to parliament, as laid down by the constitution.
Mr Ouyahia and Mr Belkhadem also clashed over plans to revise the constitution, increasing the number of presidential terms from the present two to three, with Mr Oyahia against the scheme.
Mr Belkhadem told reporters that the identity of the new government team would be known "shortly".
He said he was "honoured by the confidence placed in him" by Mr Bouteflika while conscious of the "heaviness of the task" awaiting him.
Previously a minister of state, and with a reputation as an "Islamist conservative", Abdelaziz Belkhadem is secretary general of the FLN, which enjoyed one-arty rule from 1962 to 1992.
He has also been the personal representative of Mr Bouteflika, whose state of health has attracted speculation in recent months.