The reforms were promoted by the previous centre-right government led by Silvio Berlusconi who argued that Italian politics needed more stability.
Italian parliamentarians are now likely to go back to the drawing board to decide how to ensure more stability in a country which has had 61 governments since 1945.
Just over half the 47 million registered voters turned out to cast their ballots in the two-day referendum.
The current centre-left government of Romano Prodi, was deeply opposed to the reform bill, which was sponsored by the populist Northern League party.
The Northern League leader, Umberto Bossi, had made the progression of the bill a prerequisite for his continued support of Mr Berlusconi.
He has raised the prospect that the Northern League will split from the centre-right coalition - a move that would undermine Mr Berlusconi's leadership of the opposition.
Under the reforms, the prime minister would have been granted powers to dissolve parliament, appoint and dismiss ministers and determine the general direction of government policy.
These were all powers that were deliberately kept out of the hands of the prime minister in the 1948 constitution, as a way of preventing the emergence of another figure like fascist dictator Benito Mussolini.
The bill would also redefine the role of Italy's two parliamentary chambers.
The lower house, the Chamber of Deputies, would take the dominant role in matters of national interest like foreign policy, defence and immigration while the upper house, the Senate, would become responsible for federal law.
However, most significantly, the bill would give greater autonomy to Italy's 20 regions which would have gained control over education, healthcare and law and order, and would win special representation in Italy’s Supreme Court.
Some estimates say regional government would have taken control of around 40 percent of public expenditure.