Hostages were taken and an 84-year-old priest was killed while saying mass.
The Saint Etienne du Rouvray church in Normandy was the target of two terrorists claiming to be followers of IS.
Both were shot dead shortly after they reportedly cut the throat of Father Jacques Hamel.
He was 84.
They also took members of the congregation hostage in their attack.
Paris prosecutor Francois Molins says terrorist threats can now come from anywhere, as he described the police operation.
"The police attempted to enter the church but were not able to do so because three hostages were being used as a barrier by the doors. Shortly afterwards the three hostages immediately left the church followed by the two terrorists who were carrying hand-held weapons. And then they shouted Allahu Akhbar and were attacked by the police."
The attack has prompted calls for national unity, not to allow the latest string of Islamist-motivated attacks to divide the French people.
French President Francois Hollande says his country is at war and it's going to be a lengthy battle.
"The target is our democracy and it will be our shield. It is our unity that makes our strength. French people, let us be united and that is how we will win against fanaticism. Because I can assure you, this war we will win. Long live the republic, long live France."
The United States has condemned the attack in the strongest possible terms, with White House spokesman Eric Schultz describing it as horrific.
"We offer our condolences to the family and friends of the murdered priest. And our thoughts and prayers are with the other victims of the attack, as well as the parishioners and community members of this church. France and the United States share a commitment to protecting religious liberty for those of all faiths and today's violence will not shake that commitment."
A Catholic Church spokesman says Pope Francis is horrified by the news.
The Archbishop of Rouen, meanwhile, has called on people to love their enemy.
France remains on high alert following a mass killing in Nice on Bastille Day.
A state of emergency is still in place 10 months ahead of a presidential election, where Mr Hollande is expected to seek a second term.
