The suspect, Anthony Bell, 25, was captured at an apartment complex near the church. Three children abducted with the woman were all found safe.
"This is going to be one of the worst days in the history of our city," Police Chief Jeff Leduff said after Mr Bell was captured and his wife's body was found at the apartment complex.
Her name and those of the other victims were not immediately released.
Mr Bell is accused of shooting five people at the church before fleeing, Sgt Charles Armstrong said.
The condition of the fifth person shot at the church was not immediately available.
The wife's body was found at the apartment complex but it was unclear whether she was killed there. Authorities said neither Mr Bell nor his wife lived at the complex.
Investigators did not know the motive for the shooting at The Ministry of Jesus Christ church.
"This is senseless. This is a total waste of human life," Mr Leduff said.
The small church shares space in an old warehouse with a guitar shop in a non-residential area of Baton Rouge.
Mr Bell is believed to have entered and opened fire shortly before services were to end around 10am.
After shooting five people, Mr Bell fled with his wife and with three children, including an infant, police said.
The two older children were found safely at a residence.
Hours later, police responding to a call about a shooting and cordoned off the apartment complex.
Officers found the woman dead and Mr Bell holding the baby, Mr Leduff said.
Mr Bell was arrested without incident.
At first, police said the three children belonged to the couple.
They later said it was unclear whether Mr Bell was the father of all three.