Thousands of raucous Ramaphosa supporters sang and chanted in the conference hall as rival backers of defeated candidate Nkosazana Dlamini-Zuma appeared dejected.
Ramaphosa won 2,440 votes to Dlamini-Zuma's 2,261.
"We declare comrade Cyril Ramaphosa the new president of the African National Congress," an official told party delegates in Johannesburg.
The victory puts Ramaphosa in line to succeed President Jacob Zuma, whose reign has been plagued by corruption scandals, economic slowdown and growing anger at the once-omnipotent party.
The vote was a long and acrimonious process. Delegates who had travelled from around South Africa cast their ballots after repeated delays caused by disputes over who was entitled to vote.
President Zuma was seen as backing Dlamini-Zuma, allegedly to secure protection from prosecution on graft charges after he leaves office.
But his loyalists did win senior positions in the vote, including David Mabuza as party deputy chief, meaning Ramaphosa is likely to face strong internal opposition to his pro-business reform agenda.
Related reading
South Africa's troubled ANC meets to elect new leader
President Zuma stepped down as party chief at the conference but could remain as head of state until the 2019 election.
"I hope you will cooperate with the new leadership... as we move to the 2019 elections," Baleka Mbete, party chairwoman, told delegates.
Falling public support
The ANC, which has ruled since 1994 when Nelson Mandela won the first multi-racial vote, could struggle to retain its grip on power in the next election due to falling public support.
"The party will decide if Zuma goes (before the 2019 election)," Mzwandile Mkhwanazi, a delegate from KwaZulu-Natal province, told AFP.
"Ramaphosa's victory is good for the country. We need a stable country, a president able to fight corruption. We think he is up to the task."
Ramaphosa, 65, is a former trade unionist leader who led talks to end white-minority rule in the early 1990s and then became a multi-millionaire businessman before returning to politics.
He is often accused of failing to confront Zuma while serving as his deputy since 2014.
Dlamini-Zuma was head of the African Union commission until earlier this year and a former interior, foreign affairs and health minister.
She had four children with Zuma before divorcing in 1998.
"I believe Ramaphosa will work to bring back the principles of liberal politics in the party," Amanda Gouws, politics professor at Stellenbosch University, told AFP.
"The outcome of the vote was not easy to call. What is at stake here is unity -- the new leaders need to forge unity and rebuild the image of the party."
Allegations swirled of delegates being targeted with bribes, but ANC spokesman Khusela Sangoni told reporters that the process had proceeded "smoothly".
"I'm bowing out very happy because I think... I made my contribution," President Zuma said on Monday as he walked through the vast conference centre hosting the five-day event.
Soaring unemployment and state corruption have fuelled frustration at the ANC among millions of poor black South Africans who face dire housing, inadequate education and continuing racial inequality.
Party veteran Zikalala Snuki told AFP: "It's democracy. We are happy that we managed to elect the top leadership.
"We have Ramaphosa, who may be able to steer the ship in the right direction."
The opposition Democratic Alliance party said that the ANC was "held together only by the glue of patronage and corruption, and Cyril Ramaphosa is just a new face to the same old ANC."
