Rahimov, an Uzbek who was elected in November, claimed "politically based discussions" were harming the organisation's progress as the sport battles to remain part of the Olympic programme.
AIBA said in a statement its executive committee had elected Moustahsane, head of the African Boxing Confederation, by a 23-0 vote with two abstentions.
Rahimov was elected despite being on a U.S. Treasury Department sanctions list "for providing material support" to a criminal organisation, a claim the Uzbek strongly denies.
The International Olympic Committee (IOC) has launched an inquiry into the finances and governance of AIBA, which has been in turmoil for years, and warned that it could miss out on the Tokyo Games next year.
The IOC's Executive Board has frozen planning for next year's Olympic boxing tournament and suspended contact between AIBA and the Tokyo 2020 Organising Committee.
(Reporting by Alan Baldwin, editing by Christian Radnedge)
