Brown said in a statement that he would take his experience "forward to my next chapter in the arena I know best, motorsport." He could not immediately be reached for further comment.
Formula One is facing a major shake-up after U.S. cable TV mogul John Malone's Liberty Media agreed this month to take control. The deal is likely to lead to a new push to develop the U.S. market and win fresh audiences.
Media reports have suggested that new Formula One chairman Chase Carey wants to appoint more executives to run the commercial and sporting operations, until now the preserve of 85-year-old supremo Bernie Ecclestone.
CSM said that Brown, whose agency has introduced top sponsors to Formula One, would remain involved in CSM as chairman of its global advisory board and non-executive chairman of its JMI motorsport arm.
The 44-year-old founded JMI sports marketing, which was bought by CSM, a division of Chime Communications, in 2013. IAAF President Sebastian Coe is chairman of CSM.
Liberty Media has acquired an initial 18.7 percent stake in Formula One from controlling shareholder CVC Capital Partners and plans to complete a cash and shares deal by the first quarter of 2017.
Brown, who owns a number of old F1 cars including an ex-Ayrton Senna Lotus 98T and Michael Schumacher Benetton B191, said after the deal was announced that he could not think of a better buyer.
"If you had to write down a list of who you would want to buy Formula One, Liberty would be top of that list," the executive, whose company is also active in U.S. racing series, told Reuters at the time.
"They've sold it to a strategic buyer as opposed to another financial buyer. So they will be making decisions that are strategic-led, what's best for the growth of the sport and the fans and the sponsors."
Brown was also at this month's Italian Grand Prix at Monza with Coe.
In an interview with Reuters last year, Brown voiced concerns about the sport and said it had lost some of its buzz, with budgets out of control and excessive hosting fees and ticket prices.
"The potential for the sport is untapped. I think it needs, as an industry, to be much better marketed. It's not where it needs to be," he said in that interview.
(Reporting by Alan Baldwin, editing by Keith Weir)
