The new city law, which takes full effect several months before the Olympics open on July 24, 2020, bans smoking in any bar or restaurant with hired employees.
"We've managed to prepare ourselves for such mega-sports events as the Rugby World Cup in 2019 and the 2020 Tokyo Games," Tokyo Governor Yuriko Koike told reporters.
The Tokyo law is far tougher than legislation under debate in the upper house of Japan's parliament to tackle the threat of secondhand smoke, which kills about 15,000 Japanese, many of them women and children, every year, activists say.
Last week, the lower house passed a watered-down version of a health ministry proposal to limit secondhand smoke in places such as schools and hospitals, but allowed smoking in outdoor spaces.
The Tokyo law will forbid outdoor smoking in places where children may be present.
Less than a fifth of Japanese smoke today, down from about half the population 50 years ago.
Last week, a member of parliament jeered a lung cancer patient who was testifying about the dangers of secondhand smoke.
(Reporting by Elaine Lies; Editing by Darren Schuettler)