"He continues to be present in our minds and in our hearts," Pope Benedict said, reflecting on the life of the late pope from the window of his apartment, where John Paul II died on April 2, 2005 after a 26-year pontificate.
"One year has passed since the death of our beloved John Paul II, which happened at this precise moment, 9:37 pm, but his memory continues to be more alive than ever," he said.
Some 80,000 pilgrims were massed in the square clutching lighted candles for the prayer service, a moving tribute to the lasting appeal of the late pope.
"The entire city of Rome has gathered symbolically for this moving moment of reflection and prayer," said Benedict.
Benedict said John Paul II, who suffered from Parkinson's disease, faced his long illness "with courage -- which made everyone more aware of human pain, physical or spiritual."
In a leaf taken from his famous polyglot predecessor, the German pope greeted Poles -- watching by a special live television link from John Paul II's home cathedral in the Polish city of Krakow -- in their own language.
"The memory of John Paul II is alive in us and the feeling of his spiritual presence will never fade," Benedict said in Polish. Repeating his predecessor's words, he told them: "Remain strong in the faith."
Leading the commemoration in Krakow was the late pope's faithful private secretary, now cardinal, Stanislaw Dziwisz.
"Thank you, thank you, Holy Father," responded Dziwisz, his message broadcast on big screens erected around the Vatican square.
Many pilgrims packed inside the Vatican square were moved to tears by the ceremony.
"We were here at this moment a year ago, and now we are reliving the whole thing all over again," said Grazyna Szemraj, one of thousands of Polish pilgrims in Rome for the event.
Many held up the red and white flag of the late pontiff's native Poland and chatted happily ahead of Benedict's appearance, while others stood in silent reflection.
Alicia Zapawa, a Polish woman living in Rome, said commemorating the pope's passing was "an important moment for the whole world."
Behind her, a group from Poland held up a giant banner proclaiming "John Paul the Great, Make him a Saint Now."
"I hope he will be a saint soon," said South Korean student Kang In Chul.
Jean Mrozek said he felt particularly emotional because he was in the Vatican square on October 16 1978 for the announcement of his fellow Pole Karol Wojtyla's election as pope.
"I cried so much back then. For sheer joy. I live here behind the Vatican so we practically lived together for 27 years, John Paul II and I," he said.
"I pray for him and I'm sure he's looking down on us from heaven, watching over us."
The Vatican's final anniversary event for the late Pope John Paul II is scheduled for Thursday afternoon in St. Peter's Square.