Ms Burney acknowledged "for many, many people, there is a belief that this date should change" but that a national conversation around First Nations history was more important at this stage.
"The most important thing is that the truth be told in this country … that truth be told about the way in which Aboriginal people suffered at the hands of colonisation right across this country, the truth of child removal, the truth of denial of culture, the truth of denial of country."
She urged Australians to "actually reflect today on the history of First Nations people … and on the very fact that this nation was not peacefully settled, as we were taught in the history books many years ago".
This nation was not peacefully settled
"It should be a day of truth-telling, it should be a day of recognition," she told the crowd.
The Yabun festival, meaning "music to a beat" in Gadigal language, is held in Sydney each Australia Day.
Thousands of Sydneysiders attended the 2020 event, featuring music, performances, stalls and community forums on Indigenous issues.
Australia Day is celebrated on the date when the First Fleet arrived in Sydney Cove and raised the Union Jack to signal the beginning of the colony.
Amid "Invasion Day" marches around the country, Prime Minister Scott Morrison released an Australia Day message calling for unity.
"Whether our families came here tens of thousands of years ago, generations ago as mine did on the First Fleet, or those who are taking citizenship for the first time today, we're all together as one and we can altogether be proud," he said.
"We are a free, diverse and accepting people … Our way is to see the humanity of others regardless of their ethnicity or disability or age, religion, gender, all these things. We accept and embrace people for who they are."
Material from the Australia Day Council says "more than half of all Australians participate in Australia Day, attending events organised by state governments, local councils, community groups or getting together with family and friends".
In addition, more than 27,000 new Australians will become citizens on Australia Day at ceremonies around the country.