Amanda Lampe had decided to "move on" because of changed personal circumstances, Ms Gillard said in a statement on Monday.
She will be replaced by Ben Hubbard, the outgoing boss of the Victorian Bushfire Reconstruction and Recovery Authority.
Mr Hubbard previously was Mr Gillard's chief of staff when she was deputy opposition leader and deputy prime minister.
"He brings significant public administration, stakeholder and management experience to the role," Ms Gillard said.
Ms Lampe will leave at the end of March, allowing a two-month transition period that starts from next Monday.
Ms Gillard thanked Ms Lampe for her contribution to her office, saying her extensive experience and skills had been invaluable.
"I wish her and her family the best for the future."
Ms Lampe's performance as the prime minister's chief of staff was the subject of internal Labor Party criticism during last year's election campaign.
Her resignation comes at a time when Ms Gillard battles to sell her plan for a taxpayer levy to help fund flood recovery efforts.
