Now 31, the Ugandan is still fighting and next week will step back into the Olympic ring in Rio eight years after appearing at his first Summer Games in Beijing.
As a young boy taekwondo, not boxing held Serugo's interest but the skills he had acquired proved little help when he came to the aid of an even younger child being attacked by four youths.
"I thought I could take them all on,” recalled Serugo. "But instead they were like, 'Who the hell are you?’ and they beat me so badly.
"I tried to fight them with my kicks but they knocked me down, climbed on top of me and continued hitting me. It was really bad. I couldn't see at all by the end."
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Lesson learnt.
Determined to have his revenge, Serugo joined a boxing club operated by local police and quickly developed an interest in the sweet science.
"At first it was tougher than I thought," said Serugo, who will fight in the 52kg flyweight division.
"The beatings I got in the gym were far worse than that time on the street. But I hate being defeated. If I lost, then I would go back, learn more and more techniques until I got my revenge.
“I didn't think it would teach me discipline but in the end it gave me more discipline than rugby, football, taekwondo or any other sport I tried.”
Boxing has also given Serugo and his family a better life.
"Boxing has helped my family a lot," said Serugo. “Now we have a water tank and we can sell water for more money to help pay for food and other things.
"We don't have to struggle so much anymore.”
(Reporting by Steve Keating; Editing by Rex Gowar)
