The 33-year-old lefthander from Luxembourg will meet Briton Dan Evans, who later beat Andrey Kuznetsov 6-3 3-6 6-3, in the final on Saturday with both players looking for their maiden ATP Tour titles.
Muller, who had reached the last four for a third year in a row, started well despite the afternoon heat and wrapped up the opening set in half an hour with an uncompromising forehand smash.
Troicki stepped up his game and the second set was a much tighter affair but Muller saved the two break points he faced to take it to a tiebreaker.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The Serb saved the first match point when Muller netted a return but the world number 34 would not be denied on his second, firing his 16th ace to reach the final.
"When I came here I wasn't expecting very much because I didn't play very well in Brisbane, I wasn't feeling great," said the sixth seed, who has been suffering from a virus.
"I was expecting to have one or two more matches before the Aussie Open and I'm in the final so it's a great feeling."
British world number 67 Evans stunned top seed and world number eight Dominic Thiem in the quarter-finals and made the most of Kuznetsov's misfiring first serve to win the first set interrupted briefly by rain.
Kuznetsov got his serve functioning to even up the match in the second but it faltered again in the third and Evans grabbed a key break for 3-1 before serving out the match to love after another rain break to reach his first main tour final.
"I'm excited, I'm happy with how I played, to come back in the third set and get the job done," said 26-year-old.
Evans's victory in the final match of evening made it a good day for Britain in Sydney with Johanna Konta taking the women's title and Jamie Murray reaching the men's doubles final with his Brazilian partner Bruno Soares.
(Editing by John O'Brien and Sudipto Ganguly)
