Veteran Parthiv Patel was chosen as the second wicketkeeper along with the 21-year-old Rishabh Pant, who made his test debut during India's five-match series in England.
Rohit has shown strong form in limited overs matches but was dropped from the test side following a poor return in South Africa earlier this year when he scored 78 runs in four innings.
Vijay and Shikhar Dhawan opened for India in the first match of the England series but the duo were dropped during the 4-1 defeat and also left out of India's two-test home series against West Indies.
The right-handed Vijay, however, responded to his exclusion by making a hundred and three half-centuries for Essex in the County Championship.
"We felt that Rohit's nature of play, a very good back-foot player, his game is suited for Australia," India's chief selector MSK Prasad told reporters.
"Murali Vijay was dropped from the England series. He went back and he played the county and showed his intent and performed and considering the intensity of the series, we've considered him for the Australia series."
The 18-year-old Prithvi Shaw, who made a sparkling debut with a century against West Indies, and Lokesh Rahul were the other openers in the squad.
All-rounder Hardik Pandya, who injured his back during the Asia Cup, has not recovered sufficiently to make the squad and Bhuvneshwar Kumar and Ravindra Jadeja will fill the role in Australia.
"It's very difficult to match Hardik's all-round ability," Prasad said.
"At least we don't see that kind of ability now in India. That's why no other alternatives were discussed. And we expect Bhuvi can come good, we know he can bat."
India's regular wicketkeeper Wriddhiman Saha continues to be on the sidelines, recovering from a shoulder and thumb injury, which opened the door again for Patel, who has played 25 tests for India.
There was, however, good news on the pace bowling front with India travelling to Australia at full strength with Kumar, Jasprit Bumrah, Mohammed Shami, Ishant Sharma and Umesh Yadav all available.
Off-spinner Ravichandran Ashwin and left-arm wrist-spinner Kuldeep Yadav will be the slow bowling options alongside Jadeja.
"If you see the selection we have done, it's a mix of youth and experience which we really wanted because it's going to be a high-intensity test series," Prasad said. "That's why we fell back on experience."
India will play a three-match Twenty20 series before the first test starts in Adelaide from Dec. 6.
Perth, Melbourne and Sydney will host the remainder of the tests followed by a three-match one-day international series.
Squad: Virat Kohli (captain), Murali Vijay, Lokesh Rahul, Prithvi Shaw, Cheteshwar Pujara, Ajinkya Rahane (vice-captain), Hanuma Vihari, Rohit Sharma, Rishabh Pant, Parthiv Patel, Ravichandran Ashwin, Ravindra Jadeja, Kuldeep Yadav, Mohammed Shami, Ishant Sharma, Umesh Yadav, Jasprit Bumrah, Bhuvneshwar Kumar.
(Reporting by Sudipto Ganguly; editing by Greg Stutchbury)
