Police said the 28-year-old from Adelaide disappeared in the water off Goldsmith Beach, near Yorke Peninsula, with a helicopter and boats at the scene looking for him.
"At about midday, the man was part of a group spear-fishing at a popular area at Goldsmith Beach when witnesses report seeing a shark attack the man," police said.
Locals said there had been reports of a great white shark in the area in the past week and authorities hold little hope of finding him alive.
"They haven't found any trace of him at this stage, they're still looking but it's not looking very hopeful," a police spokesman added.
"They haven't located him yet so they believe it's a fatal shark attack."
The incident comes a week after thousands of people rallied across the country against a controversial shark culling policy in Western Australia designed to prevent attacks.
The policy to catch and kill sharks off popular west coast beaches was given the green light last month after six fatal attacks in the past two years.
It is aimed at reducing the risks to water users and allows baited drum lines with hooks designed to capture large sharks to be set one kilometre (0.62 miles) offshore for a trial period until April 30.
It is not in effect in South Australia where the latest attack took place.
The move has angered conservationists who claim the trial flies in the face of international obligations to protect the great white shark, arguing there is no scientific evidence that killing them will reduce the risk to swimmers.
While sharks are common in Australian waters, deadly attacks are rare, with only one of the average 15 incidents a year typically proving fatal.
But experts say attacks are increasing in line with population growth and the popularity of water sports.
