People who had paid to be smuggled to Australia refused to leave fearing a boat would never make the journey, a court has been told.
But others embarked on the expedition, some even threatening the captain of the vessel that became known as SIEV-X warning him to continue, witnesses have told Brisbane magistrate Mark Nolan.
The testimony was heard in the committal hearing of Iraqi national Maythem Kamil Radhi who is accused of involvement in a syndicate that attempted to smuggle 421 people on an overcrowded and dilapidated vessel in 2001.
The attempt ended with 353 people drowning, 146 of them children.
Radhi, 45, is not charged over the deaths but faces one count of bringing groups of non-citizens into Australia, while two men have already been imprisoned for their involvement.
About 13 hours into the journey the boat broke down, Raad Zubari, who escaped the boat before its passengers were in danger, told the court on Thursday.
He said the captain wanted to return to Indonesia saying it was "impossible to continue" due to the type of vessel and the number of people aboard.
Mr Zubari said he saw people threaten the captain many times.
"[The captain] knows about the boat and the condition of the sea and the waves so he stopped many times. But each time they tried to convince him by money or sometime they threatened him and hit him," Mr Zubari said through an interpreter.
Quasy Al Majid, who got onto a fishing boat before the SIEV-X sank, earlier told the hearing some people offered the captain money when he did not want to continue with the journey.
The captain replied that he did not want money and the boat would not make it to Australia, according to Mr Al Majid's statement read in court.
Mr Al Majid said he saw the captain's Indonesian assistant hit on the head, while a woman with a knife threatened to kill the captain if he didn't continue on the journey.
The hearing was told Mahmod Yussef and his wife boarded the vessel but disembarked before it left, returning to shore on a smaller boat.
Mr Yussef said he told a man behind the scheme the vessel would sink.
"(I said) we can't continue, I don't want to go because many people are on board," he added.
The case against Radhi suggests he played a "facilitation" role before the ill-fated voyage left Indonesia, commonwealth prosecutor Daniel Caruana said at the start of the hearing.
Some witnesses say he was present when money was negotiated, but they "more consistently" remembered him as an organiser, Mr Caruana said.
Much of the allegation centres on taking care of logistics, including buses between hotels and beaches.
Witnesses said they were taken by bus to a hotel in Sumatra where they spent days waiting for the boat.
But there were up to eight rooms only for women and children so the men stayed on the hotel grounds.
Radhi surrendered two years ago after deciding to stop pursuing appeals against his extradition from New Zealand.
An arrest warrant was issued in Brisbane Magistrates Court for him in 2011 but the process was delayed while his eligibility for extradition was argued in courts in New Zealand, where he had been living with his wife and three children since 2009.
The committal hearing has been adjourned until 1 December for submissions.
