However, the independent review recommends a separate area be set up within the facility to accommodate "vulnerable transferees" who need to move out of the single adult male compound for their own safety.
The report by lawyer Robert Cornall was sparked by an SBS Dateline program in July, in which the former head of occupational health and safety at the centre, Rod St George, said staff knew as many as half a dozen young men were being assaulted and sexually abused.
Mr St George also claimed acts of self-harm and attempted suicide were occurring "almost daily" at the Papua New Guinea facility, where he worked for a month before quitting in April.
The whistleblower said others were forced to sew their lips together and one asylum seeker had solvent poured into his ear.
But Mr Cornall, a former head of the federal Attorney-General's Department, said he found no evidence of sexual abuse and torture occurring with full knowledge of staff.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The allegation that victims were being returned to the single adult male compound to be raped again were also untrue, his report said.
The highly redacted report - dated September 30 but only recently posted on the Immigration website - says the major allegations of sexual assault involved a young man known as "Mr A".
The review says the "general understanding" was that Mr A was sexually assaulted twice in the single adult male compound and reported the abuse.
It says staff at the centre had discussed moving Mr A to the family compound, but the Immigration Department had deemed it unacceptable "as single adult males could not live in a compound with families and children".
The report says Mr A eventually returned to the single adult male compound of his "own volition".
The allegations were reported to PNG police, but Mr A refused to speak to local authorities or to the review.
Alleged sexual assaults and harassment of up to five other young men in the single male compound were not reported by the victims and could not be verified.
The Cornall review did find incidents of self harm, assaults on guards, two men sewing their lips together and escapes.
But the report says "they were distorted or their significance was either exaggerated or misunderstood" by the television program and subsequent press reports.
Allegations of criminal behaviour were reported to police, and in some cases charges were laid.
Since all transferees accommodated at Manus have now left the country, they are beyond the jurisdiction of the PNG justice system.
Mr Cornall called on the immigration department to establish a "clear and well understood" policy for dealing with future sexual assault allegations.
Media should also be given access to the processing centre, the report says.
"Appropriate media reporting about the Manus (centre) could reduce the likelihood of the dissemination of the sort of misinformation or misunderstanding which led to this investigation," the report says.
Comment is being sought from Immigration Minister Scott Morrison.
