The Austrians said the testers came at 2255 local time in Sochi on Tuesday, disrupting her preparations for the Olympic downhill where she placed 16th.
Austria then lodged an official complaint with the IOC.
"That (test) was carried out within IOC rules and standards," IOC spokesman Mark Adams told reporters.
He said testers were allowed to perform announced or unannounced tests from 0600 to 2300 local time whether they pre- or post-competition tests.
"We have a zero tolerance policy on doping," he said.
Goergl had been among the favourites for the race, having won bronze medals in the downhill and giant slalom at the Vancouver Olympics four years ago and winning world titles in downhill and super-G in 2011.
"It's unfair on the athletes," said Austrian Alpine skiing team sports director Hans Pum. "We're in favour of doping tests but not on the evening or the night before a competition."
This is not the first time athletes and teams have complained about late or frequent tests.
Jamaica were furious over what they said were too many blood tests on subsequent Olympic champion Usain Bolt at the Beijing 2008 Olympics.
Austria has also had a sketchy doping past at Winter Olympics with Italian police and testers raiding the accommodation of Austrian biathlon and cross-coutry skiers at the Turin Olympics in 2006.
More than a dozen athletes were tested, syringes and drugs were found, while a former coach, banned from the Games, fled across the border following the raid after a tip-off to authorities.
The IOC will conduct about 2,500 doping tests during the Games, including 1,269 pre-competition controls. So far no athlete has tested positive for banned substances.
(Editing by Peter Rutherford)
