Mr Beazley promised at the weekend to abolish the individual contracts known as Australian Workplace Agreements or AWAs if Labor wins office.
Mr Beazley is already facing claims he failed to consult broadly within his own party before making the announcement, but he has denied Labor is split over the move.
Mr Andrews says Mr Beazley's proposal would damage high employment and wage growth.
The minister says scrapping AWAs would result in people being pressured to join unions.