“Now that's not in the interests of those individual migrants, but nor is it in the interests of social cohesion.”
The minister is expected to outline the policy in a speech in Sydney on Thursday afternoon.
The government already requires English skills for some migrants, but it depends on the visa. Those on skilled visas and student visas are already required to demonstrate language skills, but their spouses and families are not. The family reunion scheme and the humanitarian scheme for refugees do not have English requirements.
The government offers 510 hours of free English lessons for migrants with poor English when they arrive in Australia, but not all migrants take them up.
Labor said it would consider the proposal when the details are released, but shadow immigration minister Shayne Neumann said it would be "totally unreasonable to apply an English language test to Australia’s humanitarian program".
Refugees often have little or no English when they arrive in Australia, unlike skilled migrants who are sponsored to work in the country.
Greens leader Richard Di Natale said learning English could be a "real struggle" for newly arrived migrants who were juggling work and family life.
"Let’s help them and encourage them to learn English, but let’s not make this a punitive measure that effectively marginalises people who might be struggling," he said.
The Turnbull government attempted to create a standalone, higher-level English exam for the citizenship test last year, as part of a sweeping reform that included a new test on “Australian values” and longer residency requirements.
The bill was eventually defeated in the Senate, with Labor, the Greens and many on the crossbench opposed to the tougher English requirements, which would have been pitched at around the IELTS 6 standard.
Mr Dutton said the government would reattempt the citizenship reforms this year, suggesting the government may lower the English requirement from IELTS 6, described as “competent”, to IELTS 5, described as “modest”.
But Mr Tudge confirmed the government is actively considering making its own English test focused on conversational skills, rather than relying on international standards.
“We're looking at whether or not we do in fact make an Australian customised test, fit for purpose,” he said.
“What we don't want to see in Australia is what you sometimes see in Europe where you get parallel communities emerge, rather than the tradition of Australia [which] has been where you've got communities of people which blended together.”
Prime minister Malcolm Turnbull said all Australians had a "vested interest" in everyone being able to converse in English.
"It is plainly in everybody's interest that everyone, ideally, should have English language skills," he said.
Mr Tudge said the new test for permanent residents may also be adopted as the citizenship language test. Most people who apply for citizenship are required to have lived in Australia as a permanent resident anyway, but some gain citizenship through descent or adoption.
The minister said the revised citizenship reforms would be passed “this year”. Mr Dutton previously said he wanted the changes in place by July 1, but there are now only two sitting weeks of parliament remaining before that deadline.