Prime Minister Malcolm Turnbull will ask parliament to require the major banks to face a committee inquiry at least once a year.
The decision to make the banks more accountable came after the big four banks failed to pass on the latest official interest rate cut in full.
"The banks should have passed on the full rate cut," Mr Turnbull told reporters in Sydney on Thursday.
The Reserve Bank has cut rates by 3.25 per cent since 2011 however banks have only passed on 2.5 per cent to customers.
Last year the banks actually increased rates despite the Reserve keeping them on hold.
Mr Turnbull said the banks will be required to face the parliament's economics committee to give a full account of the way in which they are managing their affairs, their dealings with customers and their interest rate policy.
"They will be regularly accountable to the Australian people through their parliament in exactly the same way as the Reserve Bank is and APRA is," he said.
Treasurer Scott Morrison said he had spoken to all the banks on the issue.
"They have agreed to participate in the process," he said.
The Bankers' Association believes the RBA's cash rate does not mirror the actual funding costs of banks.
"Banks have explained repeatedly why the Reserve Bank does not set interest rates,” ABA CEO Steven Münchenberg said in a statement.
But banking analyst Martin Smith from East & Partners argues more needs to be done.
"By all means the banks would probably admit themselves that they haven't done enough to explain why the full rate cut is being passed on," Mr Smith said.
“Very much an interesting move by the Government, given that passing on rates are possibly only a part of the challenge. There needs to be much more done to improve transparency and trust.”
The move comes as Opposition Leader Bill Shorten ramped up his demand for a royal commission into the banks.
But Mr Turnbull said apart from being hugely expensive, a royal commission might go for a year or several years and when it comes to an end it writes a report.
"What we're setting in place here is ongoing, permanent cultural change and change that will make the banks ensure that they are accountable," he said.
NAB responds
The National Australia Bank was one of the four banks which failed to pass on the latest official interest rate cut in full.
Their CEO Andrew Thorburn issued a statement on Thursday, explaining the bank's decision, saying: "We have been paying more to both depositors and investors over the past six months and we set a rate for mortgage customers that reflected the balance between all our stakeholders."
"If we don’t get this right, it is inevitable we will lose business, and become a less competitive bank," he said.
"We look forward to the dialogue around how we balance these important themes - the delicate balance between different stakeholders; outlining the full cost of being an unquestionably strong bank; and bringing further insight to the topic of how we set our interest rates."
Chamber calls for ACCC and budget focus
As the banking sector came under scrutiny, Australian Chamber of Commerce and Industry issued a statement urging all senate members to demonstrate willingness to "act in the national interest rather than a partisan interest", in relation to the budget and the workplace relations legislation that triggered July's double dissolution election.
Chamber CEO James Pearson said each senator had "a tremendous weight of responsibility on their shoulders".
“We urge the Parliament to support legislation to restore the Australian Building and Construction Commission and improve regulation of registered organisations," he said.
"These issues were the trigger for the election and voters have demonstrated their support.
“The Parliament also needs to consider the legislation to enact May’s Budget. In particular we look forward to the Youth Jobs PaTH scheme being put into action to help more young people find work, and also to the company tax cut being legislated in full so that businesses of all sizes can invest with confidence about future tax arrangements."
