Coming into Saturday night's clash with the Central Coast, the defending premiers are without a win after three starts.
And their sluggish defence is under scrutiny after already conceding six goals - but Cole has told the doubters to back off.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"It is harsh for people to be making those sort of judgements of a team that has shown that we can defend very well," he said.
"Defence was the key to (winning the league) last year and it has to be the key again.
"The whole team has to be a solid unit in defence and clean sheets are a big part of the team's aspirations for the rest of the season."
Thirty three-year-old defender Hayden Foxe has taken most of the heat, with the former Socceroo coming under fire for a lack of pace against fleet-footed strikers.
Foxe is expected to be benched for Saturday night's game at Sydney Football Stadium, with Cole making his first run-on start of the season.
The 26-year-old admitted to being frustrated at coming off the bench in the early rounds, but refused to single out Foxe as the Sky Blues' weak link.
"It's not down to pointing fingers or anything like that, I don't think so much the individuals involved (have been the problem), but maybe our overall intensity has been outdone by other teams," Cole explained.
"Of course (I'm happy to start), it's not fun to sit and watch the team ever.
"Personally for me it's not good news when you hear the starting lineup and you're not involved.
"I always want to be involved and the last couple of weeks when I haven't been, it's just been extra motivation to work hard."
Sydney will face a tough challenge in the form of a plucky Mariners outfit who have started the season promisingly.
Despite an injury blow for the Mariners, who'll have to replace goalkeeper Jess Vanstrattan with 18-year-old Mathew Ryan after the experienced shot-stopper injured himself at training on Friday, Sydney are set for a tough encounter.
"They're a very physical team but that is something we're prepared for," Cole added.
"They've showed that they're a good team and they can beat anyone on their day, so it's a good opportunity to turn around what we've left for ourselves after the first few weeks.
"It's a big chance to show exactly how strong we're going to be this year."
