JACKSONVILLE – He understands the fans' frustration.

David Caldwell made that clear Tuesday afternoon while wrapping up the 2015 season. He made clear, too, that 5-11 isn't good enough – and that it won't be good enough next season.

The Jaguars' general manager made a couple of other things clear, too:

One is he likes where the franchise stands moving forward.

Another is he really, really supports Jaguars Head Coach Gus Bradley.

Neither was as a mammoth surprise. But along with a sneak preview/hint regarding the afternoon's major news – that Bob Babich will not return as defensive coordinator next season – those were two of the primary takeaways from Caldwell's season-ending media availability at EverBank Field early Tuesday afternoon.

Caldwell's talk Tuesday wasn't billed as a State of the Franchise, but he did make his view on the building process that began with his hiring in January 2013 clear.

"I think it's probably right where we thought it would be," he said.

Caldwell hit many major points Tuesday, and as he typically does, he spoke with candor. He thanked Jaguars Owner Shad Khan early on for patience in three seasons that have featured a total of 12 victories.

"It's great to have an owner like Shad," he said. "He's been great through this process."

He also hinted at what became the major Jaguars news of the day, saying he expected defensive staff changes. A few hours later, the Jaguars announced that Babich will not return for a fourth season in the coordinator position.

As of late Tuesday afternoon, there was no official word on the direction of the defense. There has been speculation about defensive line coach Todd Wash – a respected member of the staff – assuming the position, and Bradley and Caldwell both discussed in recent days the possibility of Bradley calling plays.

Bradley brought the team's 4-3/3-4 hybrid defense from Seattle, where he was the defensive coordinator – and play-caller – for the system from 2010-2012.

Caldwell offered no hint at the direction, but said he wouldn't have an issue with Bradley calling plays.

"I trust Gus," Caldwell said. "I would endorse what he feels to do is right. It was difficult the first couple of years because there was a lot to be done here. His focus couldn't solely be on the defense. We had to create a culture. We had to create a standard of competition. A lot of Gus's days were full trying to build that.

"We feel like we have the offense pretty much squared away. We feel like we have the culture pretty much squared away. If he feels that (calling the plays) is the best thing to do, I would trust him if he thinks he can do it. I would trust him to make that decision."

If Caldwell made anything clear Tuesday, it was that that trust in Bradley extends well beyond defensive play-calling. He said Khan's announcement last week that Bradley would return in 2016 came as no surprise to anyone around EverBank Field. He also lauded Bradley as a head coach, citing an ability to make difficult decisions, good game management, quality leadership skills, a good defensive and offensive mind "when it comes to Xs and Os" and leadership skills.

"He's terrific in front of the team," Caldwell said. "The players love playing for him and they play their tails off for him."

Caldwell also addressed the team's 12-36 record in the past three seasons, pointedly saying he did not  put the victory total on Bradley because the plan focused on building through the draft and largely eschewing free agency until the 2015 offseason.

"I don't hold Gus responsible for the first two years," Caldwell said, and added of the 5-11 record this past season, "If you would have asked me three years ago, 'Do you think you'll have a winning record by Year Three?...' I wouldn't have been sold. The fact that we were even talking about the playoffs (late this past season" tells me we're probably closer than we think we are."

Caldwell added that winning four games in 2013 was "a little bit of a minor miracle."

"From a talent standpoint, I take the blame for those first two years," Caldwell said. "Because we weren't ready to compete and we knew we weren't going to be ready to compete. The plan was we were going to wait until Year Two/Year Three to invest in free agency and give our young guys we drafted a chance to develop.

"Not that it couldn't have happened sooner, but it was going to be very difficult."

Caldwell, as Khan has done throughout the process, put no specific number on how many victories are needed next season. It's clear the number needs to be more than five.

But whatever the number, make no mistake:

Caldwell believes the process is working, and believes the needed improvement can happen this offseason. He believes the defense can improve, and that the offense – especially franchise quarterback Blake Bortles – will continue to improve.

And he darned sure believes Bradley – however involved he may be with the defense – is the man to oversee all that improvement.

Caldwell has made that clear for three years, and it was absolutely clear again Tuesday.