Defence Minister Stephen Smith has failed to measure up with the top defence officials and must relinquish his current duties, this according to former Australian Defence Force Academy (ADFA) commandant Jim Molan.

The retired military general assailed the manner that Mr Smith had presided over the sex scandal that rocked ADFA in 2011 when a female cadet, identified by The Australian as Kate, hurled allegations that a fellow male cadet filmed their intimate moments.

The sex act, Kate claimed, was broadcasted via Skype, allowing other male candidates to watch the spectacle.

Ensuing investigation led to the provisional relief of ADFA commandant Commodore Bruce Kafer, who at the same time had to endure scathing criticisms from Mr Smith, stressing then that the academy chief proved ineffective in handling the sex case.

mr Smith lambasted Mr Kafer for allowing disciplinary proceedings to be pursued against Kate instead of attending all the assistance for the female cadet to obtain justice.

However, results of the formal inquiry on the case was released this week, clearing Mr Kafer of any wrongdoing and allowing him to retake his old job, which he will assume by Thursday at the latest, The Australian wrote.

Almost immediately, cries for Mr Smith to apologise to Mr Kafer emerged, with many expressing belief that the defence chief was too heavy-handed on the military officer.

But Mr Smith refused to budge and maintained that standing down the ADFA chief last year was the right thing to do.

In an interview with ABC on Thursday, the defence chief said "I don't resile in any way from the things that I did and said at the time relating to the 18-year-old innocent victim of an alleged serious sexual abuse."

Mr Smith added that his stint so far in the defence ministry has led to the development of smooth rapport between him and other high officials of the department.

"I have a very strong and close relationship with the chief of the defence force, the vice chief of the defence force, the secretary, and the service chiefs," Mr Smith told Nine Network on the same day.

Two of those men were Chief of the Defence Force General David Hurley and Defence Department secretary Duncan Lewis and according to Mr Molan, they definitely towered over Mr Smith in terms of high standard and achievement.

"We need a defence minister that can run the department that is the equivalent of the two men of great integrity and achievement that were standing beside him," Mr Molan told ABC, adding that if Mr Smith could not at least equal the two officers then he should quit.

"I reckon that the Australian public deserve better than what we saw yesterday," the retired general said.

Also, the Defence Force Association (DFA) has suggested that Mr Smith could be using the defence portfolio to shore up his political ambitions, hinting that every action coming from Mr Smith looks forward his advancement in Labor leadership.

"It's no secret that Stephen Smith hopes to lead the Labor Party at some stage," DFA spokesman Neil James told ABC.

And doing so would require winning good points from the Australian public, specifically from female voters, Mr James added.

But Treasurer Wayne Swan came to the defence of Mr Smith, stressing that by simply reviewing the Kirkham submission would lead one to conclude that the defence minister made the right decisions last year.

"If you have a look at all of the documents that were produced yesterday ... I think you'll absolutely understand the position that Stephen Smith has taken," Mr Swan was reported by the Australian Associated Press (AAP) as saying.

The Coalition, however, believed otherwise and insisted Mr Smith "took aim at people without knowing the facts and used language which was intemperate."

"What is the point of the inquiry if the (Smith) refuses to take the word of the inquiry officer," opposition defence spokesman David Johnston told ABC.