The joke is on Labor now as reports emerged that cabinet ministers only rushed to condemn the offensive remarks targeting Opposition Leader Tony Abbott and his chief aide Peta Credlin that day after the incident.

According to The Daily Telegraph, key Labor figures attended the dinner hosted by the Construction Forestry Mining and Energy Union (CFMEU) at the Parliament House in Canberra and the list included Labor's number one and two: Prime Minister Julia Gillard and Treasurer Wayne Swan.

The prime minister, however, had already left before the night's comic guest was able to deliver his piece, which media reports said brought the entire audience in awkward silence.

But soon after, Mr Swan, reports said, delivered his scheduled speech, seemingly unmindful that nothing untoward had happened.

The incident has been compared by observers to the earlier firestorm created by Sydney shock jock Alan Jones when he let out 'hurtful comments' involving Ms Gillard's late father, Jack Gillard.

While generally silent on the matter, Ms Gillard referred to the episode in a fiery speech this week, admitting that she was offended as no key Liberal members immediately rose in protest of the questionable remarks by Mr Jones.

The same speech also saw the prime minister labelling Mr Abbott as sexist and misogynist, with vows that she would take up the cudgels for Australian women, she claims, that were offended by leader of the opposition.

Labor, however, by failing to immediately reject the vulgar joke, has lost the moral authority to wage the fight that Ms Gillard opted to pick this week, with her wrath trained on Mr Abbott, political observers said.

For Con Sciacca, who served under the Paul Keating Government, the Labor tactic had backfired and any momentum gained by Ms Gillard during her tirades against Mr Abbot at the Parliament was largely diminished right after the CFMEU function.

Attack on personal affairs of a politician normally generates public sympathy for the target and "on this occasion . . . this could well be what is happening with the Opposition Leader," Mr Sciacca told The Australian on Thursday.

"(Labor) just overstepped the mark . . . they push it too hard . . . If I was adviser to the Labor Party campaign I would tell them to drop off," he added.

Mr Swan has conceded that he reacted a bit late, telling the Australian Financial Review that "he should have raised concerns on the night rather than waiting," until the morning after the joke was unleashed.

Other Labor members also admitted that they were a bit slow in reacting correctly to the situation but Trade Minister Craig Emerson remained defensive while allowing that the joke aired during the union group function was both offensive and sexist.

"No one knew this was coming. In relation to the dinner that Mr Jones held where he said the PM's father had died of shame, there was an attempt by the Liberal Party to cover up all of that," Dr Emerson explained.

RELATED STORIES