The U.S. government saw a budget deficit of $680.3 billion for fiscal year 2013, showed Treasury Department data on Wednesday, a 37 percent drop from the fiscal 2012 deficit and marking the first time in five years that the figure has been below $1 trillion.

According to AFP, significant tax increases and spending cuts had allowed the federal government to reduce its budget deficit, though the figure is still the 5th highest shortfall of all time.

Tax receipts rose by 13.3 percent, to $2.77 trillion while government spending fell 2.4 percent to $3.45 trillion.

The budget gap has fallen at its fastest pace since World War II over the last four years, Treasury Secretary Jack Lew said, as cited by Reuters. In 2009, the budget deficit was more than $1.4 trillion, when the government spent heavily to battle recession.

"Congress must (now) build on this progress by crafting a pro-jobs and pro-growth budget agreement that strengthens the economy while maintaining fiscal discipline," said Lew.

As a percent of GDP, the budget deficit was also considerably smaller than it's been in the past five years, at 4.1 percent. By contrast, the annual deficit in 2009 topped 10% of GDP, while last year it was 6.8 percent.

"I want to say this from the get-go: If this conference becomes an argument about taxes, we're not going to get anywhere," said Representative Paul Ryan, the House of Representatives Budget Committee chairman, who is leading the Republicans on the panel.

Democratic leaders of the conference committee have already said they wanted part of any budget savings to come from increased revenue raised by closing some tax breaks for corporations and wealthy Americans.

Economy Watch