Since Tom Daschle beat a hasty retreat from his Cabinet nomination this week, it seems the noise over the Obama team’s tax woes has — at least temporarily — subsided. But lost in the Daschle-Tim Geithner-Nancy Killefer tax kerfuffle lies a greater issue still: tax avoidance isn’t exactly unusual.

According to the IRS, there’s an annual $300-billion difference between what taxpayers owe and what they actually pay. About two-thirds of that is accounted for by individual underreporting, while similar misreporting by corporations and the self-employed drain the U.S. Treasury of another $88 billion.

That’s a lot of change Uncle Sam hasn’t bothered to collect. Why? One reason is a slump in IRS staffing. Over the past decade, the number of agents that perform audits has dropped by over a third. Meanwhile in 2007, in what the Syracuse University-based Transactional Records Access Clearinghouse calls a “historic collapse,” only 26 percent of corporations holding at least $250 million in assets had their books inspected — compared to more than 70 percent in 1990. (The fact that Congress outsourced debt collection to private agencies in 2004, costing the government $37 million more than such agencies manage to collect, hasn’t helped either.)

As Congress continues to batten down the fiscal hatches, here’s something else to remember: For every dollar the government devotes to IRS enforcement, another five are recouped — a five-to-one return on investment. As Sen. Tom Carper, a Delaware Democrat, puts it, that’s some $300 billion in low-hanging fruit.