Poor oversight by the Internal Revenue Service allowed nearly 700 contract employees who owed $5.4 million in federal taxes to go undetected as of June 2012, according to a report released Wednesday by the U.S. Treasury Inspector General for Tax Administration.

The IG reviewed 13,591 contractor employees and found that five percent had unpaid federal tax debts. Of these, about half weren't on a payment plan to eliminate their debt.

Contractors were able to get away with nonpayment because the IRS reviews contractor tax compliance only every five years, or if the contractor has a break in service longer than two years, according to the IG.

In contrast, the IRS checks tax compliance for its employees three times a year, and only IRS employees can be terminated for nonpayment of federal income taxes.

“Because many contractor employees have access to sensitive IRS systems and facilities, the IRS should address tax noncompliance in a similar manner as it would for its own employees,” said Inspector General J. Russell George.

At least 319 of the contractors, owing almost $2 million of the unpaid taxes, were granted staff-like access to IRS information. Such access gives the contractor unescorted use of IRS facilities and sensitive but unclassified data, according to the IG.

Before receiving that privilege, the contractor is supposed to be tax compliant by filing all federal tax returns and having paid all taxes due, or being on a payment plan.

Five contractors didn't file returns or filed late, and were inappropriately given staff-like access. Thirteen others kept staff-like access after failing to file their returns.

A 1998 law requires the termination of IRS employees who willfully fail to file a federal tax return. Failure to pay full federal tax liability may result in termination of an IRS employee, but is not mandatory.

The IRS agreed to monitor contractor tax compliance and evaluate the identified noncompliant contractors, which could result in the termination of their contracts.

Read the full report here.