Top Internal Revenue Service officials on Thursday struggled to explain to congressional lawmakers why the agency spent $64,000 on plastic squirting fish and other "swag bag" trinkets during an over-the-top employee conference that cost taxpayers $4.1 million.

IRS managers apologized Thursday for the lavish spending that included tens of thousands of dollars for a "Star Trek" parody training video and an artist who painted portraits of Michael Jordan and Albert Einstein. But they defended the extravagant conferences, including the 2010 event in Anaheim, Calif., saying they didn't violate IRS rules at the time.

The conferences, IRS officials said, were needed to train a slew of new employees, most of whom were hired to handle the implementation of President Obama's health care reforms.

"I do not think that the conference itself, the premise that it was based upon was wrong because of the needs of our employees at that time," Faris Fink, the small-business commissioner who played Mr. Spock in the "Star Trek" spoof, told the House Committee on Oversight and Government Reform.

A new audit shows those IRS conferences cost taxpayers $50 million between 2010 and 2012, including $50,000 for the "Star Trek" video and another about line-dancing. Fink apologized for the spending Thursday.

"I think now that in retrospect, in looking back, that you take a look at the expenses, and we should have been more diligent in our responsibility to the American taxpayer," he said.

Fink told the panel he did not know about the massive price tag for the Anaheim event "until much later," even though he personally authorized the conference in April 2010.

Fink played Spock in the now-infamous IRS "Star Trek" training video, which, along with a line-dancing video that was also shown at the conference, cost more than $50,000 to produce.

"What were you thinking?" Committee Chairman Darrell Issa, R-Calif., asked Fink.

"Those videos were, at that time they were made, an attempt to, in a well-intentioned way, use humor ... to open the conference," Fink said. "The fact of the matter is, it's embarrassing, and I apologize."

News of the IRS' lavish spending comes at the a time when the tax agency is also being investigated for targeting conservative groups seeking federal tax exemptions. So far, an acting director was forced to resign and a department head was placed on paid leave because of the investigations.

Two additional "management level" employees were put on administrative leave when a series of emails revealed that they arranged free food and alcohol for a private party in violation of agency rules and then tried to keep it a secret. "It is to be kept confidential," one unnamed IRS employee warned in an email.

Agency officials, however, were unable to explain the 2,609 plastic squirting fish that were given to employees at the conference.

Fink, who along with two other managers received a $2,000 bonus for planning the conference, told Rep. John Mica, R-Fla., he has not seen the fish.

"Maybe I could offer a reward," Mica said. "I'd love to see one of the squirting fish. I'm sure that some of the taxpayers, the people that went to work on that early bus this morning out of Baltimore or my district in central Florida, love to know that the federal government has spent $64,000 on squirting fish for federal employees at a conference."