President Obama’s attorneys dropped their appeal of a court ruling that the government may not require a Bible publisher to provide free contraception to employees until the company’s legal challenge to the mandate is finished.

“Bible publishers should be free to do business according to the book that they publish,” Alliance Defense Freedom’s Matt Bowman, an attorney for Tyndale, said in a statement. “The government dismissed its appeal because it knows how ridiculous it sounds arguing that a Bible publisher isn’t religious enough to qualify as a religious employer. For the government to say that a Bible publisher isn’t religious is outrageous, and now the Obama administration has had to retreat in court.”

The Justice Department had been appealing a lower court’s ruling in favor of Tyndale, which has a religious objection to certain kinds of contraception. This is the first preliminary injunction that the Obama team has decided not to oppose.

“The contraceptive coverage mandate affirmatively compels the plaintiffs to violate their religious beliefs in order to comply with the law and avoid the sanctions that would be imposed for their noncompliance,” U.S. District Judge Reggie Walton wrote in his decision to grant the injunction last year.