A federal judge who said Supreme Court Justice Samuel Alito was damaging faith in the Supreme Court has been forced to apologize for publicly criticizing the justice.

Senior Judge Michael Ponsor criticized Alito in an essay for the New York Times, blasting the Supreme Court justice for flying an upside-down American flag outside his home in Virginia and a Revolutionary War-era “Appeal to Heaven” flag outside his beach home. Democrats complained that the flags were an indication of Alito’s bias toward Republicans and demanded he recuse himself from cases involving President-elect Donald Trump that were before the court.