A lawyer for Wisconsin Club for Growth director Eric O'Keefe, who has been subject of a secretive probe against conservative groups in the state, on Wednesday sent a letter threatening to sue state prosecutors for violating the First Amendment unless they drop their investigation.

At the center of the issue is what’s known as a John Doe investigation, which means that details of the probe remain secret and the targets of the investigation cannot speak about it.

Last November, the Wall Street Journal reported that Wisconsin prosecutors (led by special prosecutor Francis Schmitz) had been using the John Doe powers to launch a wide-ranging probe that issued sweeping subpoenas to 29 conservative groups that were instrumental in helping Gov. Scott Walker and Republican legislators survive the 2012 recall elections.

Though he was barred by prosecutors from discussing the case, O’Keefe nonetheless confirmed to the Wall Street Journal that he had been subpoenaed in October and that “at least three of the targets had their homes raided at dawn, with law-enforcement officers turning over belongings to seize computers and files.”

The probe has had a chilling effect on speech, effectively keeping many conservative groups in Wisconsin on the sidelines ahead of the 2014 gubernatorial election.

Last Friday, the Journal reported that in a secret ruling Judge Gregory A. Peterson had quashed subpoenas to conservative groups and ordered prosecutors to return seized property.

David Rivkin, an attorney who is representing O'Keefe, on Wednesday announced that he had sent a letter to prosecutors warning them they would face a federal civil rights lawsuit if they didn't drop the probe. He argued that the probe doesn't have a basis in Wisconsin law and violates O'Keefe's rights of freedom of speech and association.

The letter demands that prosecutors “close the investigation immediately, publicly exonerate Mr. O’Keefe and the Club, and dissolve the gag order that bars discussion of the proceeding.”

In a press release, O’Keefe was quoted as having said, “This investigation is political payback by elected prosecutors against conservative activists for their political successes in Wisconsin They are violating the constitutional rights of private citizens and must be held accountable.”

O’Keefe could not elaborate further, citing the secretive nature of the investigation.