The Supreme Court's historic Citizens United decision almost four years ago drastically loosened campaign finance rules for corporate and union contributions, resulting in record levels of spending during the 2012 elections.

Now the high court is considering the removal of some caps on campaign contributions by wealthy individuals, a position championed by anti-regulation conservatives and opposed by campaign finance reform advocates.

Federal law currently says that during a two-year election cycle, individuals can't give more than $48,600 to all candidates for federal office and no more than $74,600 to national party committees that make contributions to candidates.

But in one of the most high-profile cases the Supreme Court will take on this term, Shaun McCutcheon, a prominent Republican donor from Alabama, says the two-year combined cap of $123,200 is unfair and has petitioned the court to strike it down.

Several of the court's conservative-leaning justices voiced skepticism about the limits during oral arguments. Chief Justice John Roberts, who is considered the key vote in the case, said the total limit "seems to me a very direct restriction" on First Amendment rights that guarantee free speech.

Justice Antonin Scalia said the caps on donations to candidates and political parties encourage donors to give money to less-regulated political action committees, a scenario he says creates "drive-by PACs for each election."

Solicitor General Donald Verrilli, arguing on behalf of the federal government's push to keep the limits, said if they were lifted, individual donors could give more than $3.5 million in campaign contributions during a two-year cycle. Such a scenario, he said, would lead to a "real" and "profound" threat of corruption in federal elections.

But Scalia downplayed the potential threat, saying that in the modern era of escalating campaign costs, "I don't think $3.5 million is a heck of a lot of money."

McCutcheon, joined in his case by the Republican National Committee and Senate Minority Leader Mitch McConnell, R-Ky., also said it was false to assume more money in elections leads to corruption, telling the New York Times that "we need to spend more money on politics, not less," and that "we need to improve" the system.

McCutcheon added he doesn't have a problem with current law that sets a $2,600 limit per candidate in primary and general elections for a total of $5,200 per two-year cycle. But he said it was absurd for the government to cap his total contributions to all federal candidates at $48,600.

"Under the present law, I can give the legal amount — that's $2,600 — to up to 17 candidates, but if I give that same legal amount to an 18th candidate, I’m somehow corrupting the system. Really?" he said in a June speech.

But critics say that doing away with the limits would lead to big-money donors gaining even more influence over candidates and elections than they already have, squelching the political voice of what Justice Ruth Bader Ginsburg called the "little people."

"What [these campaign limits] require the candidate to do is, instead of concentrating fundraising on the super-affluent, the candidate would then have to try to raise money more broadly in the electorate," Ginsburg said. "By having these limits, you are promoting democratic participation."

Jamie Raskin, a constitutional law professor at American University, expects the court to strike down the individual limits on a 5-4 vote — the same tally that removed spending limits in the Citizens United case.

He added that the court is trending toward eliminating campaign finance restrictions altogether.

"It's pretty clear that the liberals are losing in the Supreme Court on the campaign finance question," he said. "The Roberts court has instituted a paradigm shift in campaign finance law. The presumption now is any barriers to the flow of money are unconstitutional, so this barrier too is likely to fall."