On October 9, Justice Brett Kavanaugh participated in his first oral argument as a member of the United States Supreme Court. One might think this moment confirmed the futility of continued fighting over his nomination. After all, the Constitution provides that federal judges, once appointed, serve “during good behavior”—meaning Kavanaugh now has the job for life.

But if you thought it was over, you would be wrong. One constitutional option remains for Kavanaugh opponents—impeachment. Some Democrats claim they will begin such proceedings if their party gains control of the House. If a majority of that chamber votes to impeach and two-thirds of the Senate concur, Kavanaugh would be removed from the nation’s highest bench.

Kavanaugh need not worry about the latter possibility. Even if the House impeaches, nowhere near two-thirds of the Senate would vote to remove absent some new, horrific, and much more substantiated accusation than those already made.

[Dershowitz: Constitution won't let Democrats impeach Kavanaugh]

Beyond the contemporary political landscape, history and the Constitution’s structure push against doing so as well. Historically, the Senate has never removed a Supreme Court justice from office. Only once has the House even voted to impeach. That precedent occurred way back in 1804-1805. At that time, a Jeffersonian-dominated House approved impeachment articles against Federalist Justice Samuel Chase. These articles accused Chase of partisanship against Jeffersonians in how he conducted trials and in a jury charge criticizing Jeffersonian policies. The articles themselves were the work of hyper-partisanship, too.

None of the Senate’s votes on these articles came close to removal. By declining to expel Chase, Congress set a high internal standard for removing a sitting federal judge. One could not remove a justice for holding contrary opinions. Congress could not remove even for questionable comments in a judge’s official capacity. Only corroborated illegal activity, it would seem, would do the trick.

So it should be. For beyond historical practice, the Constitution’s structure cautions against removing Supreme Court justices in any but the most extreme circumstances. The reason for this caution came up in Kavanaugh’s first confirmation hearing. On that hearing’s second day, Sen. Chuck Grassley, R-Iowa, asked Kavanaugh “what makes a judge a good one?” Kavanaugh responded that “the first quality of a good judge in our constitutional system is independence.”

Our Constitution demands that every branch of the federal government possess this trait. In Federalist 51, Madison wrote that to maintain separation of powers, “it is evident that each department should have a will of its own.” To act independently, a branch first must possess the desire to do so. That desire must then include adequate power to allow the will to act.

But the judiciary’s independence looks different from that found in the other branches. While independent of each other, Congress and the presidency hold dual dependencies to other entities. Both must act within the Constitution’s confines. But each tends to focus more on another obligation: adhering to the will of their constituents.

With life appointment, federal judges do not feel the pressure of elections or voters. They are independent of the people’s current preferences in addition to other branches. Many decry this point. They wish judges to be elected, or at least to serve limited terms. But the Founders disagreed. In Federalist 78, Alexander Hamilton declared that “[t]he standard of good behavior for the continuance in office of the judicial magistracy, is certainly one of the most valuable of the modern improvements in the practice of government.”

Hamilton tied this praise to an explanation of the particular purpose the judiciary serves. An independent judiciary, he continued, presented an essential protection against threats to individual liberty. Voters could use the ballot box to stop the tyranny of a minority, removing congressmen or presidents who threatened their rights. But what about the rights of the minority when the majority sought, in temporary fits of “ill humors,” to invade minority liberties? One could point to the law, except that political branches dependent on the majority might be tempted to ignore the law’s protections to win re-election.

One answer to this problem was the judicial power. Judges were not directly dependent on the people. Instead, they were singularly beholden to the law, in particular the Constitution. In this singular focus, Hamilton concluded that an independent judiciary comprised “the best expedient which can be devised in any government, to secure a steady, upright, and impartial administration of the laws.” Regardless of popular clamors, they would uphold the Constitution and statutes passed under it. And such steady, upright, and impartial administration of law protected all persons against momentary fits of majority tyranny.

To allow for easy, frequent removal of federal judges risks undermining this constitutional safeguard. It hazards placing judges’ will under the control of Congress. Without a will of their own, the judicial power would combine with the legislative. Quoting the French philosopher Montesquieu, Federalist 47 warned “[w]ere the power of judging joined with the legislative, the life and liberty of the subject would be exposed to arbitrary control.” The legislature, rather than being forced to submit to its own laws, could manipulate statutes’ interpretation after the fact to suite pernicious purposes. Such a circumstance could prove fatal to individual liberty.

The preceding points should give pause to those committed to impeaching Justice Kavanaugh. The votes aren’t there. History isn’t, either. But, most importantly, impeaching Kavanaugh would do great damage to our constitutional structure and the liberty it preserves.

Adam Carrington is assistant professor of politics at Hillsdale College.