House Minority Leader Nancy Pelosi, D-Calif., said Tuesday she does not plan to impeach Judge Brett Kavanaugh if he is confirmed to the Supreme Court or if he remains at the Court of Appeals if House Democrats win back the House in the midterm elections.

"Let's take it one day at a time. We're not about impeachment," Pelosi said during an interview at the Atlantic Festival when asked about the idea of impeaching Kavanaugh. "We're not about dividing the country ... [W]e have to find unity."

"You'll see us coming forward with rules of the House that are about respect for other opinions, openness, accountability, fairness and unity," Pelosi said. "Hopefully, this will work it's way through the process with the truth being made known and the truth will set us free."

"People care about what happens to them in their lives," Pelosi said, listing off key Democratic priorities for the next Congress if her side wins. "All those things are more important to people than who's on the Supreme Court. Although it has ramifications in their lives, they don't see it as immediately as some of these other issues. We're about unifying, not dividing. Let the process work its will."

"So that would not be my plan," she added. "I'm not about impeachment. I have enough people on my back to impeach the president, now we're going to talk about somebody [else]? It's about unifying."

[Click here for complete Kavanaugh coverage]

Some Democrats have said Kavanaugh should be impeached if the Senate confirms him to the Supreme Court, based on allegations that he sexually assaulted at least three women when he was younger. The FBI is looking into these claims this week, but Republicans have said they expect to vote on him by the end of the week.

Ousting a Supreme Court justice through impeachment is very similar to the impeachment of a president, and requires just a simple majority of the House of Representatives. A convictino, however, requires a two-thirds majority vote by the Senate.