The White House launched into damage control mode Thursday to deal with the growing chorus of Republicans and at least one Democrat accusing Vice President Biden of race-baiting after Biden told a mostly black crowd in Virginia that Republicans want to "put y'all back in chains."

Biden's comments were "without question" a racial appeal, said former Virginia Gov. Doug Wilder, a Democrat and the nation's first elected African-American governor.

"Slavery is nothing to joke about," Wilder told Fox News. "What [Biden] said is [Republicans] are going to do something to y'all -- not to me, not to us. So he was still involved with that separate American."

Republican presidential candidate Mitt Romney said Biden's comment was "shameful," and Romney's running mate, Rep. Paul Ryan, R-Wis., called it a sign of desperation.

"You know, these are the kinds of things you say when you're desperate in a campaign," Ryan said on Fox News Radio. "We've gone from hope and change to anger and division and blame and attack, and I think people are going to see right through this."

Biden made the comment Tuesday in Danville, Va., while talking about Republican plans to roll back banking reforms and "let the big banks once again write their own rules, unchain Wall Street."

"They're going to put y'all back in chains," he said.

President Obama jumped to Biden's defense, calling the cries of race-baiting "chatter" and "noise."

"The country isn't as divided with gaffes or some stray remark as Washington is," Obama told People magazine. "Most folks know that's just sort of a WWF wrestling part of politics. It doesn't mean anything, just fills up a lot of airtime."

This is the second time this summer that Obama has had to deal with rhetorical bombshells from Biden, whose tendency to veer off script while speaking publicly has often caused him problems. Earlier this summer, Biden got ahead of Obama in announcing the administration's support for same-sex marriage, forcing Obama to declare his support the next day.

Sen. John McCain, R-Ariz., who lost to Obama and Biden in 2008, said Obama should dump Biden as his running mate.

"There's very little doubt that whether Joe intended it or not, that comment has racial inferences associated with it," McCain told the Arizona radio station KFYI. "Now that's disgraceful."

Obama's campaign advisers rejected the suggestion, saying Biden will remain on the ticket.

"He wasn't making a racial comment," Obama campaign adviser David Axelrod said on MSNBC. "He was making a comment about the impact about rolling back all of Wall Street reforms on consumers, essentially rolling back what we've done and putting the burden back on consumers."

White House spokesman Jay Carney, who spent nearly an hour fielding reporters' questions about Biden, said Republicans are using the vice president's remarks to distract voters from substantive policy debates. He chided McCain, who picked Alaska Gov. Sarah Palin as his 2008 running mate, saying, "One place I would not go for advice on vice presidential running mates is to Sen. McCain."

"There will always be distractions," Carney said. "Nobody took it as a reference to anything except for those who are trying to make something out of nothing here and distract attention from the policy debates."