President Obama wants to focus on the community, the courtroom and the cellblock as he tries to overhaul the criminal justice system, he said Tuesday during his keynote address at the NAACP's annual conference in Philadelphia.

He said America needs to invest more in education, particularly early education, and in at-risk youth job programs. He also said police departments across the country need to listen to his task force and institute better officer training and data collection to make policing "more effective…more accountable…[and] more unbiased."

Mandatory sentences for nonviolent drug offenders need to be severely reduced or eliminated, Obama said. Giving judges more discretion could help "steer a young person who has made a mistake in a better direction."

He called on Congress to pass sentencing reform and include alternatives to prison, such as drug courts and probation.

To shine light on some inhumane prison conditions, Obama said he is traveling to a federal prison in Oklahoma on Thursday.

Inmates "are also Americans, and we have to make sure that as they do their time and pay back their debt to society that we are increasing the possibility that they can turn their lives around," he said.

"And that's why we should not tolerate conditions in prison that have no place in any civilized country," he added. "We should not be tolerating overcrowding in prison; we should not be tolerating gang activity in prison; we should not be tolerating rape in prison."

"And we shouldn't be making jokes about it in our popular culture," he said of prison rape. "That's no joke."

Obama said that he's asked Attorney General Loretta Lynch to review the penal system's solitary confinement policy as well.

"Do we really think it makes sense to lock so many people alone in tiny cells for 23 hours a day, sometimes for months or even years at a time?" he asked. "That is not going to make us safer … our prisons should be a place where we can train people for skills that can help them find a job, not train them to become more hardened criminals."

Obama said society needs to help rehabilitated ex-cons to find jobs and should make it easier for them by forbidding employers from asking about prior convictions.

Lastly, Obama called for reinstating the vote to released felons.

If "folks have served their time, and they've reentered society, they should be able to vote," he said.