The former director of the Federal Emergency Management Agency criticized President Obama Friday for not walking "off the golf course" during his two-week vacation in Martha's Vineyard to show support for the 40,000 Louisiana families who lost their homes in the state's devastating floods.

Michael Brown, who infamously oversaw FEMA during Hurricane Katrina under President George W. Bush, told Fox News' "Special Report" he supported Obama's decision not to immediately visit Baton Rouge earlier this week because emergency responders were still rescuing people stranded on rooftops throughout the city.

But by Friday, the president should have done something to show that his administration stands in solidarity with the people of Louisiana.

"He could have gone to Lilly Armstrong Airport in New Orleans and made a statement there, or at the very least, what he should have done in my opinion, is he should have walked off the golf course, gone to some office somewhere on Martha's Vineyard and talked to the people of Louisiana and assure them that he was sending [FEMA chief] Craig Fugate, [Homeland Security] Secretary [Jeh] Johnson," Brown said.

"The president needs to make sure that the entire Cabinet, that all of the departments and agencies of the federal government, know that if the FEMA director asks for anything, by damn, give it to him because that person is there on behalf of the president," Brown added.

The Obama administration has not commented on the successful rescues of 20,000 people and only 13 deaths.

"So on the one hand, yes, the president kind of botched it by not at least making a public appearance, but instead only photo ops on the golf course and also now is the time because you're now in a recovery phase, and it's much easier," Brown added.

The president also has not visited southern California, where wildfires the size of San Bernardino, Calif., have forced 82,000 residents to evacuate.

Brown said a statement from the commander in chief that the government understands people's problems would be a good place to start.