Libertarian presidential nominee Gary Johnson teased GOP nominee Donald Trump Tuesday evening for his calls to build a wall along the U.S.-Mexico border.

"He was watching the Olympics really closely to see how high the Mexican pole vaulters could go," Johnson told Fox News' "The Five" panel Tuesday evening, echoing a statement he made two weeks ago at the start of the Olympics.

The former New Mexico governor wouldn't name the biggest difference between his and Trump's immigration views and instead said his rival had a "complete misunderstanding of the issue." Johnson said he would "absolutely not" support Trump's "crazy" plan to build a wall along the border.

He defended his anti-wall stance and said every U.S. citizen would illegally cross a national border if it meant providing a better life for his or her family.

"The reason that there are 11 million undocumented workers is the jobs exist, but they can't get across the border legally. So they cross illegally, just as any one of us would do if we needed a job to support our family," Johnson said.

He added he would "keep the control the way it is at the border" while making it "as easy as possible for somebody that wants to come into this country," but did not share details on immigration policies he would support.