His area of expertise is mathematical teaching/learning. He was undoubtedly talking about the average person, I doubt it was supposed to be an absolute neurological-level statement. And as a general statement/stereotype, there are expected to be exceptions. I doubt he got to be a Stanford math professor without seeing some gifted math students himself. His statement can still stand, despite yourself as a counterexample.
A little more about him: His CourseEra course is "Introduction to Mathematical Thinking". It isn't about math, it's about how to think mathematically. He commonly talks about the pitfalls people make with basic mathematical approaches. He works with helping them understand approaches to math and how to deal with thinking abstractly and purely logically. Some people pick up all that stuff implicitly with little effort, some people never really master it. Given his position, I think he sees a pretty raw view of the average person's approach toward math.
A little more about him: His CourseEra course is "Introduction to Mathematical Thinking". It isn't about math, it's about how to think mathematically. He commonly talks about the pitfalls people make with basic mathematical approaches. He works with helping them understand approaches to math and how to deal with thinking abstractly and purely logically. Some people pick up all that stuff implicitly with little effort, some people never really master it. Given his position, I think he sees a pretty raw view of the average person's approach toward math.