I would guess that's right - AVX only came onto the scene in 2011 so most code will be able to fall back on an SSE implementation if AVX is unavailable, and presumably an AVX emulation which is a bad fit versus the hardware is actually slower than the SSE emulation.

A quick search suggests a couple of people already running into this issue with the older generation Mac Pros that also don't support AVX but it doesn't seem that widespread.