Depends on the base you use for orders of magnitude. In base 10, 100→5,000 ms is 1.7 orders of magnitude, but in base 2 (which I think is useful because we intuitively understand that twice as long or half as long make a big difference) it is 5.6.

I don't know if natural logarithms make sense in this context, but if so: using base e 100→5,000 ms is 3.9 orders of magnitude.

Regardless, it is IMHO a reasonably big deal, a bigger deal than 'an order of magnitude faster' indicates.