Well, besides the fact that we don't actually live in a "Retina world", the use of JPEG, even on high quality settings, for an artificial (= not taken from a camera) image pretty much precludes the ability to produce derivative works from it without a much more serious loss of quality. That's not to mention that there are some things which high resolutions cannot make up for, like color bleeding and other pretty serious fuzzing that JPEG can introduce at low quality settings.

The point is that at high DPI, the "serious fuzzing" is much less noticeable for a variety of images. A retina screen has 4x the amount of pixels. Which means the 2x2 undersampled color errors are probably below our perception threshold. Especially if you pre-blur your high res image to a small degree, acting like an anti-aliasing filter when you scale it down by a factor of e.g. 1.5. For mobile, where users will be zooming a lot anyway, this is a very useful strategy to consider, which can absolutely beat PNG in image quality. Doesn't mean you have to do it everywhere, but I'd prefer it to having to download megabytes of images on retina devices.

It is very rare that a website has only perfect vector line art in its images. PNG sucks at compressing non-trivial gradients for example.

You've set up a false dichotomy, I think. Why use high-res low-quality images instead of just sticking with low-res high-quality images? It's all going to look fuzzy when you zoom in.

Also, palette images should get some love too. A high-res dithered GIF or 8-bit PNG might meet your needs (looks good on retina displays at normal zoom, stays crisp when zoomed in, smallish file size) better than the other options.