By downloading a shared key over TLS rather than the provider's public key?

No difference from the perspective of the token consumer. From the perspective of they token generator, it means rotating per-tenant keys rather than a single keypair.

I addressed this elsewhere (https://news.ycombinator.com/item?id=16072690) but to quickly recap: that's not the hard problem, and hardened SAML IdPs that have the option of exploiting this turn out to have per-tenant keys anyway so that they can get cryptographic binding instead of counting on audience restrictions being checked.

Additionally, your TLS terminating stack is much better hardened than median in-app crypto code.