Yes. That's how the current GDPR law works. And it seems reasonable to me.
Accepting payments in USD is a pretty clear signal that a website mostly cares about US users, and if someone else uses the website, oh well, it happened, doesn't really change anything under GDPR.
(although you probably could have French support anyway, just expand the website to Canada as well ;), of course it isn't the same French as in France, but it's still French somewhat, EU currencies are trickier however)
I would argue that this is a bad definition because what you described is just operating in a jurisdiction but without localization.
I prefer the definition where you're making money(directly or indirectly) by providing services in a jurisdiction.
An example would be collecting French users data and then selling it to ad agencies(or being acquired by Google?).