Yeah, I get that.. it's not unreasonable. However, why does Google Home force you to turn on location tracking in general, that's what I don't get.
It sounds like apps scanning for SSIDs could easily be handled by a specific permission for that. No need to enable location tracking (turning 'Location' "on" in Settings) for this. It doesn't make any sense (except for what it looks like: Google wants to get your location, by any means).