At least for Nix you _can_ cross compile, but cross compiled and native compiled derivations are not identical - so if you go the cross compiled route you will end up building everything.

Another option is to use Qemu and binfmt_misc - that way you do get the result of "native" compilation, but it's slow.