The gist of it is that physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits. Random data does not have this property, and cannot be learned easily. Super fascinating.
Thank you!!! as a Physics PhD that was one of the first videos I found on deep learning, and having no idea what a big deal his insights were I promptly forgot who the speaker was (remembering only it was a name I knew intimately from my time in Physics). Have frequently gone back to try to find this unsuccessfully.
Indeed, the connections seem profound. It seems to be a general-purpose optimal algorithm for, well, optimisation. And that would explain why the universe, our brains and AIs all trend toward it.
It could also be just that intelligence tends to mirror the outside world, but that seems a bit arbitrary.
I was thinking more along the lines that if our minds process outside information in a way that makes sense of that information, partially through simulating it, it does not seem so strange if the structures end up matching the outside structures through some form of convergent evolution.
From a cursory reading of that article I do not see it argue the same thing.
That the universe is the best simulator of itself? That say, simulating water flowing through a pipe, the system doing the simulation re-formulates itself into something that resembles the pipe and the water?
> That the universe is the best simulator of itself?
What is this even supposed to mean? Also, "pipe" and "water" are ridiculously high level constructs, categorisations made by humans. Neither says anything about structures inherent to the universe.
I mean that when working with symmetries, information flow, and fundamental building blocks, certain structures just tend to pop up naturally. Hence fractals and geometrically shapes in places where you might not expect them. Or how laws of thermodynamics suddenly seems to be everywhere in biology all of a sudden now that we started looking[0][1].
https://www.youtube.com/watch?v=5MdSE-N0bxs
The gist of it is that physical data tends to have symmetries, and these symmetries make descriptions of the data very compressible into relatively small neural circuits. Random data does not have this property, and cannot be learned easily. Super fascinating.