Everything we can see (which is called our "Hubble sphere") started off as a tiny little (and very massive) speck of stuff at the beginning of time. There were infinitely many tiny little specks, and they all became different Hubble spheres. But all the Hubble spheres all overlap each other, forming one continuous space. The same thing was true about the little massive specks at the Big Bang.

As I said above, at the time of the Big Bang, the universe was like a very dense sheet of rubber, and then it started rapidly stretching, getting less and less dense over time, until it is now the density that we see around us.

Infinities can definitely give you a headache, though. For instance, there are just as many odd integers as there are integers.