You are absolutely right. If you look at most housing forums on the internet, for example most people looking to buy housing were absolutely certain there was a bubble going on. They just knew that (i) ordinary people in their neighbourhood could not afford the houses in their neighbourhood and (ii) there were no rich people coming in buying those houses either so something had to give.