You do not have the statistics to back that statement up.
People usually have less time for games once they have kids, but they have less time for basically everything. Super Mario Bros came out 28 years ago; it's existed for essentially the entire life of anyone younger than mid-30s, into people in their 40s for it being around since their childhood. And that still doesn't even cover people that grew up with an Atari 2600 (36 years ago) or grew up in the culture of arcades (Pong was 41 years ago).
Even getting past the "competitive level" that the GP is going on about, the comparison to spending the afternoon "watching the game" is exactly right, and it's really not that big a deal for most people.
(the best statistics you'll find, by the way, put the median age of gamers at 30--meaning the every single person in the group you think soley cares about games is matched by a person older than that--though the methodology is so-so and it includes a wide variety of "gaming", including on the ever-present mobile device)
There's nothing stopping a competitive gamer from having a job or family. You also don't need to play 10 hours a day to remain competitive.
If you sleep 7 hours a day and work 40 hours a week there's still 9 hours a day to do whatever you want and free weekends assuming you work a typical 9-5 job which might be common but certainly isn't what everyone does.
How you choose to spend those remaining 9-10 hours a day is up to you. Some people would rather watch TV for 3 hours a night and sleep more so they lose almost half their free time because they enjoy sleeping in and watching some shows. Nothing wrong with that but not everyone spends their time doing that.
Well, statistically speaking, most of them do. They get a job, family, etc.