Video compression shouldn't introduce that kind of latency (I get <10ms with Steam in-home streaming), but other than that you're correct, of course.