> it seems like the Google Home devices start buffering immediately after the wake word, but the Voice PE doesn't.

Google Home devices are always buffering. The wake word just tells it to look back in the buffer and start processing.