I'm curious about the observed PPS rate. 65 Gb/s is annoyingly large, but network interfaces generally hit pps limits first.
The bandwidth graph in this post and post mortem entry is quite interesting. A lot of incoming bytes from customer origins. I'd guess the system cache hit ratio is only 60-70% at peak, dropping to maybe 20-30% during trough. From that I would assume the cache width is quite small, maybe 8-12 hours LRU?
I could be misreading that if the average object size is closer to 5kb than 50kb, or if a large number of customers are using it a proxy only fashion.