Crucial question - is the calibration a one time effort where the same calibration can be used to calculate any number of results within the calibrated space? In other words, can you amortize the calibration cost across 1,000 runs and come out an order of magnitude ahead?
Just saw this! As Aaronson mentions, coupled with the IBM work, if it does work out, it should be possible to verify that the 53-bit samples are actually valid through the cross entropy test, which would be very exciting.
See the comments on Kalai's post from Peter Shor, where Kalai attempts to address this, and my reply to mantap on this thread where I try to give my poor understanding of the critique.
I'll just say the line between being unfairly critical and raising valid points seem to be a very thin one.
Especially when Peter Shore is disagreeing with the critique (and I agree with him).
Saying that a quantum computation does not qualify because it included (a short) classic computation does not seem a fundamentally fair criticism to me.
