And when that bug is fixed, assuming modern update practices, it is fixed for everyone. In effect, human drivers can only learn from their own mistakes whereas autonomous systems learn from every mistake made by their system's drivers.

But can you really fix a bug in a machine learning algorithm? Sure, you can retrain it on the data which caused the crash, but could that mean you created another bug somewhere else?

That's exactly why I am not looking forward for self-driving cars. Rash drivers in most cases do know they are doing something stupid and dangerous (like going way over speed limit, overtaking where visibility is limited, etc.) AI on the other hand is prone to "honest mistakes" which can be deadly.

They shouldn't be because even as a cautious driver you are subjected to noncautious drivers and, more importantly, no matter how cautious you are a well designed AI is almost certainly better.

Both cautious & rash drivers face the same external factors, but latter are more likely to wreck themselves, sometimes without involving others. So probability of an accident for latter are higher.

AI may be good at reducing the average number of accidents, but when a tricky situation occurs, the probability of accident could be in this order "rash > AI > cautious".

To be honest, i had the same assumptions on cautions and rash drivers. But then i learned to know my new boss- he drives ralley-race cars- and to be honest, i feel much safer in such a ralley car taking to its physical limits by someone who knows the car- then in the car of a cautious driver, who is just cautious because he/she is aware of how little they know/controll the actual physical event of driving a car.

I know such a thing is nont quantifiable thus, the laws min(ability) applies, none the less, one should not judge from appearance here, but rather from actual incidence lists.