No, my bar for proselytizing about security practices is adequate, not perfect. The distinction is that adequate is an absolute bar, not a relative one, so "better" and "worse" are irrelevant until it is achieved since a "better" solution that is inadequate is not a solution that can be used (it is inadequate) and does not provide clear directions to an adequate solution. It is like climbing trees to reach the moon, no matter which is "better", you still aren't going to get there. The solution may be fundamentally different than what is expected.
That does not mean that a "better" inadequate solution is not a path to adequate, it could very well be if the "better-ness" can scale all the way, but that is hard to judge. One strategy for doing so is trying to estimate how far you are from good, that is the point of quantitative analysis. Using the tree to the moon example, if you could estimate the distance to moon, you would quickly realize that every tree you know of is pretty far from the moon, so maybe a different strategy is warranted. In this case, I want to estimate the "security" of an adequate solution. Is $1 enough, $1K, $1M, $1B? For what problem domain? How far are "best practices" from that goal? That is how I would decide if best security practices are worth listening to. The other point of quantification is to compare systems, you claim Google has better security practices, how much better? 10%, 50%, 100%, 1000%? That would change how compelling listening to their practices over others would be.
As you stated above, bug bounties are order of magnitude cost of discovery which, in my opinion, is a reasonably good quantitative security proxy. The Pixel 4 kernel code execution bounty is up to $250K. The iOS kernel code execution bounty is up to $1M. That appears to indicate that Google's offering is less secure by this metric. Even ignoring that, is $1M enough protection for a phone model line (since a bug in one is a bug in all, so a zero-click kernel code execution could potentially take over all phones, though in practice it will probably not even assuming such a vulnerability were used to achieve mass infection)? There were more than 200 million iPhones sold last year, so that is only a per-phone value of 0.5 cents, is that an adequate amount of security? Personally, I think no and I would bet the average iPhone buyer would be less than pleased if they were told that (still might not change their buying habit though). What do I think is adequate? Not sure. $50 is probably fine, $5 seems a little low, $500 is probably high since that is approaching the cost of the phone itself. If I use $50 as the metric of adequate, they are 10,000x off from the measure of adequate which seems pretty far to me. Think about the difference in practices between $100 and $1M and that needs to happen again, how do you even conceptualize that? Even at $0.50 they are still off by a factor of 100x, 1% of adequate from this perspective.
On the point of overestimating the "risk" for most software, I half agree. I believe the truth is that almost nobody cares about security, so the cost of problem for insecurity is almost nil. Companies get hacked and they just keep on going their merry way, sometimes even having their stock prices go up. However, I believe this is also an artifact of misrepresenting the security of their systems. If people were told that the per-unit security of an iPhone is 0.5 cents, they might think a little differently, but instead they are told that the new iPhones are super, duper secure and all those pesky vulnerabilities were fixed, so it is now totally secure again, this time we promise, just ignore the last 27 times it was not true.
On the other hand, large scale systemic risks are massively underestimated. Modern car models are internet connected with each model using a single software version maintained through OTA. This means that all cars in a single model run the same software meaning that bugs are shared on all the cars. If a major vulnerability were discovered, it could potentially allow take over of the steering, throttle, and brakes by taking over the lane-assist, cruise control, and ABS systems. If this is done to all cars of a given model at the same time, it is extremely likely that at least thousands would die. Even ignoring the moral implications of this, that would be a company-ending catastrophe which puts the direct economic cost of problem at value of the company which is a few billion to tens of billions for most car companies. Again, $1M is pretty far from this level, and there is no evidence that such techniques scale 1000x. Any solution that only reaches the $1M level, even if it is "best practices", is not only inadequate for this job, it is criminally negligent in my opinion and I believe most people would agree if it were properly explained to them.
Your focus on bug bounties ignores the existence of other things, like grey hat big vendors and I house security teams. Tavis Ormandy doesn't get big bounty payouts from Google or Apple or Microsoft, he gets a big paycheck instead, but is more effective than pretty much any freelance big bounty hunter.
And again, you consistently overestimate the value of a hack. You're not going to get root on every device. So the idea that apple is spending 5c per device isn't correct.
Again, you're overestimating the risk by imagining a magic rootkit that can simultaneously infect every device on the planet. That's not how things work. It lets your imagine these crazy values of a hack, but again: that's not how things work.
If it did, you'd probably see more hacks that infect everyone so that some organization can extract minimal value from everyone. But you don't see that.
Why? Because that's not a realistic threat model. State actors who, at this point are the only groups consistently capable of breaking into modern phones aren't interested in financing. They're interested in targeted attacks against dissidents.
So anyway, what makes you believe that Googles safety isn't adequate for it's systems, since at the moment anyway, they aren't manufacturing cars.
I am not focusing on bug bounties, I am focusing on cost of discovery which seems like a pretty good metric. Bug bounties just provide a means of learning the cost of discovery that is publicly available and where over-stating is harmful. If you have some other good quantitative measure of security that is publicly available and where over-stating is harmful that would be very helpful.
I stated in a parenthetical that I did not believe they would actually root every device in practice. I used numbers, you can change the numbers to whatever you believe. If someone wanted to mass infect devices using a zero-click kernel code execution, how many do you think they would be able to infect? Let us call that X. $1M is order of magnitude the cost of discovery (since bug bounty ~= cost of discovery) for such a compromise on iOS. Divide $1M / X, that is the per-unit value. Does that number seem good? I said $50 is probably adequate. Therefore, for that to be adequate given this model, you would need to expect a zero-click kernel code execution deployed to mass infect would to infect 20,000 or fewer phones. Do you believe this is the case? If so, then your logic is sound and in your mind iOS security is adequate. It is not for me, since I do believe it would only infect 20,000. As a secondary point, that is only the cost of the compromise with no infrastructure. If they spent another $1M developing techniques for effective usage of the compromise such as better ways to deploy, infect, control, hide, other compromises, etc. how many do you think they would be able to infect? Let us call that Y, Y >= X. In that case I would do $2M / Y to determine the adequacy.
As a counter-example, large-scale ransomware attacks which extract minimal value from large numbers of people occur and have been increasing in frequency and extracted value. Why aren't there more given how easy it is? I don't know. Why didn't somebody crash planes into buildings before 9/11 or drive trucks into people before the 2016 Nice truck attack? These attacks were not very hard, possible for decades, and seem to be highly effective tools of terror, but for some reason they were not done. Hell, it is not like anybody can stop someone from driving a truck into people right now, why aren't terrorists doing it every day given we saw how effective it is and how hard it is to prevent? My best guess is that so few people actually want to engage in terrorism or economic hacks that only an very tiny fraction are done at this time.
This leads into the next point which is that state actors are not the only entities that CAN break into phones; financing a $1M cost of discovery is chump change for any moderately sized business. The government is just one of the few entities who want to as a matter of course and face minimal repercussions for doing so. If you are not the government and hack people for financial gain you can go to jail; not a very enticing prospect for most people. This means that the impact of a compromise is not different, it is just less probable at this time. However, that is not a very comforting prospect since it means you are easy prey, just nobody is trying to eat you yet. And this ignores the fact that since any particular target is easy, if someone is targeted in particular they are doomed. Essentially, not being compromised is at the mercy of nobody looking at you funny because if someone wants to compromise you they can. To provide an example of why this is a problem, if I were a terrorist organization, I would be running an electric power generator and transformer hacking team with an emphasis on bypassing the safety governors and permanently destroying them. It does not matter that there are more economic targets to hit, as long as they choose one in particular they can cause incomprehensibly large problems.
As for Google's security, if I use my default security assumption (based on experiences with other security organizations) that a skilled red team with $1M would be able to compromise and establish a persistent presence with material privileges and remain undetected for a week, then I believe their security is inadequate since I believe such an outcome would easily be able to extract $1M, let alone damage if the goal were just destruction. If the goal were pure damage, I believe that such presence, properly used, should be able to cause at least $100M in damage and I would not find it unreasonable if it could cause $10B in damage if the goal was optimized damage to Google in both material and market ways with no thought for the consequences if caught.
To separate this out for you, there are two primary statements here:
1. The damage that a skilled red team can cause assuming it has a persistent presence with material privileges and remains undetected for a week.
2. The cost of creating a persistent presence with material privileges that remains undetected for a week.
I assert (1) is probably ~$100M. I assert (2) is my default of $1M. Any situation where (1) is materially higher than (2) is inadequate in my opinion, so a convincing counter argument on your side would be convincing me of a value for (1) and (2) where (2) is higher than (1). I find it unlikely you would convince me of a lower number for (1). So, you would need to claim that (2) is ~100M for Google. If you believe so, what is your justification? The minimal standard that would cause me to consider further (not convince, just not directly dismiss), which you are under no obligation to provide, would be: You stating that you talked to an internal security person at Google and they firmly claim that (2) is higher than 100M (I will take you at your word). If you do not know what to ask, you can try: "If you were in charge, would you feel comfortable going to DEFCON and Black Hat and putting out a prize for $100M if anybody could do (1)?". The other requirement is you stating (again, I will take you at your word) that you talked to an internal security person at Google and they inform you that this has been tested internally using a red team with resources in the general vicinity of $100M or more. There are potentially other strategies that might pass the minimal bar, but that is one that I could think of that would be pretty solid. Again, I am not demanding you do so, but if you wish to engage on this point then I don't think any other type of response is particularly productive.
If you believe a zero click iOS compromise would infect more than 20k devices, can you give an example of such a thing happening?
If not, why not? Do you believe that 20000 people would never noticed such a thing over a sustained period?
As for 2: there are public examples (again, Aurora) of teams with more funding being caught in less time. So I think you are underestimating the security capabilities of Google (and similar companies). For example, are you familiar with beyond corp?
5 zero-click compromises. Thousands per week for a total of 2 years before discovery. The 5 chains being: 3 months, 6 months, 10 months, 6 months, 3 months each. At thousands per week, that is 12k, 24k, 40k, 24k, 12k new compromises per chain at a minimum, probably closer to 5x those numbers. Incidentally, at the bottom of the initial post they mention: "I shan't get into a discussion of whether these exploits cost $1 million, $2 million, or $20 million. I will instead suggest that all of those price tags seem low for the capability to target and monitor the private activities of entire populations in real time." which is consistent with my perspective. As a secondary point, I do not claim that Google does not have good offensive security techniques.
Looking at Project Aurora. The wikipedia page states that the attacks began mid-2009 and Google discovered them mid-December. So a potential 6 month window before detection. Google also declares that they lost intellectual property, though the nature of that is unclear, so could be anything from one random email to everything. Given that they already lost information, the attack could have already succeeded in its goal by the time of detection (6 months is a really long time to exfiltrate data, you could literally dump terabytes of data if you had a moderate unimpeded internet connection), "We figured out that we were robbed 6 months ago." is a failure in my book. There is also little information as to the difficulty of the attack. They say it was "sophisticated", "we have never seen this before", but that is what literally everybody says. If you have access to a specific breakdown on the techniques used that would be helpful.
You're not likely to get any more information on Aurora than what's on the wikipedia page. It includes some breakdown of the attacks (among other things, zero days in internet explorer).
> At thousands per week, that is 12k, 24k, 40k, 24k, 12k new compromises per chain at a minimum, probably closer to 5x those numbers.
That assumes every visitor uses iOS 10-12. Which is...not likely. My understanding is that these sites were likely Chinese dissident forums, and I don't think that iOS 10-12 makes up even half of browsers in china. Nor does it make sense that every user is unique. This isn't to downplay the danger of these attacks, but no you're likely looking at compromising 1-2K devices total when it comes down to it.
But again, you're looking at state actors (not even nation state actors at this point, but like the Chinese version of the NSA/CIA) with hundred million or billion dollar budgets. If those are the only people capable of exploiting your software, you're doing an objectively good job.
That does not mean that a "better" inadequate solution is not a path to adequate, it could very well be if the "better-ness" can scale all the way, but that is hard to judge. One strategy for doing so is trying to estimate how far you are from good, that is the point of quantitative analysis. Using the tree to the moon example, if you could estimate the distance to moon, you would quickly realize that every tree you know of is pretty far from the moon, so maybe a different strategy is warranted. In this case, I want to estimate the "security" of an adequate solution. Is $1 enough, $1K, $1M, $1B? For what problem domain? How far are "best practices" from that goal? That is how I would decide if best security practices are worth listening to. The other point of quantification is to compare systems, you claim Google has better security practices, how much better? 10%, 50%, 100%, 1000%? That would change how compelling listening to their practices over others would be.
As you stated above, bug bounties are order of magnitude cost of discovery which, in my opinion, is a reasonably good quantitative security proxy. The Pixel 4 kernel code execution bounty is up to $250K. The iOS kernel code execution bounty is up to $1M. That appears to indicate that Google's offering is less secure by this metric. Even ignoring that, is $1M enough protection for a phone model line (since a bug in one is a bug in all, so a zero-click kernel code execution could potentially take over all phones, though in practice it will probably not even assuming such a vulnerability were used to achieve mass infection)? There were more than 200 million iPhones sold last year, so that is only a per-phone value of 0.5 cents, is that an adequate amount of security? Personally, I think no and I would bet the average iPhone buyer would be less than pleased if they were told that (still might not change their buying habit though). What do I think is adequate? Not sure. $50 is probably fine, $5 seems a little low, $500 is probably high since that is approaching the cost of the phone itself. If I use $50 as the metric of adequate, they are 10,000x off from the measure of adequate which seems pretty far to me. Think about the difference in practices between $100 and $1M and that needs to happen again, how do you even conceptualize that? Even at $0.50 they are still off by a factor of 100x, 1% of adequate from this perspective.
On the point of overestimating the "risk" for most software, I half agree. I believe the truth is that almost nobody cares about security, so the cost of problem for insecurity is almost nil. Companies get hacked and they just keep on going their merry way, sometimes even having their stock prices go up. However, I believe this is also an artifact of misrepresenting the security of their systems. If people were told that the per-unit security of an iPhone is 0.5 cents, they might think a little differently, but instead they are told that the new iPhones are super, duper secure and all those pesky vulnerabilities were fixed, so it is now totally secure again, this time we promise, just ignore the last 27 times it was not true.
On the other hand, large scale systemic risks are massively underestimated. Modern car models are internet connected with each model using a single software version maintained through OTA. This means that all cars in a single model run the same software meaning that bugs are shared on all the cars. If a major vulnerability were discovered, it could potentially allow take over of the steering, throttle, and brakes by taking over the lane-assist, cruise control, and ABS systems. If this is done to all cars of a given model at the same time, it is extremely likely that at least thousands would die. Even ignoring the moral implications of this, that would be a company-ending catastrophe which puts the direct economic cost of problem at value of the company which is a few billion to tens of billions for most car companies. Again, $1M is pretty far from this level, and there is no evidence that such techniques scale 1000x. Any solution that only reaches the $1M level, even if it is "best practices", is not only inadequate for this job, it is criminally negligent in my opinion and I believe most people would agree if it were properly explained to them.