Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

if machine-learning can find all these holes

why can't machine-learning write a product from scratch that is flawless?



Who said it can't? https://news.ycombinator.com/item?id=47759709 appears to be a nearly flawless (per spec) zip implementation.


the AI found no bugs in AI code

sure buddy


Could you please stop breaking the site guidelines? We've already asked you once.

https://news.ycombinator.com/newsguidelines.html


Sorry, I got carried away, the whole blind trust in AI tools gets me wound up a bit too fast, I'll try to be better.

> We've already asked you once.

there are no notifications of any kind about that or the fact the comment has been flagged so I genuinely didn't notice previous one and only noticed this after I noticed rate limiting.


No, a collection of fuzzers and the lean proof assistant found (almost) no bugs.


Because the problem is asymmetric: the attacker only needs to find one hole at one time. The defender has to be flawless forever.


It’s easier to break something than it is to make something that cannot be broken.


LLMs certainly make it more feasible to rewrite a product in a memory-safe language, eliminating a whole class of bugs.

Flawless software is hard for an LLM to write, because all the programs they have been trained on are flawed as well.

As a fun exercise, you could give a coding agent a hunk of non-trivial software (such as the Linux kernel, or postgresql, or whatever), and tell it over and over again: find a flaw in this, fix it. I'm pretty sure it won't ever tell you "now it's perfect" (and do this reproducibly).


If humans can find bugs, why can't humans write flawless code?

Whatever the answer to that conundrum might be, LLMs are trained on these patterns and replicate them pretty faithfully.


Have you ever met a security engineer? I’ve never met one who was also a good engineer (not saying they don’t exist, I just haven’t met one). Do they find vulnerabilities? Sure. Could they write the tools they use to find vulnerabilities, most probably not.


How do you define flawless though?

The CVEs here have their fair share of silly C problems, but also more rigid input validation and handling. These more rigid validations exclude stuff which may even be valid by the spec, but entirely problematic in practice.

As examples, take a look how many valid XML documents are practically considered unsafe and not parsed, for example due to recursive entity expansion. This renders the parsers not flawless and in fact not in spec.

Or, my favorite bait - there should be a maximum length limit on passwords. Why would you ever need a kilobyte sized password?


Just because something is good at finding bugs, it may not find all the bugs. Finding a bug only tells you there was one bug you found, it doesn't tell if the rest is solid.


You could argue the answer to this question depends on if you believe P=NP




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: