The idea is apparently that a model that is bad at fixing its own mistakes might become better if you train it on this task using reinforcement learning.
If you give the model access to specialized tools (e.g. web search for question answering) the knowledge doesn't have to be stored in the model weights, which leaves some room for improvement. You'd still be overfitting to benchmarks (since different tasks might require different tools) but not necessarily to specific benchmark questions, so within-domain generalization could be quite good.
As an example for a similar approach, Teapot AI has trained very small models https://teapotai.com/models to only answer questions where the answer can be found within the context window, and although not perfect, they do quite well at this compared to larger, more general models.
good point I have the feeling larger models (20b+) rely too much about their stored knowledge and sometimes fail to use tools because they think they know the answer. smaller specialized tool calling models could be the smart route for the future
The article explicitly addresses this, adjusts US heat deaths upward using a comparable method to the European figures, and still finds that there are many more heat deaths in Europe.
E.g. from your first link:
> ... some researchers say the actual number is far higher. One study that examined data from the late 1990s to the early 2000s concluded that the average number of fatalities annually was roughly 10,000.
It's six times higher in Europe.
Also note that the hook is an elderly man dying when "His air-conditioner, set to 70, was blowing hot air. The temperature inside was nearly 110 degrees." Meanwhile in Berlin, I have no AC I could set to 21°C at all... Fortunately, annual highs so far have been below 39°C, but even that is hard to bear.
Europe not only has plenty of old people, they're also unlikely to have AC at home, so every year during a heat wave, a large number of them (though still a small share of the population) don't make it.
Those threads have lots of comments that are both obviously LLM-generated and also flagged by users (e.g. https://news.ycombinator.com/item?id=48262928 ) So if anything, the spam filter is still not strict enough.
And if a newbie's not-particularly-noteworthy comments are only visible to people with showdead on, is that really so bad? If they keep lurking and occasionally posting, at some point they'll say something interesting enough that people will vouch for the dead comment. That would probably actually lead to a more pleasant first interaction than getting downvoted for a low-effort reply.
Yes, if your LLM sandbox had a huge hole in it guarded only by asking an LLM whether the stuff coming out is low-risk, you would indeed get sand into all kinds of inconvenient places.
So don't do that. If you want to sandbox an LLM, all output of any consequence needs to pass through a human brain qualified to evaluate whether those consequences are desirable or not. If you don't want to do that because reading LLM output is exhausting, you're free to discover the consequences in some other way, but that doesn't mean sandboxing isn't a solution. It just comes with the tradeoff that you can't outsource all decisions to LLMs.
You could certainly bolt GRAM onto an LLM, but that won't magically improve its reasoning.
It's a special-purpose design for constraint-satisfaction problems with simple rules, but complex interactions. E.g. when solving a Sudoku, the set of valid choices at every step is easy to determine, but you could make a series of valid choices that back you into a corner where no more progress is possible and you have to backtrack.
Meanwhile, LLM reasoning failures are more often of the kind where a choice is clearly invalid (as judged by a human observer), but the LLM picks it anyway, because the underlying rule is complex and context-dependent and the model only learned an imperfect approximation that often breaks down.
My vision for what might happen: an LLM emits a "neural constraint satisfaction task" in latent space, kicks a "neural tool call" into a non-LLM architecture, runs that architecture, gets a latent answer back, attends to the answer to generate better text answers for problems that benefit from improved constraint-satisfaction.
But that's a very hard thing to implement, and the gains are uncertain. Thus "might".
Yes, widespread automation of knowledge work is unlikely to decrease total production, so all the goods and services people currently demand will still be provided, but the power dynamic of who is consuming and who is providing might flip around. So a bunch of formerly upwardly-mobile people could end up at the bottom of the social hierarchy while others whom they used to look down upon will be able to afford servants for the first time in their lives.
I see little evidence that any social role reversal is going to occur. This technology is soon going to price out average to poor people when they have to pay true token costs. It might be the case that only the rich and powerful have access to the powerful models.
Poor people with hard-to-automate occupations don't necessarily need to be able to afford token costs for social role reversal to occur. They only need to be able to hire an even poorer person who used to earn a salary that exceeded even those token costs and who was laid off as a result.
After going without food for a few days, even a software dev is going to have to swallow their pride and consider alternative employment options. You might think they could just take a poor person's existing job, but why would for example a meatpacking plant hire a software dev with zero meatpacking experience when they already have lots of experienced meatpackers and meat demand hasn't gone up? Meanwhile, the meatpackers might like it if they could have someone babysit their children, cook for them and clean their now much larger house (which used to belong to a software dev who fell on hard times and had to move out). And a steady supply of desolate characters holding up cardboard signs saying "I used to be a software dev, but now I would do literally anything for a meal" could put such luxury into their affordable range.
Python's import system is extensible: https://docs.python.org/3/reference/import.html#import-hooks It might be possible to create a custom finder that will return 1.0.0 when running "import dependency_c" in dependency_a but 2.0.0 for the same import statement in dependency_b. You'll need to work around the module cache in sys.modules, though. And good luck trying this on a package that also hooks the import system...
The idea is apparently that a model that is bad at fixing its own mistakes might become better if you train it on this task using reinforcement learning.
reply