> why one person potentially being responsible for hundreds or thousands of deaths is acceptable
I am not sure who exactly is that one person ? Is it Altman, who is according to many people not that knowledgeable in AI in the first place; the scientist who found a breakthrough (who is it ?); is it the president of the United States who is greenlighting the strikes; the general who is choosing the target (based on AI suggestions); the missile designer; the manufacturer; the pilot who flew the plane ?
I get the point of concentrating power in fewer hands, but the whole "all the problems of this world are caused by an extremely narrow set of individuals" always irks me. Going as far as saying there is just one is even mor ludicrous.
I’m fine with holding them all accountable to varying degrees. For example, yes, ultimately the president is responsible, but so is the person who dropped bombs instead of refusing an illegal order; just like the street dealer, gang banger, trafficker, and cartel boss are all guilty of all of their various crimes.
What do you find difficult to understand about that?
Ah the old 'everyone is responsible so nobody is responsible' canard.
I will give you a helpful rule of thumb: when in doubt the guy with a bank account larger than the total lifetime income of hundreds of thousands of people is probably the one to blame.
Ah the old ‘in case of doubt just go after the rich guy’. That makes stuff simple doesn’t it ?
You can establish responsibilities just by counting the number of zeroes in a bank account.
On top of this, it works for everything: the same dude is responsible for wars, the climate, world hunger, child cancer and your bathroom mirror being fogged this morning.
He lost access to the wallet either by mistake (never even saved the key) or because he willingly destroyed the key for philosophical reasons. Or he is just dead.
You can already do that today by hiring a security researcher. I can guarantee you that Apple has access to people of a higher caliber than my startup.
I could see a world where 1 year from now I can have glassing do a full sweep of my codebase for a given price (say: $10k). Running that once a year is within my means and would make my software much more secure than it is today.
Yeah but even Carlini who is a good security researcher said he has found more valid vulnerabilities in the last week than his entire career before this. That sounds like it’s clearly better/faster/cheaper than a human security researcher that would cost $300,000 a year.
I spend well over that of my employers money on pentesting every year. I’m absolutely certain Claude could perform as good or better a job using what’s available today.
It had crossed my mind that an AI agent pentester would be an interesting product to build. Once again though, the labs are just going to build it because it’s a thin thin wrapper.
Beyond existing software with vulnerabilities, the really important aspect of this for Anthropic et al is that the gigatons of code that are being generated every day needs to be secured.
There are quite a few such startups already out there. Results are mixed so far. Though I believe they get much better over the coming months and years.
Bait what exactly ? Getting the user to type "yes" ? Great accomplishment.
Sometimes I want the extra paragraph, sometimes I don't. Sometimes I like the suggested follow up, sometimes I don't. Sometimes I have half an hour in front of me to keep digging into a subject, sometimes I don't.
Why should the LLM "just write the extra paragraph" (consuming electricity in the process) to a potential follow up question a user might, or might not, have ? If I write a simple question I hope to get a simple answer, not a whole essay answering stuff I did not explicitly ask for. And If I want to go deeper, typing 3 letters is not exactly a huge cost.
I’m not privy to their data on what this does to engagement, but intuitively it seems like the extra inference/token cost this incurs doesn’t align with their current model.
If they were doing it to API customers, sure, but getting the free or flat-rate customers to use more tokens seems counterproductive.
We’ll see how this plays out. It’s a turbocharged version of enshittification, at a time when other models are showing stronger growth in B2B and other valuable markets.
I canceled my ChatGPT subscription and jumped to Claude, not for silly political theater, but just because the product was better for professional use. Looking at data from Ramp and others, I’m not alone.
So human become just a provider of those 6 digits code ? That’s already the main problem i have with most agents: I want them to perform a very easy task: « fetch all recepts from website x,y and z and upload them to the correct expense of my expense tracking tool ». Ai are perfectly capable of performing this. But because every website requires sso + 2 fa, without any possibility to remove this, so i effectively have to watch them do it and my whole existence can be summarized as: « look at your phone and input the 6 digits ».
The thing i want ai to be able to do on my behalf is manage those 2fa steps; not add some.
This is where the Claw layer helps — rather than hoping the agent handles the interruption gracefully, you design explicit human approval gates into the execution loop. The Claw pauses, surfaces the 2FA prompt, waits for input, then resumes with full state intact. The problem IMTDb describes isn't really 2FA, it's agents that have a hard time suspending and resuming mid-task cleanly. But that is today, tomorrow, that is an unknown variable.
It's technically possible to use 2FA (e.g. TOTP) on the same device as the agent, if appropriate in your threat model.
In the scenario you describe, 2FA is enforcing a human-in-the-loop test at organizational boundaries. Removing that test will need an even stronger mechanism to determine when a human is needed within the execution loop, e.g. when making persistent changes or spending money, rather than copying non-restricted data from A to B.
Reading through the discussion I was also thinking of the other fly.io blog post around their setup with macaroon tokens and being able to quite easily reduce the blast radius of them by adding more caveats. Feels like you could build out some kind of capability system with that that might mitigate some risks somewhat.
Regarding sexism; most tournaments in Chess (including the world championship) are fully open and are thus gender netral: anyone can participate regardless of sex/gender and will compete on equal footing.
Women only categories have been created to give women visibility because they mostly were not able to reach advanced levels in the open format.
Some women choose to compete with men (Judit Polgár being a somewhat recent example) but most go straight to the women only tournaments to have a shot.
The men vs women « bias » is not unproven, they litterally had to create entire categories of competiton to account for it.
That’s true for “tips and tricks” knowledge like “which model is best today” or “tell the model you’ll get fired if the answer is wrong to increase accuracy” that pops up on Twitter/X. It’s fleeting, makes people feel like “experts”, and doesn’t age well.
On the other hand, deeply understanding how models work and where they fall short, how to set up, organize, and maintain context, and which tools and workflows support that tends to last much longer. When something like the “Ralph loop” blows up on social media (and dies just as fast), the interesting question is: what problem was it trying to solve, and how did it do it differently from alternatives? Thinking through those problems is like training a muscle, and that muscle stays useful even as the underlying technology evolves.
> what problem was it trying to solve, and how did it do it differently from alternatives?
Sounds to me like accidental complexity. The essential problem is to write good code for the computer to do it's task?
There's an issue if you're (general you) more focused on fixing the tool than on the primary problem, especially when you don't know if the tool is even suitable,
It does seem like things are moving very quickly even deeper than what you are saying. Less than a year ago langchain, model fine tuning and RAG were the cutting edge and the “thing to do”.
Now because of models improving, context sizes getting bigger, and commercial offerings improving I hardly hear about them.
I’m not sure destroying other people’s property is the best way to make them sympathetic to your cause.
I don’t own a Ring camera (or any similar device), but the idea that someone could spend time unnoticed on my porch, messing with my stuff, right where my daughter likes to play on weekends, makes my skin crawl.
If that happened to me, I’d probably just double down on security to be honest. Knowing that some people actually feel it's the right thing to do makes me wonder if I shouldn't start today.
To be clear, I have no issue with someone peacefully informing people in their neighborhood about the potential dire consequences of enabling "share images of my doorbell with the government or other private agencies", that's all fine to me. But if you feel the need to impose your views by harassing me about it or by breaking the law to get your point across, you won't get an ally in me.
It's always the same. Go back and think about the history you read and stories you've loved. Were you upset when the Rebels destroyed the Empire's property? Should they not have blown up the death star? Should they have gone through "proper channels". Go look at any revolution that you side with, tell me they didn't destroy property. I understand your comfortable but there are literally minorities, often times US citizens, getting rounded up and denied their rights. So you can sit idly by and criticize those that fight this system. However, you are so obviously on the wrong side of history and you would recognize it in any other era except your own.
> I’m not sure destroying other people’s property is the best way to make them sympathetic to your cause.
We're in a slow moving civil war at this point. Looking for sympathy stopped making sense a long time ago. You're either pro humanity or pro property tbh
>We're in a slow moving civil war at this point [...] You're either pro humanity or pro property tbh
You don't realize this type of thinking is exactly what contributes to the "civil war"? Same with all this virtue signaling where if you're even slightly for some sort of immigration enforcement you're labeled as not being "pro humanity" or whatever, and then a populist gets in power because the other side's rallying cry is "there's no illegal on stolen land". In the wake of the killing of Renée Good, Trump's approval on immigration was 48% approve to 52% disapprove. In the same survey, who do you think voters trusted more on immigration? Still Republicans, 44% to 33%.
> You don't realize this type of thinking is exactly what contributes to the "civil war"?
Of course. But we need meaning and values in our lives, both of which have been absent from politics my entire life. At some point we're due for course correction, or I can't bear to live here anymore.
> if you're even slightly for some sort of immigration enforcement you're labeled as not being "pro humanity" or whatever, and then a populist gets in power because the other side's rallying cry is "there's no illegal on stolen land".
Both of these people are liberals detached from reality. The opposing side would stand for better material conditions for everyone.
If you're not going to ally with the people fighting the surveillance systems that are currently being used by the secret police to disappear and kill people what does that make you. My cause doesn't need your sympathy it needs to stop this horror. I'm not quite saying "with or against" but you are saying "against."
>If you're not going to ally with the people fighting the surveillance systems that are currently being used by the secret police to disappear and kill people what does that make you.
1990s Ireland:
A: "hey guys, maybe it's a bad idea to set off bombs in public places to promote Irish independence. You won't get an ally in me."
B: "If you're not going to ally with the people fighting British that are currently subjugating the Irish what does that make you. My cause doesn't need your sympathy it needs to stop this horror. I'm not quite saying "with or against" but you are saying "against.""
It’s not that nobody cared, it’s that the cost of building and maintaining CLIs, relative to the usage they got, often didn’t make economic sense. In fact, this is the first time I’ve seen someone want to use Slack via a CLI, not a TUI, an actual CLI. APIs, on the other hand, had plenty of real usage and made business sense, so most services offered them.
With AI, two things have changed: (1) the cost of building a CLI on top of a documented API has dropped a lot, and (2) there’s a belief that “designed for agents” CLIs will enable new kinds of usage that weren’t practical before and that will move the needle on the bottom line.
There are plenty of “chill and peaceful” city and town builders that trade realism for prettier, more idealized places.
In more simulation-focused games, cycling and walking paths are often available, and you can use them, but they come with many of the same constraints they face in the real world. In practice, that means they are usually not efficient as the primary way to move large numbers of people across a large city.
Reading your comment, it sounds like you want a game that is realistic in most respects, but treats transportation differently, in a way that makes your preferred options the optimal strategy. That is going to be hard to find, since transportation is a core part of city-building sims, and developers tend to pick either realism or a more utopian/fantasy model rather than mixing both in a single game.
That's not what I want at all. I want a more realistic sim that deals with issues such as sprawl, food deserts, transportation elasticity of demand, mental health issues (and their impact on crime and productivity), and a network-flow theoretical model of transportation and commuting contributes to all this. Building a bunch of sprawling suburbs that feed into a dense downtown core should make your citizens' commute times shoot way up and lead to misery.
A well-built large city isn't just going to be 100% biking and walking paths, it's going to have streetcars, light rail transit, subways, and buses as well as roads with cars. The difference is that people shouldn't be forced to commute across the entire city to get to work because you decided to cram all of the commercial zoning into one downtown core.
> The difference is that people shouldn't be forced to commute across the entire city to get to work because you decided to cram all of the commercial zoning into one downtown core.
Isn't the point that they should be, if that's how I choose to build a city, and they don't have to be, if you choose to build it otherwise? The entire point of a sandbox city-builder is, I assume, that it's a sandbox, and not a dogmatic interpretation of a childish Reddit meme.
It was pointed out elsewhere in this thread that SimCity already distorts reality in an ideological way: it lets you have tons of traffic without worrying about parking. It just gives you magical free underground parking everywhere that you never have to think about, in order to avoid the usual suburban parking sprawl hellscape.
The point is to illustrate that SimCity isn't a blank-slate, value-free sandbox city-builder. It has rules and those rules have been made deliberately unrealistic in ways that favour North American style cities.
It's like a fluid dynamics sandbox that causes water to flow uphill rather than settling into the valleys.
Car-centric transportation is not efficient. Not remotely. They have absolutely terrible bandwidth, and they balloon the size of cities apart the more you try to increase the speed to bring them closer together.
If you think Simcity and Cities: Skylines are realistic depictions, then ask yourself why Simcity famously has no visible parking whatsoever (or don't: the devs are on record saying they excluded it because it made the cities look terrible, there's no need to speculate here), or ask yourself why Cities: Skyline added car pokeballs (where drivers get out of the car and put the car in their pocket) or straight-up delete cars when traffic gets too heavy.
> Reading your comment, it sounds like you want a game that is realistic in most respects, but treats transportation differently,
It's the opposite, no? Most city builders cheat to be able to be fun. Like, with the amount of roads one build in Sim City, half the map would have had to been parking lots to account for that amount of traffic. But that's boring gameplay, so they remove that constraint to make a fun game. Aka you never have to deal with the consequences of making your city car dependent.
Edit: See another comment from CalRobert about exactly this.
The original SimCity was perfection - you could build no roads and nothing but rail! ;)
Cities Skylines with all the DLC and the right transportation mods gets pretty “realistic” in that you can build a transit paradise but the car still exists.
[citation needed] that some combination of "New Urbanism, traditional neighbourhood design, streetcar suburbs, one-way streets, bike paths, walking paths, mixed-zone walkable villages (light commercial with residential), smaller single-family houses and duplexes, triplexes, houses behind houses." is not in fact optimal! (For certain objective functions)
I am not sure who exactly is that one person ? Is it Altman, who is according to many people not that knowledgeable in AI in the first place; the scientist who found a breakthrough (who is it ?); is it the president of the United States who is greenlighting the strikes; the general who is choosing the target (based on AI suggestions); the missile designer; the manufacturer; the pilot who flew the plane ?
I get the point of concentrating power in fewer hands, but the whole "all the problems of this world are caused by an extremely narrow set of individuals" always irks me. Going as far as saying there is just one is even mor ludicrous.
reply