The issue is that most tasks do not require frontier-level intelligence, but companies like OAI can really only profit off of the frontier. Capabilities from a year or two ago are so outdated that even OpenAI gives it away for free and there are many other models biting at their heels. In other words they are spending huge amounts of money to cash in on a depreciating asset.
So one possible future is that frontier-level training becomes so expensive and the use cases so sparse that it simply isn’t viable to keep going bigger.
Multiply "inference + backwards pass (~2x inference cost) + activations (vram overhead)" by batch size (thousands) to get to the actual RAM and compute cost. Optimizer like ADAM adds only two or three model-sized overhead.
And last, but not least, you need only one hidden layer kept in RAM for inference, but you need all of them (61 for Deepseek models) kept in RAM for computing gradient for one sample.
Microbatch size is a hyperparameter, it can be set to 1 and work just as effectively. With gradient accumulation it's equivalent even. Large batch sizes are used to increase parallelism, and sometimes to reduce variance in the loss signal (at the cost of increased bias).
Batch size is frequently limited by compute bottlenecks well before memory.
Is it possible this is survivorship bias? Maybe other forums with much less strict moderation simply wouldn’t have survived long enough to complain about.
A condo will have an HOA which is responsible for things like fixing the roof.
However, not all HOAs are actually financially responsible. So they might raise monthly fees, issue “special assessments” (lump-sum charges that can be $10k+) or take on loans. And they decide when they will do that.
Or, if we consider the fact that an LLM’s performance depends on the task’s similarity to others in the training set, it could be that one person is doing a fairly novel task and another is doing something very well represented in online code.
That doesn’t mean they like it now, and more importantly it doesn’t mean it was actually the best choice at the time.
I used to be on a team that used Mongo for a relational database because Mongo was trendy at one time. So all joins had to be done in the application layer.
Why would that make you more conservative? Republicans don't have any answers to fix the US healthcare system. It would be far more plausible that someone that experienced the failure of the health care system would vote for a populist position here, and for healthcare the more convincing populist position is certainly the left one.
The current healthcare system was mostly shaped by Democrats with the Affordable Care Act (sometimes called Obamacare) at a time when Democrats held a filibuster proof majority.
The ACA is grounded in a lot of political policy going back to the Nixon era, and draws from quite a bit of conservative ideas. The individual mandate itself, for example, was a Heritage Foundation proposal from the late 80s, and was ironically one of the main targets of Republican objection during and after the implementation of the ACA.
The ACA is, in fact, warmed over RomneyCare(tm) (a Republican, please note) from Massachusetts.
The fact that Kentuckians loved Kynect and hated Obamacare--which are the exact same thing (aka the ACA)--tells you everything you need to know about the Republican voting public.
Additionally, the fact that Democrats took a pro-corporate conservative policy package and rebranded it speaks volumes about how much daylight actually exists between the parties when you ignore the culture war rag waving.
You call that "the uniparty", while other here at the same time say they want less polarisation and more bipartisan consensus.
Make up your mind.
(I am old enough to remember that Obama convinced the Democrats that it was better than the existing system, and that small progress was better than keeping the status quo or waiting another decade to have maybe, someday the numbers to pass a better bill)
Through what mechanism does dissatisfaction with our for-profit healthcare system lead to support for eg. expansionist foreign wars or aggressive policing policies?
That's misunderstanding the paper. The correlation here is with outcomes, not support. Republicans may very well be more "jaded" with the healthcare system. But that doesn't explain why they die early.
It’s not just the price of keeping the servers humming. You have to pay people to maintain the software. So now you’re paying hundreds or thousands of people to maintain zombie software.
Yeah, there are costs for sure. But this is "zombie software" with millions, probably tens of millions of users. And Google has ~80,000 engineers? And the company prints an incredible amount of money.
I think the real cost/risk here is not financial, but strategic, i.e. preventing a loss of focus.
The common critique of Googles actions - the organization has profits, therefore there is no problem engaging in less profitable activities — strikes me as superficial.
It’s not about investing any given portion of revenues, it’s investing optimally. There are opportunity costs that must be considered in investments (and that means Net Present Value calculations).
Google’s revenue and profits are for the shareholders. When revenue is directed back into the business the question simply isn’t if the whole business will make money, it’s if this investment is optimally profitable considering all the alternatives. If a support engineer on Google+ generates $X over 5 years, but that same resource would generate $3X working on Gemini then dictating eternal Google+ support is robbing future Google of revenue.
Investments need to be individually justified, but also better than the alternatives to make fiscal sense. Even though that sucks for pleased consumers.
Yes, but this is exactly my point. You can't perfectly calculate exactly how much revenue is lost due to breaking trust with users because you repeatedly sunset projects that they've relied on.
Maybe if Google had those support engineers on Google+ for 5 years generating $X, that would've created enough trust that Gemini could now generate $4X.
Of course it is possible Google will create something orders of magnitude better but I doubt it. Amazon is already doing product recommendation on their front page and while it certainly drives sales it doesn’t turn people into zombies buying products they don’t want.
What did work really well was the “1 click buy” button. Reducing the friction for people who already want to buy is usually a much higher ROI than persuading people who previously weren’t interested.
I really don't get why 1CB isn't more common than it is.
The obvious answer is "patents", but software patents aren't valid in Europe; and besides, the Amazon patent has expired; and European stores are, if anything, far, far more horrendous than American ones.
I guess upsells are more profitable than one-click these days?
That assessment highly depends on your region I assume. They are far from 7 day shipping for me. In fact, it's the same 2-3 days as Amazon for non-Prime customers here. Speaking of non-Prime, free shipping is also at 50€.
The limited selection is certainly true but mainly it misses drop shipping stuff and what's essentially a mirror of aliexpress with a huge markup. I don't really miss those. In fact, it requires a considerable amount of time to navigate through those offers on Amazon, so not having them is a plus in my book.
Me, too. I love that they have tables with return and warranty percentages so you can see which products other people kept and which products only keep breaking.
Thank you - never heard of galaxus.com before, but it is fantastic ! So much more high-quality than the Temu-cheap-shit-reselling storefront that Amazon is nowadays.
My favorite example of why advertising/marketing is important but not overwhelming/unstoppable is in the hit-based industries: If the industry had a "make a superstar" button or a "make a blockbuster movie" button they could just keep mashing, they'd be mashing it constantly. You wouldn't see franchises go from top-of-the-world to decline like Marvel or Fast and Furious. You wouldn't see expensive new bombs or failure-to-launch reboots. There is countless chatter out there about industry plants, organic vs PR-shilling word of mount campaigns, etc. But... if that's all it took, it would be wayyyyy more constant. A lot of words spilled recently about a band, Geese, and how their buzz wasn't as organic as it seemed... for a band that's still, at the end of the day, quite niche and small-in-audience...
It's very hard to get a huge hit without marketing - even great word of mouth benefits from amplification - but it's also near-impossible to force a hit into an audience that isn't vibing with it. The highest-grossing movies, or highest-listened pop music, is a combo of marketing + accurately hitting extremely-common/trendy tastes. See also iPhone marketing vs Windows Phone marketing. I thought Windows Phone was better; none of my friends or coworkers was convinced even after I showed them my phones. The mass media consumer may not have thought much about their tastes or tried very hard to be more adventurous, but that doesn't mean they don't like the stuff they're eating.
I think this is still in many ways bad - at the very least, it's incredibly inefficient to have a billion-dollar zero-sum "pick this one over that one" industry. But I don't think it's a deadly threat. (See also any number of "tons of money and star-power behind them" failed political candidates too...)
Meanwhile, here in Japan I've been in touch with several companies, some small, some quite large (e.g. some building companies) which have a policy of no advertising. No marketing. None. They don't do it. They mostly do have web sites, but that's all. They say they rely on word of mouth, and from what I can tell it works, they're all busy. They say that instead of using money for advertising they want to use everything for the products.
I think marketing is one factor for success, quality is another, but an under-appreciated third one is just getting more shots on goal until one finally succeeds.
I think this is one of the most under-appreciated explanations of why American-style capitalism succeeded where soviet-style communism failed. In theory, communism is more economically efficient, as you don't have multiple companies duplicating work and re-inventing variations on the same idea. In practice, it's much harder to judge an idea than a finished product, so the best way to make a good product is to do it by evolution, not up-front design. If the whole country is oriented around making X happen, X must happen, whether it is a good idea or not.
In general, markets are a really elegant solution to the allocation of goods. It’s really difficult to plan out how steel should be allocated or how many pairs of shoes we should make this year.
Even worse, many Soviet factories were incentivized to lie about the quantity of goods produced or sacrifice quality to meet quotas. So as a first order, approximation, markets are really nice solution to this.
Of course there are many known market failures, which can disrupt such a fragile system if not addressed. For example, monopolies and information asymmetries.
Some people also believe that part of the failures of the planned economy was a limitation of the technology at the time. Without computers, it wasn’t that easy to track the outputs of factories to enforce quality. Those people argue that a modern day planned economy might look a little bit more like Amazon does internally. Maybe still not a great place to live though.
So one possible future is that frontier-level training becomes so expensive and the use cases so sparse that it simply isn’t viable to keep going bigger.
reply