Too late, NK already completed all the markdown files needed to both create their hypernuke and recreate the hurricane machine Dick Cheney had left Obama.
Yes. They wrote an AI prompt to cut a tree and gave a presentation about how this is the future of tree cutting. They did not cut the tree, nor build a robot that cut the tree. They fantasised about building a robot that builds a robot that cuts the tree.
I love Groq, but across every single line break in your post there is a glaring issue that is easy to refute with in 15 seconds, even without 300t/s of throughput.
You wasted all of your commentary on snark and sadly unfunny humour, and yet still managed to add nothing.
Groq is more performant for the growing categories of inference-based tasks, wherein Nvidia's advantage in inference depends bulk/batch processing which will make up a smaller category over time, in relative terms.
The future of AI Silicon is inference, and the cost structure of AI data centres is constrained around the current necessity to have 'high GPU utilization' otherwise, the cost / amortization of the chips doesn't work out.
That cost structure is a limitation of Nvidia architecture.
Groq serves a lot faster, and without the limiting batching requirement, which opens hosting arrangements common in most classical hosting scenarios aka without necessarily the high utilization requirements.
Groq has bespoke hardware, lack of CUDA, much lower memory desnsity obviously and they don't have the deep distribution networks and leverage over TSMC that Nvidia has - but pound for pound, were we to be able to 'fire up a server' for our inference needs, it would be Groq, not Nvidia that we'd turn to.
Were they not a later market entrant and didn't have those barriers to entry, they'd be gigantic.
Yes, this! I am dying for need of zerotrust enterprise features and am about to have to actually talk to one of the enterprise sales folks, which will chew up a bunch of time and add stress I’d rather avoid.
Money is the most honest transaction, if people are paying money to anthropic it means its solving their pain. Its the same reason OpenAI and others are copying Anthropic.
This is some late-stage capitalist cope. Money goes to the person who can hoodwink people into thinking their product will do something they care about, not the person with the product that actually does something people want. There are a litany of examples of people swearing up and down by products that have been scientifically proven to __DO NOTHING__. The correlation between popularity and quality is tenuous AT BEST.
With this line of thinking, nobody would have ever built refineries, or fabs, or clouds.
The frontier labs have fantastic margin on inference. You do not understand how fantastic. And they have license to change inputs at will based on profitability.
They are not only innovating on models and tooling, they are innovating on cogs (I wrote this btw, and I’m not going to stop writing this way because Claude discovered it’s brilliant).
Speaking of models, the cost of training is not scaling nearly as fast as demand for inference. Training used to be the biggest cost by far, now it’s not.
So margin is increasing, and guess what else is happening? Customers are finding value. And the customers that are finding value are also the ones who happen to have huge enterprise budgets.
And while this is happening, so is implicit collusion (and lock in, and hype, and all that). And so prices are going up.
They’re going to be just fine man, there is no inference bubble.
They can modulate supply. It’s all going to be fine. You should invest.
> The frontier labs have fantastic margin on inference. You do not understand how fantastic. And they have license to change inputs at will based on profitability.
This. The gross margin on inference is at least 95% if not higher - several open weight models on my tiny consumer DGX Spark easily replace the 15 dollars a day I was paying in tokens for Claw usage with a dollar a day electricity. You add data centre overhead and depreciation, the theoretical net margin will trend lower but depreciation is always far more aggressive than actual product degradation. The old NVIDIA GPU on a 9 year old second hand gaming PC I bought still serves up a small Gemma 4 variant quite reasonably.
To say nothing of the fact that they can just add "figure out how to change the answer to this question to benefit X" at the top of their system query.
It is baffling that any government lets either themselves or their local companies use these tools. Utterly baffling. The potential for total security compromise through these models is ... essentially 100%.
> The frontier labs have fantastic margin on inference.
Source?
The OpenAi filing will be very interesting indeed.
("trust me bro" statements from sama et al does not count, since I don't trust them)
Edit:
The best argument I have seen look at the price of inference from smaller companies running open models. And assuming they are profitable-ish. Their prices are lower than the OpenAi and Anthropics best models, so maybe they do make money on inference (ignoring all other costs)
We're losing a lot of workers. Some might consider that a good thing if they hate the idea of a large exploited underclass (or just people of a certain skin color), but it could be a problem for farms if there aren't enough hands to harvest in time.
Let us hope this results in farms offering Americans better wages for working on farms. Only 1% of the retail cost of produce is due to labour. Farms could double wages to workers and it would barely move the needle on inflation.
I agree, but the farmers aren't pulling in massive amounts of cash either. There are record profits being siphoned from the wages of the workers, and from the profits of the farmers, and from the pockets of the consumer at the register. We'd all be better off if we tracked the money down and took it back.
Put Wall Street out of work and have the bankers work the farms out there in rural Conneticut and NJ and NY. It'll be good for them, put some hair on their chest and give them Real World Experience (tm)
if they are large corporation owned farms as many small family owned farms are going to go bankrupt in the next 2 years because of the price of fertilizer and diesel and higher interest rate.
He destroys trade relations so the farmers can’t sell their produce, then bails them out by taking on more government debt so farmers can dump soybeans in a ditch.
I like my H2D, but it definitely hit a brick wall. I replaced it, and had the same issue with one of the nozzles not printing consistently.
This was after around 700 hours, which isn’t terrible, but working with their support is exhausting. I don’t think I’m going to touch it again until winter, unfortunately.
On the tooling I've made? Feel free to ask here or via email, the Salesforce one was for a fairly large (50k+ employees) company that uses it very extensively but as it often happens they're stuck with a ton of legacy crap. They have some of the AI tooling from SFDC but barely use it (lack of training, or interest) so this solves their immediate problem.
I'm very keen to use their new dynamic workflows (cf's durable execution engine) which would let agents write workflow steps, that way my users can ask an agent to do stuff like "run this report daily and email it to me" and it can work with minimal setup (very basic example, but you get the idea)
reply