Finished reading the article, having really enjoyed it (I grew up with Terry's books), came back to the HN comments and the top comment is someone ranting "dIS iS aye-EyE sLoP"
What a terrible, terrible timeline we live in now. Seriously. I genuinely hate it.
We're building a product [1] to compete with DataGrip, a JetBrains product.
Many people told us we were crazy to compete with such a mature product as DataGrip before we got started.
It has been fascinating to speak to people who use database apps and to learn about their experiences.
Now, we have many, many customers telling us that they have cancelled their DataGrip/JetBrains sub and have switched to using our product, mainly due to speed but also cost.
I'd love to give this a try, I work at a privacy sensitive company, can I ask in advance is it possible to opt out of:
> We automatically collect certain information when you use DB Pro:
> Device information (IP address, browser type, operating system)
> Usage patterns and interaction with our services
> Log files and analytics data
> Cookies and similar tracking technologies
My Neo runs like a sowing machine with 8GBs. So well that I honestly cannot think of any instances where I have felt the system slowdown. I use the device appropriately though -- I'm not trying to run SOTA LLMs locally or anything. For web browsing, light programming, and an iPad replacement, the Neo has exceeded my expectations. Compared to my M4 Mini, I honestly cannot tell a difference.
One of my favourite things of being on HN is reading comments like this. Namely, devs who worked on games I played growing up. I absolutely love hearing stories from their past about little technical nuances like this comment. The more technical / specific, the better.
I'd honestly love to compile a book of "war stories" told by devs like netcoyote.
This is a great idea, but respectfully, if you're going to get traction you need to be the one instigating getting people to talk to you. Have a pitch, have an explicit ask, and be willing to put effort into making it happen.
Correct. Order-of-magnitude-wise, it's roughly the same as the alternatives.
In the context of writing a new service for a new company, you should not spend one second thinking about whether your technical choices will allow you to serve 100,000 requests per second, or 150,000 requests per second. If you are, you are focusing on the wrong thing. If you get to 1,000 requests per second with a real paying client base you already achieved more than most dream of.
On the other hand, if you are optimizing a mature distributed low-latency equity trading system that is consuming ten's of thousands of market data ticks per second, a 50% improvement in performance on a 20 machine cluster might turn into some real $$$ savings. But that's not what this article is about.
Counterpoint, Meta is currently (and for the last decade) trying to rewrite MySQL so it is basically Postgres. They could just change their code so it works with Postgres and retrain their ops on Postgres. But for some reason they think its easier to just rewrite MySQL. Now, that is almost certainly more about office politics than technical matters...but it could also be the case that they have so much code that only works with MySQL that it is true (seriously doubtful).
You are just mislabling good architecture as 'premature optimization'. So I will give you another platitude...
"There is nothing so permanent as a temporary software solution"
Your article completely ignores operational considerations: backups, schema changes, replication/HA. As well as security, i.e. your application has full permissions to completely destroy your data file.
Regardless of whether most apps have enough requests per second to "need" a database for performance reasons, these are extremely important topics for any app used by a real business.
The irony isn’t lost on us, trust me. We spent a while debating whether to even publish this one.
But yeah, the page cache point is real and massively underappreciated. Modern infrastructure discourse skips past it almost entirely. A warm NVMe-backed file with the OS doing the caching is genuinely fast enough for most early-stage products.
props for actually publishing it tbh. transparent engineering takes are so rare now, usually its just seo fluff.
weve basically been brainwashed to think we need kubernetes and 3 different databases just to serve a few thousand users. gotta burn those startup cloud credits somehow i guess.
mad respect for the honesty though, actually makes me want to check out db pro when i finally outgrow my flat files.
I'm feel like I could write another post: Do you even need serverless/Cloud because we've also been brainwashed into thinking we need to spend hundreds/thousands a month on AWS when a tiny VPS will do.
Serverless is cheap as hell as low volumes. Your tiny VPS can't scale to zero. If you're doing sustained traffic your tiny VPS might win though. The real value in Cloud is turning capex spend into opex spend. You don't have to wait weeks or months to requisition equipment.
id 100% read that post. the jump from free tier serverless to why is my aws bill $400 this month for a hobby project is a rite of passage at this point. a $5 hetzner or digitalocean box with dokku/docker-compose is basically a superpower that most newer devs just bypass entirely now.
You are both right, with the exception that it requires knowledge and taste to accomplish, both of which are in short supply in the industry.
Why setup a go binary and a json file? Just use google forms and move on, or pay someone for a dead simple form system so you can capture and commmunicate with customers.
People want to do the things that make them feel good - writing code to fit in just the right size, spending money to make themselves look cool, getting "the right setup for the future so we can scale to all the users in the world!" - most people don't consider the business case.
What they "need" is an interesting one because it requires a forecast of what the actual work to be done in the future is, and usually the head of any department pretends they do that when in reality they mostly manage a shared delusion about how great everything is going to go until reality hits.
I have worked for companies getting billions of hits a month and ones that I had to get the founder to admit there's maybe 10k users on earth for the product, and neither of them was good at planning based on "what they need".
> weve basically been brainwashed to think we need kubernetes and 3 different databases just to serve a few thousand users. gotta burn those startup cloud credits somehow i guess.
I don't think it makes any sense to presume everyone around you is brainwashed and you are the only soul cursed with reasoning powers. Might it be possible that "we" are actually able to analyse tradeoffs and understand the value of, say, have complete control over deployments with out of the box support for things like deployment history, observability, rollback control, and infrastructure as code?
Or is it brainwashing?
Let's put your claim to the test. If you believe only brainwashed people could see value in things like SQLite or Kubernetes, what do you believe are reasonable choices for production environments?
i think you missed the "on day 1" part of my comment. k8s, iac, and observability are incrdible tools when you actually have the scale and team to justifiy them.
my point is strictly about premature optimizaton. ive seen teams spend their first month writing helm charts and terraform before they even have a single paying user. if you have product-market fit and need zero-downtime rollbacks, absolutly use k8s. but if youre just validatng an mvp, a vps and docker-compose (or sqlite) is usually enough to get off the ground.
> i think you missed the "on day 1" part of my comment. k8s, iac, and observability are incrdible tools when you actually have the scale and team to justifiy them.
No, not really. It's counterproductive and silly to go out of your way to setup your whole IaC in any tool you know doesn't fit your needs just because you have an irrational dislike for a tool that does. You need to be aware that nowadays Kubernetes is the interface, not the platform. You can easily use things like minikube, k3s, microk8s, etc, or even have sandbox environments in local servers or cloud providers. It doesn't matter if you target a box under your desk or AWS.
It's up to you to decide whether you want to waste your time to make your life harder. Those who you are accusing of being brainwashed seem to prefer getting stuff done without fundamentalisms.
Definitely appreciate the post and the discussion that has come from it... While I'm still included to just reach for SQLite as a near starting point, it's often worth considering depending on your needs.
In practice, I almost always separate the auth chain from the service chain(s) in that if auth gets kicked over under a DDoS, at least already authenticated users stand a chance of still being able to use the apps. I've also designed auth system essentially abstracted to key/value storage with adapters for differing databases (including SQLite) for deployments...
Would be interested to see how LevelDB might perform for your testing case, in that it seems to be a decent option for how your example is using data.
Except that eventually you'll find you lose a write when things go down because the page cache is write behind. So you start issuing fsync calls. Then one day you'll find yourself with a WAL and buffer pool wondering why you didn't just start with sqlite instead.
What a terrible, terrible timeline we live in now. Seriously. I genuinely hate it.