Rendered at 21:51:26 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gk1 1 days ago [-]
Folks, in your launch posts remember to describe what you're launching and what it's for. Ideally in the opening paragraph.
- Intro: Explains problem but not what is Neki or what it's for.
- Why Neki: Explains alternatives and why they suck but not what is Neki or what it's for.
- How does Neki work: Describes the technical components but not what is Neki or what it's for.
- What you get beyond sharding: Explains how to run/deploy but not what is Neki or what it's for.
- What is a platform preview + Try Neki today: 140 words on the definition of "preview" and links to get started... but not what is Neki or what it's for.
Edit: The landing page has 100x more useful information up front -> https://neki.dev/
Edit #2: They've since added a "What is Neki" section. Good reaction time.
DrammBA 1 days ago [-]
Folks, your blog post sucks at marketing.
Edit: your marketing site actually has excellent marketing information.
Edit #2: They've since replaced the blog post with a link to the marketing page. Good reaction time.
ahoka 14 hours ago [-]
I scrolled through the landing page and even the "Get started" page and I have still no idea what is this. Is this a managed db? Something I can run myself? No idea.
1 days ago [-]
dwb 1 days ago [-]
The landing page which is linked at the start of the second sentence of this page…? I don’t completely disagree with you, but this is mostly hypertext working as intended, is it not?
fastball 1 days ago [-]
Yeah this meta commentary on HN is very confusing for me. The structure of the article is literally
- short paragraph saying "we launched neki"
- short paragraph saying why neki was needed
- paragraphs about what neki is (with a header)
That seems like a very reasonable structure for a blog post like this one. Not being able to get to the third paragraph of a blog post seems like a "you" problem, not a blog problem.
crooked-v 18 hours ago [-]
The "what is" section didn't exist when they commented.
simonw 21 hours ago [-]
Sadly, in 2026, you should never assume anyone will click a link.
entrope 15 hours ago [-]
Why "sadly"? The person writing a link should provide enough detail for readers to figure out whether they should follow the link. I rolled my eyes the first time I saw this entry on HN because it just said "Neki" and the (generic) target domain name. A two-word description got added, and that makes it interesting instead.
simonw 9 hours ago [-]
It's sad because I think links are the best thing about the web, and it's increasingly clear to me that many internet users either misunderstand or actively avoid them.
Why do people post screenshots of an article on social media instead of linking to it?
Sometimes ignorance, but often because the web is such a hostile experience these days - paywalls and subscribe banners and cookie warnings and slow pages.
That's sad.
owebmaster 9 hours ago [-]
If that's what you think, why you keep posting comments linking to your blog?
If you know that to be true and still add your links everywhere ("helping"), isn't this a spammer's confession?
simonw 8 hours ago [-]
Those are for the people who still respect links.
Atreiden 1 days ago [-]
The top concern I've gotten from dev teams when proposing HA distributed postgres (e.g through RDS Aurora global) is that eventual consistency is not suitable for many workloads.
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
mcpherrinm 1 days ago [-]
Assuming it works the same way as their MySQL product, Vitess:
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
ronfriedhaber 1 days ago [-]
Therefore, Consistency guarantees are based on implementing application-specific partitioning (correctly)?
osalberger 5 hours ago [-]
The consistency guarentee assuming repeatable read or higher on each shard is parallel snapshot isolation, which is technically still higher than postgres read committed.
If you have serializable on each shard and you chop transactions so you have no cross shard transactions, the application is technically still serializable, it just behaves like two independent deployments of the application for different shards
Same here, first thing I need to know when considering a distributed system is how consistency is handled. If it's eventual consistency, what is the replication lag like? If it's strong consistency, can their network handle that? What happens when a node goes down?
harshitgangal 1 days ago [-]
[dead]
fsuts 23 hours ago [-]
Kinda ironic that Planetscale built their company on open source Vitess that google built
And now planetscale have made a Vitess like equivalent for Postgres which they have made proprietary
hiyer 21 hours ago [-]
They have made it proprietary so that providers like AWS can't build their own service with it, thus eating into PS's income.
Google is big enough to survive open-sourcing Vitess, k8s, angular, etc. Planetscale isn't.
samlambert 20 hours ago [-]
correct. AWS do anti-competitive stuff to us all the time we can't make it any easier for them.
Onavo 5 hours ago [-]
What a convenient excuse
figmert 16 hours ago [-]
Google, Azure, and AWS already have their highly available PostgreSQL alternatives.
ksec 7 hours ago [-]
>Kinda ironic that Planetscale built their company on open source Vitess that google built
That Youtube built. I know at the time Youtube was already part of Google but I don't think of Vitess as a Google Stack.
xyzzy_plugh 1 days ago [-]
I love how the Planetscale CEO has been shitting on multigres[0], Supabase's equivalent to Neki, and gloating about how superior Neki is, and yet Neki is not even open source!
Sam, it feels like you're just playing games. It's hard to take anything you say or do seriously. Are you going to OSS this thing? Or have you decided to backpedal?
Congrats on the checks notes closed launch, I guess?
Just launch the product and let it speak for itself. The constant need to take shots at competitors comes across as pretty childish.
vincnetas 17 hours ago [-]
Wow, that thread has bro energy maxed out. Is this normal modern way of communicating for top leaders or i'm getting old?
tpetry 15 hours ago [-]
Sam is always shitting on others. For years he has been screaming how bad PostgreSQL is compared to MySQL. Until getting silent about this topic for a few months and then launching PostgreSQL support. Seems he had to realize than the market for PostgreSQL is too big to ignore.
pm90 15 hours ago [-]
No it isn’t normal, its just this one person. I don’t personally find this language particularly appealing, especially for a serious discussion on technology. But hey, its a free country, they’re a startup and its one way of grabbing the scarcest of resources: attention.
samlambert 1 days ago [-]
How does open source have anything to do with Neki's ability to scale against a service that doesn't even do sharding.
margalabargala 1 days ago [-]
They both are characteristics of software that people consider when deciding whether to use it.
Open source being by far the more important of the two, since it means the user is more insulated by terrible corporate decisions thus making it more suitable for long term planning.
samlambert 1 days ago [-]
have you ever worked for a customer of AWS? If so you have contributed heavily to the destruction of commercial open source.
tvink 1 days ago [-]
I've never known in a shorter time that a CEO is too defensive to be a business partner faster.
therein 1 days ago [-]
It is the only reason I come to the comments section of PlanetScale posts on HN.
Their CEO always immediately comes in and makes a fool out of himself. He is what you get when you mix narcissism and being a midwit.
margalabargala 1 days ago [-]
I have not in fact, no, despite working 12 years as an SWE.
Unlike you, probably, if we're slinging unfounded accusations.
xyzzy_plugh 1 days ago [-]
But your customers are, of course, somehow different? What point are you even trying to make?
Commerical open source is fine. It's nobody else's fault if you raised too much to make it work sustainably.
samlambert 1 days ago [-]
My customers aren't pretending it's a bad thing that Neki isn't open source.
cschmatzler 1 days ago [-]
A year ago in the initial announcement post you explicitly mentioned open sourcing it.
> When we are ready we will release Neki as an open source project suitable for running the most demanding Postgres workloads.
Acknowledging the benifits of keeping a commercial product closed source is one thing, claiming open source has no value is another thing.
Sad to see this coming from a company that was itself built on the foundation of an open-source project like Vitess.
samlambert 1 days ago [-]
who said open source has no value? I worked at GitHub for 6 years i clearly love open source.
We maintain Vitess using our company funds. Why are you going after a small company for defending itself against companies like Amazon but not criticizing the reason we have to do this.
I would love to open source Neki and I have not said we won't.
xyzzy_plugh 1 days ago [-]
See? You're proving my point. All you do is play games. Are you even capable of engaging without confrontation?
xyzzy_plugh 1 days ago [-]
You've simply chosen not to listen, considering that they've voiced those concerns to me.
fwip 1 days ago [-]
modeling your public relations strategy after matt mullenweg, I see
Twirrim 1 days ago [-]
It's very effective, I'm certainly persuaded to never touch their products.
A remarkable own goal given that if they'd literally held their tongue, I'd be at worst indifferent.
samlambert 8 hours ago [-]
People think comments like this really bother me as if I believe that I've missed out on any dollars at all from you, as if you have the buying power or ability to drive any workloads that would in any way move the needle on my revenue.
I want to let you know that I have customers that pay PlanetScale millions of dollars and have told me they came to the product because of my Twitter account. So you will never make me feel bad about this.
fwip 5 hours ago [-]
Sure, Jan.
kolanos 24 hours ago [-]
Do they get the same backlash if they went with a source available license similar to MongoDB or Elastic?
Yannik_Sc 14 hours ago [-]
I love the offer which is running on the "PlanetScale Metal".
They claim to provide unlimited IOPS (I/O operations Per Second).
I would like to get some of these drives myself. Sounds like a technical miracle.
On the other hand claiming such miracles makes me wonder which other aspects in this are are actually a working miracle or rather a mythos/false marketing claim.
cobbzilla 11 hours ago [-]
> unlimited IOPS
I think this is marketroid for “we don’t limit them” not a statement about physics
kelp 7 hours ago [-]
Correct.
__s 8 hours ago [-]
m8gd local nvme
hanspagel 1 days ago [-]
I need this for my side project that has zero users, but I wonder if Neki will scale well enough.
ehfeng 1 days ago [-]
have you tried simultaneously vertically and horizontally scaling?
hanspagel 1 days ago [-]
My plan is to automatically boot a new Kubernetes cluster for every db query.
I wonder how the performance will be, but I can just rewrite everything in Rust.
Tepix 16 hours ago [-]
Make it serverless.
erulabs 1 days ago [-]
I love to see this but as a heavy Vitess/MySQL user I do fear the split focus from PlanetScale. Hoping to see continued improvements on the Vitess side as well.
Selfish doubts aside, congrats to Planetscale on the launch!
imjosh-dev 1 days ago [-]
don't worry, we still give a lot of love to vitess. at the end of the day they are both databases. the beauty of doing both is we can take learnings from each product and apply it to the other.
vt_throwaway 1 days ago [-]
[dead]
dvtkrlbs 1 days ago [-]
Is this going to be open sourced? I thought this was the plan at the start. But ow I am confused.
khy 1 days ago [-]
Man, I was deep into this blog post before I was able to answer the question "what is Neki?".
If I don't know what I'm reading about by the third or fourth phrase, I skip it. Empty language comes from empty minds and I don't have time for that.
kaycebasques 1 days ago [-]
I am on a 12 Muni right now that is plastered with a Neki ad, and I looked up neki.dev literally 10 mins ago to see what this product is about. Did they time the release of this bus ad with the launch date?
kelp 1 days ago [-]
Bus ads have been running for a little while now. A few billboards around too.
Tepix 16 hours ago [-]
So apparently the neki router uses many parts of Postgres. But is it open source itself?
Apparently not.
kelp 7 hours ago [-]
The neki router shares no code with Postgres.
However we have a whole team working to improve Postgres and we always prefer to contribute those efforts to the Postgres community.
ripe 1 days ago [-]
Neki is a sharded Postgres implementation. It introduces one level of indirection between the application and the Postgres shards.
Found this from neki.dev.
fastball 1 days ago [-]
You can also find that in first section of the linked post (after the intro)...
eile23 17 hours ago [-]
How are cross-shard joins and transactions handled? Are there query patterns that would require application changes despite using the same drivers and ORM?
imjosh-dev 4 hours ago [-]
cross shard joins break down into single shard selects, and are joined in the routers. There are some query patterns that are currently unsupported, but we are working on adding support for them. The biggest thing to keep in mind when running on a sharded database of any kind is decreasing the number of cross shard queries.
klaussilveira 8 hours ago [-]
How does it deal with big tables (8B+ rows)? Does it split the table onto multiple shards or each shard only gets full tables?
> This adds some undesirable side effects, however. First, the coordinator becomes the bottleneck.
Apart from being able to use any node as a coordinator (and you can load balance them to avoid having "multiple connection strings), there's a new pattern which effectively allows you to have as many coordinators as you want. They are effectively "data-less" nodes. We have devised and implemented this pattern in StackGres [1].
> Adding a shard with more resources for a noisy tenant, or many small shards for a wide shard space requires substantial manual configuration of not only the database servers themselves, but wiring them up together with Citus.
Adding nodes (infrastructure) is what operators solve. In StackGres, adding new nodes means editing one/two characters from your YAML file: the integer number that represents the number of workers that you have.
Shard rebalancing is fully built-into Citus as a UDF, which you can call (manually or in an automated manner) over Postgres protocol.
> Managing backups is also external to Citus, so operators still need to build the proper infrastructure
Agreed, but it's also solved (see distributed backups in StackGres [2]).
> PgDog is a spiritual successor to PgCat, both of which improve on Citus's architecture substantially.
Unsubstantiated why. I assume it's because of the assumption that a proxy model is superior than Citus. To which I have to say that Citus model is also a proxy model, where the proxy just happens to be Postgres, which unsurprisingly, speaks Postgres protocol. Sure, there are nuances that we could debate in this area and we can say that Citus is not a "pure proxy", but that doesn't lead to concluding that a proxy model is better --it's arguably not.
A C2 level diagram included in the blog post would be helpful. What are the drawbacks to using Neki aside from the obvious (new software has bugs)?
arrowleaf 1 days ago [-]
The first thing I want to know when looking at a distributed system is how consistency is handled, ctrl+f for "consis" on both the blog post and neki.dev has zero hits. So I can only assume it will have eventual consistency with a considerable replica lag for real-time systems.
kamikazechaser 15 hours ago [-]
Not even freeware/shareware?
ragall 1 days ago [-]
I see no mention of foreign keys, or any other constraints, across shards. If, as I suspect, they're not implemented, it would still be useful but at the level of Spanner 10 years ago.
aabhay 1 days ago [-]
Cross shard transactions are “coming soon”. So, this is just like having multiple postgres clusters.
spongeboi 1 days ago [-]
Cross shard transactions are supported, but we don't have support for atomic 2pc cross shard transactions, its WIP
levkk 22 hours ago [-]
Curious to compare your implementation to ours [1] some day (if you guys open source it).
Are you one of the developers ? If so, you need to add a specific page explaining in detail the guarantees that this gives.
For example: point 08 says "Assign different tables or workloads to different shard groups" and point 02 says "Split hot shards as workloads grow". How do those interact ? Can a single table be split across multiple shards ? If so don't you need 2pc to enforce primary key constraints ?
The intro pages just read as AI slop.
vira28 1 days ago [-]
Obviously welcome more tools for Postgres and sharding but it’s a tough hill to climb against the hyper clouds. Best wishes to the team.
samlambert 1 days ago [-]
The largest database to ever run on Aurora MySQL runs on PlanetScale, and the largest database to ever run on Aurora limitless also runs on PlanetScale.
OrangeDelonge 18 hours ago [-]
What do you mean? Your comment reads like PlanetScale uses Aurora under the hood, you mean PlanetScale can handle bigger DB’s than Aurora?
theanirudh 18 hours ago [-]
He means that they moved from AWS to Planetscale and yes it handles much larger DBs.
tedd4u 1 days ago [-]
We went as far as we could on GCP before PlanetScale. Spanner is amazing but also amazingly expensive. And CloudSQL, also amazing as long as you don’t need write scaling. PlanetScale (Vitess+MySQL plus their branching / deployment and monitoring tools) is just not a combination offered on GCP. And while I didn’t use Aurora, from talking to and exploring AWS it doesn’t really have this combination either.
samlambert 21 hours ago [-]
It makes me so happy to hear you had a good experience. Thank you for being a customer.
phoenix24 1 days ago [-]
is this open-sourced?
Hasz 1 days ago [-]
Independent of the product, love the branding design for this page.
xnx 1 days ago [-]
What does this compete with? Google Cloud SQL?
jbonatakis 1 days ago [-]
Multigres is the one that comes to mind
kelp 1 days ago [-]
Once they implement sharding...
jbonatakis 1 days ago [-]
Yet. But it’s built by the guy who created Vitess and cofounded PlanetScale, so while it doesn’t have sharding implemented at the moment I think it would be foolish to discount it. Plus it’s open source.
kelp 1 days ago [-]
The PlanetScale team spent years improving Vitess with lessons learned from many different high scale customer workloads. We continue to invest in and improve Vitess. It's a substantial and growing part of our business.
Vitess today is not the Vitess of 2021. About 70% of the current codebase was written in the last five years, based on lessons we learned running it at scale.
Neki is built for Postgres from the ground up, by the engineers who did that work.
On the Multigres comparison, I'll admit we get a bit salty, mostly because "Vitess for Postgres" invites the comparison. But to date there's no public evidence of a working sharding implementation in Multigres. Their sharding design doc landed this week, and it lists cross-shard query planning and resharding execution as explicitly out of scope.
Neki does sharding today, including online resharding that switches traffic without a maintenance window. It isn't open source, but you can spin up a cluster and try it yourself.
uberman123 1 days ago [-]
Sugu? The guy that "barely worked" and "shipped nothing at PlanetScale" according to Planetscale's CEO?
karlmush 1 days ago [-]
Anyone who has worked with Sugu knows this simply isn’t true. Sam Lambert is lying.
samlambert 21 hours ago [-]
every metric we have proves otherwise
samlambert 1 days ago [-]
We literally have the counterfactual of the old product and company before he left. We can fully discount Multigres doing anything.
1 days ago [-]
karuppiah7890 1 days ago [-]
It's not Open Source Software. So, I'm not even interested. Lol
samlambert 1 days ago [-]
lol are you an amazon customer?
23 hours ago [-]
overfeed 1 days ago [-]
lol are you an amazon wannabe?
samlambert 21 hours ago [-]
yes
rs_rs_rs_rs_rs 1 days ago [-]
Will this be open source at some point?
samlambert 20 hours ago [-]
undecided right now
ezekg 8 hours ago [-]
I assume it's already on your radar, but look into fair source: https://fair.io.
_1tan 1 days ago [-]
Cool, but we're on MySQL - anything similar available? Percona XtraDB?
tedd4u 1 days ago [-]
That is what the “regular” PlanetScale is all about. It uses the Vitess open source project to do write scaling through shading. It is very powerful and customizable and stable. It was built to run YouTube. Former happy user here.
IceWreck 1 days ago [-]
Vitess/Planetscale?
Neki is essentially a version of Vitess for Postgres fom the same guys.
kelp 1 days ago [-]
Many of the engineers who built Neki also worked on Vitess for years. But it's a ground up new implementation designed to specifically target Postgres.
vt_throwaway 1 days ago [-]
[dead]
dveeden2 16 hours ago [-]
TiDB: Distributed and MySQL compatible
MrDrMcCoy 18 hours ago [-]
MariaDB Galera or Vitess.
manlymuppet 21 hours ago [-]
I think this will become the new standard.
behole 1 days ago [-]
Site design is tight. I like it.
samlambert 20 hours ago [-]
thank you!
xyst 1 days ago [-]
only planet scale? Rookie numbers.
I am looking for interdimensional and interuniversal scale. Which of you trust fund babies has a startup which is working on this problem?
samlambert 21 hours ago [-]
i have been struggling to hire enough physicists
_kidlike 1 days ago [-]
can we choose read-after-write consistency at query time, if needed?
parthdesai 1 days ago [-]
The tech might be amazing, but the CEO comes across as an asshole
tonyhb 1 days ago [-]
Sam is not an asshole. His twitter is great, he's definitely opinionated, and if you haven't had the luck of meeting him you might misread his tone as grumpy. It's not. It's sarcastic/dry humour and honest.
They build good stuff and care.
mbesto 1 days ago [-]
You can build good stuff, care, and be very opinionated...you can also be all of those things and still be an asshole.
buremba 21 hours ago [-]
being opinionated doesn't mean you can attack people you compete or don't like every other week
browningstreet 23 hours ago [-]
His X.com persona is def that of an asshole.
samlambert 22 hours ago [-]
very kind of you to say tony
cwisecarver 1 days ago [-]
He's actually quite nice over zoom so he just comes off as funny on twitter once you've met him.
JeremyNT 23 hours ago [-]
Perhaps he's quite nice over zoom, but since very few of us are going to meet him in that context, the rest of us are left to judge him by what he posts. And as they say, you never get a second chance to make a first impression.
Based on what I've read so far, with his shit-talking the competition, he seems like... a bit of an asshole.
overfeed 1 days ago [-]
> he just comes off as funny on twitter once you've met him.
Never heard of the guy before today, but his dickish, edge-lord aura (on X and in this thread) makes me not want to meet him. If I had to choose between buying from him, or some other random dude, I'll hear out the random dude first.
tgma 20 hours ago [-]
> If I had to choose between buying from him, or some other random dude, I'll hear out the random dude first.
If you have such luxuries, I bet you don't have enough scale to be really profitable for him.
overfeed 19 hours ago [-]
Oh no, how am I ever going to avoid giving my custom to someone who has allegedly cornered the market on - checks notes - a db proxy? Give me a break.
I can think of at least 4 other ways of scaling horizontally without switching DB engines. I'd have to be mightily constrained to have this particular product as the only option.
tgma 17 hours ago [-]
Sure, I don't doubt your ability to do so yourself. I just question your ability to trust some random unknown figure with your data if you have enough scale. That's quite different, actually.
overfeed 1 hours ago [-]
TBF, I hadn't heard of the guy before this thread either, but I don't hang out in the unexpected-scaling-demand-with-no-infra-team circles, which appears to he the target market.
I can fully understand why someone would think that.
parthdesai 1 days ago [-]
There's obviously more to the story than outsiders are aware, but personally it just leaves a bit of sour taste in my mouth when the tech and blog posts you guys put out are amazing.
(I'm a paying PS customer)
tombert 1 days ago [-]
Genuine question, since I'm a little out of the loop; why do you say that?
nothing i have said is untrue. a founder of a company leaving to compete out of greed is a bad thing and i am glad to call it out.
mbesto 1 days ago [-]
Did anyone say you said anything untruthful? No, they just called you an asshole. You can both be an asshole and be right.
PS - Just some advice that I learned in sales 101 early in my career...focusing on shit talking the competition generally isn't a strategy that works well.
omnicognate 1 days ago [-]
> You can both be an asshole and be right.
"You're not wrong Walter, you're just an asshole." - The Dude
prisonguard 1 days ago [-]
Unless you are at the basketball court where talking smack is a valid winnign strategy
ragall 1 days ago [-]
> You can both be an asshole and be right.
Only petty people can hold that view.
margalabargala 1 days ago [-]
In case you were unaware, if someone's defense against being called an asshole is to say "well nothing I said was false" they're almost certainly an asshole.
No one accused you of dishonesty.
nateberkopec 21 hours ago [-]
I, for one, like it when CEOs let loose a bit and show some personality. It's better than everyone walking around talking like a media trained robot.
19 hours ago [-]
halilBB 13 hours ago [-]
[flagged]
arpinum 1 days ago [-]
[dead]
fatata123 11 hours ago [-]
[dead]
nicechianti 1 days ago [-]
[dead]
vaibhav2614 1 days ago [-]
Pretty exciting to see a sharded Postgres product in the market.
The pitch is compelling. I wonder how many teams will be able to operate sharded database setups in production as a result of this.
- Intro: Explains problem but not what is Neki or what it's for.
- Why Neki: Explains alternatives and why they suck but not what is Neki or what it's for.
- How does Neki work: Describes the technical components but not what is Neki or what it's for.
- What you get beyond sharding: Explains how to run/deploy but not what is Neki or what it's for.
- What is a platform preview + Try Neki today: 140 words on the definition of "preview" and links to get started... but not what is Neki or what it's for.
Edit: The landing page has 100x more useful information up front -> https://neki.dev/
Edit #2: They've since added a "What is Neki" section. Good reaction time.
Edit: your marketing site actually has excellent marketing information.
Edit #2: They've since replaced the blog post with a link to the marketing page. Good reaction time.
- short paragraph saying "we launched neki"
- short paragraph saying why neki was needed
- paragraphs about what neki is (with a header)
That seems like a very reasonable structure for a blog post like this one. Not being able to get to the third paragraph of a blog post seems like a "you" problem, not a blog problem.
Why do people post screenshots of an article on social media instead of linking to it?
Sometimes ignorance, but often because the web is such a hostile experience these days - paywalls and subscribe banners and cookie warnings and slow pages.
That's sad.
If you know that to be true and still add your links everywhere ("helping"), isn't this a spammer's confession?
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
If you have serializable on each shard and you chop transactions so you have no cross shard transactions, the application is technically still serializable, it just behaves like two independent deployments of the application for different shards
And now planetscale have made a Vitess like equivalent for Postgres which they have made proprietary
Google is big enough to survive open-sourcing Vitess, k8s, angular, etc. Planetscale isn't.
That Youtube built. I know at the time Youtube was already part of Google but I don't think of Vitess as a Google Stack.
Sam, it feels like you're just playing games. It's hard to take anything you say or do seriously. Are you going to OSS this thing? Or have you decided to backpedal?
Congrats on the checks notes closed launch, I guess?
0: https://github.com/multigres/multigres
Just launch the product and let it speak for itself. The constant need to take shots at competitors comes across as pretty childish.
Open source being by far the more important of the two, since it means the user is more insulated by terrible corporate decisions thus making it more suitable for long term planning.
Their CEO always immediately comes in and makes a fool out of himself. He is what you get when you mix narcissism and being a midwit.
Unlike you, probably, if we're slinging unfounded accusations.
Commerical open source is fine. It's nobody else's fault if you raised too much to make it work sustainably.
> When we are ready we will release Neki as an open source project suitable for running the most demanding Postgres workloads.
Has this changed?
https://web.archive.org/web/20260516145550/https://planetsca...
https://web.archive.org/web/20260910204345/https://planetsca...
Sad to see this coming from a company that was itself built on the foundation of an open-source project like Vitess.
We maintain Vitess using our company funds. Why are you going after a small company for defending itself against companies like Amazon but not criticizing the reason we have to do this.
I would love to open source Neki and I have not said we won't.
A remarkable own goal given that if they'd literally held their tongue, I'd be at worst indifferent.
I want to let you know that I have customers that pay PlanetScale millions of dollars and have told me they came to the product because of my Twitter account. So you will never make me feel bad about this.
They claim to provide unlimited IOPS (I/O operations Per Second).
I would like to get some of these drives myself. Sounds like a technical miracle.
On the other hand claiming such miracles makes me wonder which other aspects in this are are actually a working miracle or rather a mythos/false marketing claim.
I think this is marketroid for “we don’t limit them” not a statement about physics
I wonder how the performance will be, but I can just rewrite everything in Rust.
Selfish doubts aside, congrats to Planetscale on the launch!
Apparently not.
However we have a whole team working to improve Postgres and we always prefer to contribute those efforts to the Postgres community.
Found this from neki.dev.
- PgDog is open source
- It runs anywhere, not just your cloud
- It's built for Postgres, not a MySQL pivot
> This adds some undesirable side effects, however. First, the coordinator becomes the bottleneck.
Apart from being able to use any node as a coordinator (and you can load balance them to avoid having "multiple connection strings), there's a new pattern which effectively allows you to have as many coordinators as you want. They are effectively "data-less" nodes. We have devised and implemented this pattern in StackGres [1].
> Adding a shard with more resources for a noisy tenant, or many small shards for a wide shard space requires substantial manual configuration of not only the database servers themselves, but wiring them up together with Citus.
Adding nodes (infrastructure) is what operators solve. In StackGres, adding new nodes means editing one/two characters from your YAML file: the integer number that represents the number of workers that you have.
Shard rebalancing is fully built-into Citus as a UDF, which you can call (manually or in an automated manner) over Postgres protocol.
> Managing backups is also external to Citus, so operators still need to build the proper infrastructure
Agreed, but it's also solved (see distributed backups in StackGres [2]).
> PgDog is a spiritual successor to PgCat, both of which improve on Citus's architecture substantially.
Unsubstantiated why. I assume it's because of the assumption that a proxy model is superior than Citus. To which I have to say that Citus model is also a proxy model, where the proxy just happens to be Postgres, which unsurprisingly, speaks Postgres protocol. Sure, there are nuances that we could debate in this area and we can say that Citus is not a "pure proxy", but that doesn't lead to concluding that a proxy model is better --it's arguably not.
[1]: https://stackgres.io/doc/latest/administration/sharded-clust...
[2]: https://stackgres.io/doc/latest/administration/sharded-clust...
[1]: https://docs.pgdog.dev/features/sharding/2pc/
For example: point 08 says "Assign different tables or workloads to different shard groups" and point 02 says "Split hot shards as workloads grow". How do those interact ? Can a single table be split across multiple shards ? If so don't you need 2pc to enforce primary key constraints ?
The intro pages just read as AI slop.
Vitess today is not the Vitess of 2021. About 70% of the current codebase was written in the last five years, based on lessons we learned running it at scale.
Neki is built for Postgres from the ground up, by the engineers who did that work.
On the Multigres comparison, I'll admit we get a bit salty, mostly because "Vitess for Postgres" invites the comparison. But to date there's no public evidence of a working sharding implementation in Multigres. Their sharding design doc landed this week, and it lists cross-shard query planning and resharding execution as explicitly out of scope.
Neki does sharding today, including online resharding that switches traffic without a maintenance window. It isn't open source, but you can spin up a cluster and try it yourself.
I am looking for interdimensional and interuniversal scale. Which of you trust fund babies has a startup which is working on this problem?
They build good stuff and care.
Based on what I've read so far, with his shit-talking the competition, he seems like... a bit of an asshole.
Never heard of the guy before today, but his dickish, edge-lord aura (on X and in this thread) makes me not want to meet him. If I had to choose between buying from him, or some other random dude, I'll hear out the random dude first.
If you have such luxuries, I bet you don't have enough scale to be really profitable for him.
I can think of at least 4 other ways of scaling horizontally without switching DB engines. I'd have to be mightily constrained to have this particular product as the only option.
(I'm a paying PS customer)
https://x.com/samlambert/status/2098075451373834681?s=20
PS - Just some advice that I learned in sales 101 early in my career...focusing on shit talking the competition generally isn't a strategy that works well.
"You're not wrong Walter, you're just an asshole." - The Dude
Only petty people can hold that view.
No one accused you of dishonesty.
The pitch is compelling. I wonder how many teams will be able to operate sharded database setups in production as a result of this.