A new approach that balances agility, innovation, parity, and stability across the NATS client ecosystem
In this talk, Tomasz Pietrek, Open Source Engineering Manager at Synadia, introduces Orbit. Orbit is an umbrella project to house anything that doesn’t quite fit within the tier 1 NATS clients
By separating stable core functionality from experimental features, Orbit allows the Synadia team to maintain performance, stability and parity in core NATS clients while creating a space for innovation, language-specific implementations, and community contributions without worrying about API compatibility guarantees.
This approach solves the tension between having opinionated defaults (which don't work for all use cases) and maintaining lightweight, unopinionated core clients, with upcoming features including schema registry, key value codecs, and JetStream utilities. By enabling faster iteration on new features in Orbit before potentially merging them back into clients, Synadia aims to improve both stability and innovation across the NATS ecosystem.
"We need a way to move forward with the NATS ecosystem, with all the contributions of useful utilities that may not have full parity, but at the same time to have a very stable, performance oriented base that can work without any breaking changes. And that's how we get to Orbit"
— Tomasz Pietrek, Open Source Engineering Manager at Synadia
Go Deeper
Full Transcript
Tomasz Pietrek:
Hey, I'm Tomasz, the manager of the open source engineering team at Synadia. Let's talk about Orbit, our new way to extend NATS clients. Before we go there, I would like to tell you the story why we needed this in the first place and what challenges our clients face on a daily basis. So whenever we see a PR raise by our members or community, we often face a trade off. If it's added utility, it can be useful for many people, but at the same time it grows the complexity of the client.
If it's some new kind of an abstraction, again, this abstraction can be useful. It could bring the users closer to what they need without building their own solutions around NATS clients. But if it's language specific, it can actually cause a parity problem. And a parity is a really important thing for us because we really strive that all tier one clients, the most used ones, have parity. So whenever our users are switching between languages or using more than one language in their stack, they can be sure that all the features are available in all of the languages.
Another thing is being opinionated or not. Whenever there's new thing in the NATS server that we want to support in the clients or just in the abstraction, it's quite often the question what are the opinions? Do we have them here or not really? Like timeouts. And you can think, yeah, sure, have some opinions, some defaults, They make sense.
And that's what we thought for quite a long time too. However, with NATS growing so much and the ecosystem thriving, there are so different and vast use cases of NATS that it's really hard to find actual sensible defaults and having opinions that make sense for everybody. So every opinion that we try to land in the client, it's just a matter of time, we will regret it, and we would rather not have it there. But again, those opinions are useful without any opinions, any defaults. The clients are pretty hard to work with.
So that's another challenge we face. How to handle this? How to have opinions and at the same time be able to have a client that can work with both cars driving across places where networking can be very bad, and cloud environments when the connectivity is usually really good. This is really hard to fix in the clients by having you cannot have both. You you have to pick one, and we couldn't pick one.
Another thing is low level abstractions low level API, sorry, and abstractions on top of them. Our NATS clients used to be quite low level. You could do publish, you could do subscribe, or just stream operations, but a lot of our users just want a specific feature out of the box. A great example is key value. But again, bringing key value to the clients brings this challenge that now you have more things in the client, more complexity in the client, parity harder to maintain, and, again, opinions or not.
Both seem to be not very good. So this, until recently, all those obstructions did land in the clients, but also raised the problem of API. Because when our clients, most of them are already 1.2 version, stable version, and when we introduce new APIs, we cannot just break those APIs at will. We cannot say, yeah, part of the API is stable, part of it is new API is experimental, because the server would basically not work with it really nicely. And our users should not worry, is it new API or old API?
This is just the same version of the client. It should work according to Semver. So we landed in the place that whenever we introduced a new API, even if we marked it as experimental, like service API, key value API, we we have to keep the API even if we found problems with them, like things we could improve by breaking it and changing it. And that never made us very happy. We are happy that we could pro give you the promise of clients not having breaking changes, but at the same time, the API, of the new APIs especially, they didn't look great.
We just another problem we have to somehow solve. And again, whenever we added one of those abstractions, like key value or object store, they quite often needed some new dependencies. And our users really care about number of dependencies in the clients, and we felt that it does not look great to grow the numbers so vastly. And on the other hand, to just implement everything on our own didn't sound like a good solution either. So all of those challenges brought us to the point where we realized we need a new way to address those issues.
We need a way to move forward with the NATS ecosystem, to to all of the contributions of useful utilities that may not have full parity, but at the same time to have a very stable, performance oriented base that can work without any breaking changes. And that's how we get to Orbit. So Orbit is implemented or available in it will be available in R tier one languages, but for now it's available in Go, Rust, Java, JavaScript, and dot net, and Python and others are coming soon. And what it does, it resolves most of those conflicts I told you before by having a separate set of libraries under Orbit umbrella where you can pick one of the utilities, each separately versioned, and use it on top of the client. This way we are able to focus the clients on their own work and add extensions to the Orbit.
Let's talk about a few examples of what you can do already. Request many was one of the first things that landed in Orbit, and it landed in Orbit, not in the clients, because we thought it very simple to have request many, so basically the user can send one request and get many replies, but we very quickly realized that there are quite a few strategies you can terminate the responses. And those felt like opinions. And as we talked a few minutes ago, those opinions never fit everybody. So that's why we decided to put request many in Orbit, allowing ourselves to not only have opinions and some saying defaults defaults, but also to be able to quickly iterate over the API.
Another thing is language specific publishers. So in some languages, there are different patterns. How would you handle publishing messages in different ways or different idioms? We could never agree in a way how we do them across languages and maintain the parity, which is really important, especially for so often used API. So that's why some languages, some Orbit libraries have some publisher tools.
There are more things like SystemClient in NATS, same for the NATS context that allows you to use NATS context from the CLI. Some languages have retryers, and all of the languages will have direct batch, the new feature of 2.11, that will be iterated over. It is a client feature, but we thought it would be really good if we could iterate over the API and make it really nicely in Orbit first before we merge it back. So those are a few examples what Orbit can do already and what things didn't land in the client. And that also makes pretty clear focus of of the client versus the Orbit.
So the client will now focus on performance, stability, parity, lightweight API guarantees, being unopinionated. So it should fit most, if not all, use cases of the NATS users. We focus especially on this performance and stability and parity without adding more dependencies. With this in mind, we of course want to give you those high level abstractions, those new APIs, be able to accept your PRs with more custom ideas, and all of that will land in Orbit. We will land their new features soon, and we are very happy to accept your contributions there too.
So this splits the focus, and I think that if you remember the first slide, it it really addresses most of the issues we had with our clients by splitting those two in very distinctive responsibilities, stable and performant versus new abstractions, past opinions. Yeah. So that's basically the split. However, the Orbit has also a, hopefully bright future. Things like key value codecs should land in Orbit pretty soon.
There will be more utilities for the core NATS, for the JetStream. We are thinking about some other iterations with ObjectStore and other APIs. We will be incubating more APIs that are technically not server features they should let in the client, but we want to iterate over them first in the Orbit, and then we might merge them back in the clients when they reach the stable state. So we never need this we don't have the problem of having whenever API lands, it has to stay as that is there. Plus, as we said, more collaborations should be approved there because we allow to be not not have full parity in Orbit and also to have some discrepancies across Orbit libraries.
So that should make also a much nicer experience for our community, which is really great, and you would love to approve more of PRs. And that's basically it about Orbit, and about clients, because it shifts the client's focus heavily too. And you can find the links here. Probably the best way is to look at synadia..org and just search through Orbit to see all the libraries. But you have links here for all the five available already.
As said, Python and others will come in soon. So that's about Orbit. If you have any questions, I'll be hanging around here. Thank you very much.
Nate Emerson:
Excellent. And here's Tomasz. Super excited to hear about Orbit, and I think it's just a really cool model for iterating and innovating more quickly on some of the client libraries while keeping those core client libraries really stable and keeping rock solid parity between them. So we had a couple questions coming through. What first of all, what's the next big capability coming to Orbit?
And we have a related poll that we're gonna launch at the same time. Huge shout out to everyone answering the polls. The participation there has been awesome. I think people want some t shirts. But we've got another one on just kinda which of these features are you most looking forward to with Orbit's kind of improvements and cutting edge additions?
And, also, there's a short answer there so you can fill in something if you've got an idea or something particular that you're excited about.
Tomasz Pietrek:
Yeah. So I think that the first thing it's it's interesting question what will land what the biggest thing will land because we'll talk about it a bit later today. And the big thing you saw the list of items that we will put in soon. Those are quite smaller on on purpose. The big one that will land hopefully soon is schema registry.
So, yeah, I know that many people wait for it. We also are really happy to finally work on it and have a place where we can also iterate about and and work over it because it was also facing the challenges I was talking about. So that's I think that's the the the biggest one that we'll talk a bit later. I'm sure that Byron will talk about it too.
Nate Emerson:
Yeah. And then somebody was asking if Orbit's ready for Jetstream, which you can speak a bit to that. But, yeah, I believe the core libraries handle Jetstream. There's more features that Orbit's going to unlock, basically, for interacting with Yes. Materialized views.
Tomasz Pietrek:
Orbit is mostly built on top of the client, so it can have extensions in both core NATS and in Jetstream or things on top of core NATS and Jetstream. So yeah, it's definitely ready for Jetstream. When we see some One of the great examples that are already there retrying is quite often in Jetstream. It's tricky and very opinionated. We couldn't put proper retry strategies in the clients because it's so different for banking and so different for cars that it's impossible to have one solution.
And those are the things that will land there on top of the standard JST to make it like quality of life improvements for people.
Nate Emerson:
And then we've got one person who's asking about Orbit for C. So is there anybody spearheading Orbit for the c library?
Tomasz Pietrek:
Yeah. It it will come. Hopefully, pretty soon. We we haven't started yet, but for for the request many and others, we we should have it pretty soon. I don't give you a promise for a specific date, but it's not not ignored or or or or whatever.
It's it just waits for its time, hopefully soon. We just mostly have to, for the c, do the scaffolding for how to build libraries on top of clients to make it properly compile. It it's mostly toolchain challenge, not not work challenge, how to figure it out properly.
Nate Emerson:
Right. And then there was a choice three in that poll. I believe that was my mistake setting up the poll. So apologies for that, but I like the people are interested in Door number three. And then one one question from chat.
When you use Orbit over the client, it's I don't think it's quite exclusive that you would have to use Orbit over the core client. Right?
Tomasz Pietrek:
No. You use you use the client and then it is a bit different on language basis because we try to be idiomatic for each language. Like, in in some language, like in Rust, it can be basically an extension to the client. So you load the EXT trade, and it's suddenly you have all those additional capabilities of Orbit. And with any other languages, it'd be kind of a bit different.
But it's not exclusive, definitely. It's extending. Though, of course, there will be if you will build abstractions like schema registry or the Keybody codecs, they will you don't will not need to directly load the client.
Nate Emerson:
Right. And here's a longer question that just came in, but is a bit interesting. So Pedro is asking about test suite for the NATS clients. He said, in my experience, writing a comprehensive test suite for a NATS client is very difficult when you take into account network disconnections, reconnection, server pool, error handling, incorrect timings. Doesn't matter how many tests you've written, you still feel there's a forgotten corner case.
Did you ever look in any kind of method or framework that helps put the numerous test cases in order and kind of prove out corner cases or some proxy for coverage?
Tomasz Pietrek:
Yeah. That's interesting topic that could probably take over the the whole ref income if we talk it out at length. I'll just cut it as short as possible, which means that yeah. We have in the clients those other tests. You see those are mostly testing if the features work correctly.
And they do test error cases, not just heavy puffs, but they don't test network problems, packet loss, dying servers and clusters. For that, we have a separate test which are not yet public, where we run a lot of chaos testing overnight, overnight runs. Plus, we have the anti testis tooling that you can read about on the blog. It's also linked in the 2.1 release where Marco writes about how we leverage anti test as a tool that does allow for a great way to deterministically reproduce very one in a million bucks. So there are many layers of this.
What you see in the tests, in the clients, is just the basic layer to ensure that the clients work properly in the right network and host conditions.
Nate Emerson:
Alright. Well, thank you so much for joining us, Tomasz. Really appreciate
Tomasz Pietrek:
Thank you very
Nate Emerson:
that. And, again, there's some links in the chat right now if you wanna check out Orbit. The Synadia blog had a recent post on Orbit, and that includes language links for all of the languages. And Zia also included a query on GitHub that'll give you all of the repos right there in GitHub.
