Orbit: NATS Extensions and the Future of the Clients

Extending the Clients vs keeping them simple

Each time we receive a pull request for a NATS client, we face a trade-off: enhancing the specific client often leads to cascading impacts on other libraries and increases overall API complexity and parity challenges, creating tension between improvement and simplicity. Inevitably, once we add a new feature to one client, requests to add the same feature to all the others follow.

It’s unfortunate to reject useful functionality and potentially discourage community engagement, but it’s also unreasonable to let complexity and parity challenges grow unchecked. There is an additional argument that each client will be used differently too and maybe not all clients need to do all the things. An example would be how Python is used heavily with AI and ML, which might skew the NATS requirements. We felt that we couldn’t win this battle — no matter what we decided, whether we add or ignore new APIs and features, we always seemed to be losing something.

This is where Orbit enters the scene.

Orbit

By creating an umbrella project to house anything that doesn’t quite fit within the client, we address most of the issues outlined above. For example, a pull request adding a useful feature to the Go client that doesn’t make sense for other libraries can be included as an Orbit extension. Orbit makes no promises of parity across languages.

A new API can live in Orbit with its own versioning, allowing for rapid development and improvement rather than quick stagnation and the clients remain minimalistic, stable and performant with feature parity.

This approach draws a clear line between what belongs in the client and what belongs in Orbit:

Client

Focuses on stability, performance, and reliability
Remains as un-opinionated as possible, supporting a wide range of use cases
Minimizes dependencies

Orbit

Hosts new experimental APIs
Contains language-specific extensions to the clients
Provides opinionated abstractions and patterns

With responsibilities divided this way, the NATS client ecosystem can grow with fewer compromises and greater flexibility. Orbit can also be used as a staging ground for candidate client features.

Request Many

The first common extension in Orbit is "request many". Our users are accustomed to two main patterns in Core NATS: pub/sub and request/reply. However, there are many situations where you might need multiple responses from a single request.

You can achieve that by setting up subscriptions and sending requests with the reply subject matching the subscription subject, but there is quite a bit of nuance to decide when the responses are all done, how to handle timeouts, etc. This is why we decided to provide an opinionated extension that will allow users to focus on their business logic, rather than on the plumbing and figuring out how to implement the patterns.

Let's walk through some of them.

Chunked responses

When the response might be too heavy to fit into a single message, you might want to chunk the response into multiple messages and then have some way to terminate the subscription when all chunks are received.

Chunked responses

There are few ways to achieve that goal.

Max messages - a simple approach, where the requester knows the exact number of responses to receive.

1
msgs, err := natsext.RequestMany(nc, "subject", []byte("request data"), RequestManyMaxMessages(50))
2
if err != nil {
3
    // handle error
4
}
5
for msg, err := range msgs {
6
    if err != nil {
7
        // handle error
8
    }
9
    fmt.Println(string(msg.Data))
10
}

Sentinel - when the number of responses is unknown, but the responder knows when it’s done, sentinel message can be used. Sentinel is the last messages that fulfils the predicate. Most common one is simply an empty message.

1
use nats_extra::request_many::RequestManyExt;
2
let client = async_nats::connect("demo.nats.io").await?;
3

4
let responses = client
5
    .request_many()
6
    .sentinel(|msg| msg.payload.is_empty())
7
    .send("subject", "payload".into())
8
    .await?;

Discovery

You may not always know how many responses to expect, so max_messages might not be suitable. You might also need to gather responses from an unknown number of responders, perhaps to determine how many are available. In this case, you can use stall_timeout, which waits a maximum amount of time between messages and stops if that threshold is reached.

Discovery

1
use nats_extra::request_many::RequestManyExt;
2

3
let client = async_nats::connect("demo.nats.io").await?;
4

5
let responses = client
6
    .request_many()
7
    .stall_timeout(Duration::from_secs(1))
8
    .send("subject", "payload".into())
9
    .await?;

In this scenario, the iterator closes if no new messages arrive for at least one second after the last message. This provides strong assurance that you’ve received responses from all available and functioning responders.

There are more options and patterns available in the request_many extension, and this post only highlights a few. Check the documentation for your preferred language to learn more.

NATS Context Connection Helper (Go)

We also added a way for Go developers to connect to NATS Systems using the contexts created and maintained by NATS CLI. This package handles the connection and supports most of the settings found in NATS including Socks Proxy support, Custom Inboxes and more.

Link to the package: natscontext

NATS System API Client (Go)

Another utility added to orbit.go is the NATS System API Client. It exposes APIs to interact with the NATS server monitoring endpoints. The client can be used to get information from either a specific server (by server ID) or from the whole cluster. With this package users can easily use NATS to monitor their servers and clusters, without HTTP or any external tooling (like prometheus-nats-exporter or nats-surveyor).

Link to the package: natssysclient

Final thoughts

Orbit currently contains the request_many pattern in most tier-1 languages and some additional utilities, but we’re just getting started. Expect more to come soon. The foundations have been laid to remove complexity and API sprawl from the tier one clients and Orbit is our vehicle. The currently available libraries are:

Java - https://github.com/synadia-io/orbit.java
Go - https://github.com/synadia-io/orbit.go
Rust - https://github.com/synadia-io/orbit.rs
JavaScript - https://github.com/synadia-io/orbit.js

Feel free to kick the tyres and explore. We welcome feedback and hope this approach solves the challenges that you have no doubt experienced in this OSS project or others.