NATS Weekly #31

Week of Month 13 - 19, 2022

🗞 Announcements, writings, and projects

A short list of announcements, blog posts, projects updates and other news.

⚡Releases

Official releases from NATS repos and others in the ecosystem.

nats.rs - async v0.15.0 (now with JetStream!) and non-async v0.21.0
nats.net - v0.14.7

⚙️ Projects️

Open Source and commercial projects (or products) that use or extends NATS.

Real-time whiteboarding (preview) - by Jeremy Saenz

🎬 Media

Audio or video recordings about or referencing NATS.

Let’s Remix Distributed Database Design! - by Joran Dirk Greef, TigerBeetle (not a talk about NATS, but a fanstatic talk nonetheless AND some of the ideas may come to the NATS server someday…)
Choosing Cloud Native Technologies for the Journey to Multi-cloud - by Adelina Simion, Form3
Disrupting the Downtime Continuum - by Taylor Thomas & Brooks Townsend, Cosmonic

💬 Discussions

Github Discussions from various NATS repositories.

nats-server - How to achieve horizontal scaling with multiple Nats Server?

💡 Recently asked questions

Questions sourced from Slack, Twitter, or individuals. Responses and examples are in my own words, unless otherwise noted.

How can I define subject mappings on my accounts?

Subject mapping provides a way to take messages published on a subject (including wildcards) and map them to one or more subjects for subscribers. Refer to the linked documentation for all the use cases it supports (token reordering, partitioning, traffic shaping, failure testing, A/B testing, etc.)

As defined in the docs, this is shown as server configuration, however subject mappings can be declared at the account-level when using the JWT authentication method. This feature is supported into the nsc tool in the add and delete subcommands.

What does the `unique_tag` configuration option for JetStream do?

Last week I highlighted the experimental “server tags” feature which allows for declaring one or more tags describing any property of the server. Most commonly, this pertains to location, such as region and availability zone, but it could indicate resource quality such as CPU cores, memory size, or storage IOPs.

For example, if we have a handful of servers deployed in the us-east1 region of Google Cloud across three availability zones, we may use the following tags on each server:

cloud:google
region:us-east1
az:a, az:b, az:c

Then we could create a stream defining the placement tags:

nats stream add --replicas 3 --tag cloud:google --tag region:us-east1

One observation made after this feature was introduced was that an R3 stream (a stream configured with three replicas) with tags like the above, could result in the placement of all three replicas in the same availability zone, such as az:b.

In NATS server v2.8.0, the jetstream.unique_tag option was released to define a constraint that any replicated stream created must have replicas placed on unique servers defined by this tag prefix.

jetstream {

unique_tag: "az:"

}

This field declares that any given stream replica must have a unique az: tag relative to its other replicas (in addition to the other tags defined as part of the placement configuration). In the example above, this means each replica would be guaranteed to be placed on az:a, az:b, and az:c rather than servers in the same AZ.

Thanks to Jeremy Dartigalongue in Slack for bringing up this interesting case.

Let’s setup the scenario and then discuss. Given a stream USERS which binds to users.* and a NATS user named sue with a subscribe permission on users.sue, the question is whether Sue will only receive messages published to users.sue.

With core NATS, a nats sub call binds directly to the subject, so if an explicit allow on users.sue is defined, then that would be the only subject Sue can create a subscription on.

However, with JetStream it is more nuanced. Recall that a stream binds to a set of subjects (e.g. users.*) and a publish/request to users.sue (or users.joe) would result in the stream receiving, persisting, and ack’ing the message.

In order to actually subscribe to the messages in the stream, a consumer must be created. In the case of a push consumer you would need to define a deliver subject (or the default being a randomly generated one) that clients would subscribe to. This is different from the actual publish subject, i.e. users.*.

In the case of a pull consumer, a batch of messages are fetched using a dedicated service API ( $JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer>) and the messages are published from the server to a random _INBOX.> subject that the client is subscribed to. All of this is abstracted away in the client SDKs, but it is important to note that a subscription via a consumer to a stream does not actually subscribe to the publish subject, users.* in this case.

The implications of this is that permissions of application-defined subjects when using JetStream only map one-to-one for publishers. In other words, if a users.sue permission for publishing is defined, then Sue would indeed be blocked from publishing users.joe. But there is not the same symmetry on the subscription side since different subjects are being used via the consumer APIs.

One workaround if the cardinality of the subject space is not too large is to create a dedicated consumer for each user with the FilterSubject defined. Corresponding permissions on the relevant $JS.API.CONSUMER.> subjects would need to be defined as well to prevent the user from subscribing to a different consumer.

Alternately, if consumers are created out of band, if an explicit DeliverSubject is defined, the user permission could be scoped to that subject rather than the publish subject. This would also not require defining special permissions on the $JS.API subjects.

☝️ Edit: Actually this still does not work since messages are pushed to that subject without being filtered by the permission subject.

One open question is whether these permissions should result in implicitly filtering messages relative to the connected client’s user permissions. Currently, user JWTs are not pushed to the server so this permission information is not available even if this mechanism was desired.

☝️ Edit: Although the user JWTs are not pushed to the server, it is provided when initiating the connection between the client and server.

Another scenario is what happens if a two clients with different user permissions attempt to consume from the same stream? Should this be possible? Or can a consumer only be shared by client’s with the same user?