Synadia has been on a roll lately.

We recently were awarded the [Stratus Award](https://www.bintelligence.com/blog/2022/9/26/59-leading-cloud-computing-companies-products-and-services) for [Cloud Disruptor](https://www.einpresswire.com/article/592479318/synadia-named-a-global-leader-in-cloud-computing) of the year. What we love about this award is that it is not for a single cloud service, rather it recognizes the truly game changing capabilities provided by Synadia and the [NATS.io](http://NATS.io) platform for enabling our customers to deliver globally distributed applications across any cloud, geography or device.

On the product front, the NATS Server [version 2.9 release](https://nats.io/blog/nats-server-29-release/) was a major milestone which was a result of the combined efforts from over 40 community contributors and the Synadia team. The 2.9 NATS server release provides both significant new capabilities as well as platform stability and resiliency. One of the repeated requests from our users has been around simplifying the challenge of delivering consistent data to highly distributed edge environments, reliably and with very low latency. Not an easy thing to accomplish on a global basis to edge devices such as autos, wind turbines, or satellites. NATS Server v2.9 provides great functionality to enable this data locality use-case both simply and securely.

Our ecosystem also continues to expand with amazing partnerships and joint solutions. Here are just a couple of the recent highlights:

- [**Cosmonic** ](http://cosmonic.com)\- WebAssembly is arguably one of the hottest technology trends for building globally distributed multi-cloud applications. Learn how NATS is powering the Cosmonic wasmCloud platform - <https://synadia.com/blog/wasmcloud>
- [**HarperDB**](https://harperdb.io) - Listen to our own Byron Ruth discuss with Margo McCabe how NATS is powering a globally distributed data platform - <https://rss.com/podcasts/harperdb-select-star/635752/>
- [**ControlPlane**](http://controlplane.com) - When SAFE Health Systems needed a platform to securely deliver healthcare data on a global basis without managing cloud infrastructure, they selected ControlPlane & Synadia -   <https://synadia.com/articles/synadia-control-plane>
- Synadia has partnered with the [5G Innovation Lab](https://5goilab.com/) to accelerate a reference architecture for cross industry cloud native & data streaming applications deployments. We are currently engaged with several very interesting discussions about the integration of NATS with 5G. Stay tuned.
- The world’s largest [**IoT Community**](https://iotpractitioner.com/) invited Synadia to join their mission of educating & fostering global collaboration on best practices and business value. We are pleased to be featured in the latest whitepaper covering innovations in [distributed computing & cloud native applications](https://iotpractitioner.com/intelligence-report-volume-16-cloud-native-applications/).

Given all of the progress and activity we thought it was time to get our global community together to more formally update everyone. Our upcoming event will be a ½ day virtual event November 10th, full of product updates, demos and use-cases. We are calling it [**RethinkConn 2022**](https://synadia.com/events/rethink/2022) to highlight how Synadia is fundamentally transforming the distributed computing and connectivity market. Please register for this free event <https://synadia.com/events/rethink/2022>.

We are also looking forward to seeing everyone at Kubecon + CloudNativeCon in Detroit next week. Join Tomasz Pietrek, Jeremy Saenz & Todd Beets from Synadia for their NATS Project Maintainer talk “Connect All the Things! Using NATS to Simplify Communication Everywhere”, Thursday October 27, 2022 2:30pm - 3:05pm ET. [Add it to your Kubecon schedule today](https://kccncna2022.sched.com/event/182OD)!

Synadia is on a roll!

Synadia on a Roll

## Matrix Dendrite

[Matrix](https://matrix.org/) is a federated protocol for real-time communications built on top of distributed data structures and [Dendrite](https://github.com/matrix-org/dendrite) is an implementation of a Matrix homeserver built following a microservice architecture. It was originally intended to help us to overcome some load issues that were present in earlier monolithic implementations at the time. As a result, flexibility, performance, and scalability have been three critical areas of focus in our development.

## In the Beginning with Kafka

When we first started the Dendrite project, we were using Kafka as a means to distribute events and asynchronous tasks between microservice components. In addition to being able to scale up and down as needed, we also needed durability in the message queue to ensure that if parts of the service were restarted, no tasks or events were lost. Given our need for deployment flexibility, it’s equally important for all of the microservices to run under a single process as it is to be able to split them out and run them under separate processes - or even on different machines. We wanted to be able to target embedded and small platforms, as well as the datacentre.

While Kafka allowed us to get earlier versions of Dendrite working quickly, we eventually found it to be relatively heavy when used in a full polylith microservice deployment and didn’t scale down as well as we had hoped for smaller deployments. To successfully run under a single process without a standalone server handling the events and tasks, we were also forced to mock up our own lightweight Kafka-ish queue implementation, which carried with it additional maintenance burden. We weren't satisfied with the performance of either mode of operation and found that data retention was especially difficult to manage.

---

> ### "NATS gave us everything we used Kafka for, but had a number of additional benefits."
>
> \-- Neil Alexander, software developer, Matrix
>
> ---

## Why [NATS.io](http://NATS.io) is a Better Fit for Dendrite

We evaluated a number of different alternatives and eventually landed on [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream), which in effect gave us everything we used Kafka for but had a number of additional benefits. For a start, it was considerably easier to get running. The [NATS Server ](https://github.com/nats-io/nats-server)consumed far less system resources. Being written in Go, we were able to embed NATS into our own monolith-style binaries so we didn't even need a separate server instance in those smaller deployments. We contributed some patches to make it possible to communicate with the NATS Server in-process without using external sockets; this made it possible to both easily target mobile devices, and venture into the web browser under WebAssembly too.

---

**NATS Benefits vs. Kafka:**

- **Consumes less system resources**  -- Less infrastructure spend than Kafka
- **Written in Go** -- Easy to embed, no need for separate server instances like in Kafka
- **Simpler data retention** -- Using JetStream, easier to manage and configure retention than Kafka
- **Reduced management overhead** -- Easier for developers to manage and maintain than Kafka
- **Reduced codebase complexity** -- simpler code stack due to elimination of complicated persistent storage requirements

---

For us, switching to JetStream was transformational. Performance is excellent overall. We make heavy use of the interest-based retention policies to ensure that work is cleaned up when it is complete. We take advantage of being able to manually acknowledge items to ensure they were processed successfully before cleaning up. We've been able to successfully manage pools of workers from NATS streams and considerably reduce the amount of complexity in our codebase in a number of places which no longer need their own persistent storage. We've even had it successfully running in application-embedded scenarios on iOS and Android as a part of our peer-to-peer demos.

In summary, our team is always on the lookout for the right tools and architectures for the performance and flexibility our users require. As part of our evolution we made the shift from Kafka to NATS, and have not looked back. Stay tuned to see what interesting updates lay ahead next year for Dendrite!

## About the Author

Neil Alexander is a software developer working on [Matrix (and Dendrite)](https://matrix.org) at [Element](https://element.io). He is interested in security, privacy, decentralised systems, and computer networking. Neil also has experience in end-user compute and enterprise-scale systems architecture.

Dendrite Project moves from Kafka to NATS.io

Dendrite Project moves from Kafka to NATS

We proudly announced last week the[ first release](https://github.com/nats-io/nats-server/releases/v2.9.0) of the 2.9.x series! 🚀 This is a major release for us in ways I’ll go over in this post.

First thing’s first. Thank you to the **48 contributors** ❤️ through GitHub issues, discussion posts, and pull requests. Beyond those direct contributors, hundreds of folks in our Slack community helped out with their questions, use cases and testing of early versions.

The 2.9 release focuses on improving the [JetStream](https://docs.nats.io/nats-concepts/jetstream) persistence layer in multiple ways including:

- ⚖️[ Increased stability and resilience](https://nats.io/blog/nats-server-29-release/#stability-and-resilience)
- 🌏[ Greater scale and mobility](https://nats.io/blog/nats-server-29-release/#scale-and-mobility)
- 🔧[ Improved operability and security](https://nats.io/blog/nats-server-29-release/#operability-and-security)

# JetStream Elevating NATS to a True Global Utility for Communications

Announced in March 2021, JetStream extends NATS to provide *at-least-once* and *exactly-once* delivery guarantees as a first-class subsystem of the NATS server. The community quickly embraced JetStream and we have seen a lot of novel use cases that require enhanced scaling and performance. JetStream contributes to the big vision for NATS of providing a **connective technology** for all communications use cases between applications in the cloud, on the edge, and even in space.

JetSteam truly takes NATS to the next level, not only by adding persistence and the ability to better handle IoT / edge applications that suffer from intermittent connectivity but also with entirely new capabilities like an integrated [key-value](https://docs.nats.io/using-nats/developer/develop_jetstream/kv) and [object store](https://docs.nats.io/using-nats/developer/develop_jetstream/object) (beta). With JetStream, NATS becomes a true utility tool – building on the vision of NATS creator Derek Collison, to make connectivity a global utility supporting numerous use cases with a single tech stack. Like the rest of NATS, all of this comes in a fully [open sourced](https://github.com/nats-io/nats-server) package, wrapped into a small,[ static binary](https://docs.nats.io/nats-server/installation), with no (as in zero) dependencies. 🎉

# **Taking JetStream to the Next Level**

The 2.9 release takes all of this goodness to the next level. Some of the specific benefits the 2.9 brings to users include:

- Reduced time and bandwidth for replication catch-up
- Improved message distribution
- Better handling of memory usage for containerized environments
- Replica and mirror-based direct gets
- Stream and key-value message republishing
- Consumer replica, storage, and filter subject change
- Server tags and portable streams

NATS 2.9 also adds key new capabilities in security and operability including:

- AES-GCM cipher for JetStream file-based encryption
- Account purge operation
- Templates for scoping signing keys
- New and enhanced subject mapping functions.

A special thanks to the core NATS contributor team for working so hard over the last few weeks to get this release right. Please read the full release [blog post](https://nats.io/blog/nats-server-29-release/#stability-and-resilience) on the [NATS.io](http://NATS.io)[ project blog](https://nats.io/blog/). You’ll find code examples and much greater detail.

Please [download NATS 2.9](https://nats.io/download). Take it for a spin. Tell us what you think on [Slack](https://slack.nats.io) or[ GitHub](https://github.com/nats-io). And if you are interested in [contributing to ongoing NATS development](https://nats.io/contributing/), we’d love to have you.

Credit: Stephen Leonardi / Unsplash

NATS Server 2.9 Release: More Stability, Greater Scale, Better Security

WebAssembly aka Wasm was first published in 2017, allowing developers to bring their code to the browser to execute applications in high performance. In 2019, Mozilla introduced WASI - Web Assembly System Interface. This allowed Wasm apps to access resources beyond the browser, and run far more powerful systems.

As a result, WebAssembly (Wasm) has become a very hot topic with distributed application developers & architects. It is driving a major revolution in application platform design, even moving beyond the public cloud as a destination - and to the rapidly growing Edge ecosystem. 

We see a natural affinity for the integration of Wasm and NATS, and have seen numerous developers implementing the two together. This is both for large enterprises trying to re-architect brittle cloud environments designed for vendor lock-in but also for emerging startups & SaaS disruptors that are building new global data businesses without any legacy technical debt.

Services like <a href="https://wasmcloud.com/" target="_blank">wasmCloud</a> are creating a whole new paradigm for Cloud-Native developers by eliminating entire classes of problems such as security, portability and device independence.  

If you want to see more of this in action, this recent article from Brooks Townsend, a maintainer on the wasmCloud team, is an excellent overview of the possibilities with Wasm and NATS.

Explore the article <a href="https://wasmcloud.com/blog/globally_distributed_webassembly_applications_with_wasmcloud_and_nats/" target="_blank">Globally Distributed WebAssembly Applications with wasmCloud and NATS</a>.

WebAssembly Applications with wasmCloud and NATS

wasmCloud

## The evolving and demanding nature of edge systems requires a new set of characteristics. 

In the [last post](../blog/adaptive_edge_1), we talked about the requirements we see emerging for applications as we move from a Cloud Economy to a Connection Economy where smart connectivity is the primary challenge rather than compute and storage. To handle the requirements of modern distributed applications, Adaptive Edge Systems should offer the following capabilities:

## 1. Performant
Adaptive Edge systems should be at least as performant as incumbent systems and ideally considerably more performant on a per compute basis. This can be measured in various metrics including latency, message or stream throughput, or  compute required per million messages, to name a few. This also usually indicates lower energy consumption, a critical component of IoT and remote environment requirements. 

## 2. Easy to deploy and maintain
One of the key problems with incumbent messaging and streaming platforms is the challenges in deployment and the heavy infrastructure requirements. Because Adaptive Edge systems must be dynamic and will likely encourage experimentation, these systems must be simple and fast to deploy and easy to maintain. A good rule of thumb is that, if a team or dedicated engineer is required to maintain a solution, then it is not suitable for Adaptive Edge use cases.

## 3. Lightweight and works in any environment
Incumbent messaging and streaming systems generally require considerable compute capacity as well as supporting systems. For example, Apache Kafka requires not only a large compute instance but also a JVM; this usually means Kafka installations require multiple instances and are not suitable to install in resource constrained environments common in IoT or other Adaptive Edge applications. For an Adaptive Edge connection fabric to be effective, it must be able to run on all types of edge devices from small sensors and RaspberryPIs to standard physical servers and cloud infrastructure. The connection fabric must also maintain all features at any scale; the fabric must be environment agnostic.

## 4. Resilient and self-healing
While this may be a requirement for any enterprise software system, connection fabrics are the backbone of SLAs and the glue that binds multi-cloud applications to reduce downtime and improve latency. For this reason, Adaptive Edge connection solutions must boast higher-than average reliability and durability — more akin to what is expected with telecommunications systems and other regulated technologies judged to be mission critical to society. If portions of a connection fabric crash or slow down, then other nodes in the cluster should instantly absorb any traffic and workloads.

## 5. Client / language agnostic
Modern applications may be composed of dozens or hundreds of smaller applications and microservices. Those smaller applications and microservices may use different languages. It is not uncommon to see a half-dozen languages represented in different functions of a distributed application. Adaptive Edge connection solutions must support a wide range of languages and also have a straightforward integration schema using a well-structured API.

## 6. Multi-cloud and cloud-agnostic
Most Adaptive Edge applications do need to connect to the cloud. To future proof those applications and enable better SLAs for their services as well as reducing single point of failure, connection fabrics should be able to bridge clouds and provide multi-cloud capability. This will include moving workloads from one cloud to another and load balancing across clouds. The same capability is also useful within clouds.

## 7. Location independent
This describes both geography and application or system topology. Admins and users should be able to add and remove clients on Adaptive Technology connection fabrics without requiring configuration changes. Similarly, clients should be recognized by the fabric no matter their location.

## 8. Tolerates connectivity disruptions
The adaptive part of Adaptive Edge includes the ability to handle disruptions in connectivity and then resume connections while sharing data asynchronously and when possible with centralized application servers. This also implies that Adaptive Edge systems must be able to deliver both batch and stream processing, and switch between the two depending on circumstances and conditions.

## 9. Persists data intelligently
Data persistence is essential for giving Adaptive Edge applications the ability to run backend processes or make business logic decisions without having to call back to a cloud data center. More advanced connection fabrics integrate a key/value and object store with data persistence, to simplify the application architecture and remove the need to deploy a distributed database alongside the connection fabric.

## 10. Aggregates and distributes data intelligently
Adaptive Edge systems must support data aggregation at the edge. Data that is salient to the cloud gets to the cloud. Data that is only salient at the edge stays at the edge. This minimizes unnecessary data tromboning, saving on bandwidth and infrastructure costs and allowing application architects to better optimize data handling and business logic.

## 11. Secure by default
Adaptive Edge applications are dynamic in ways that break old security paradigms tied to IP addresses. Securing more complex topologies with dynamic client and node structures requires security to be a part of  the connection fabric and not applied as a secondary or tertiary layer, or as a mix of different integrated technologies. One solution is to assign clients to accounts and limit accounts to namespaces. This makes segregation default and delivers true multi-tenancy as part of the architecture. Of critical importance - the connection fabric never stores or accesses private keys.Security is bifurcated from connectivity.

## 12. Supports multi-tenancy
Related to security, multi-tenancy for connection fabrics enables the use of the fabric as a global utility traversing applications or clouds. For example, a true Adaptive Edge solution can silo message and streaming data for a microservice limiting reception or subscription to a specific namespace, even as other microservices in the same fabric are limited to communicating with a different namespace.

## 13. Scales quickly, horizontally and vertically
Part of any connection fabric’s ability to adapt must be rapid scalability to handle bursts in demand or topology shifts when part of a cluster shuts down or moves traffic from one fabric server to another. Quick scaling should translate into end users never noticing that their connection fabric has had to scale up or add capacity. This scaling should be equally viable horizontally (adding more servers) or vertically (adding capacity to existing servers).

## 14. Client simplicity and naivete
Adaptive Edge connection fabric clients should remain simple and stateless, focused on connecting, publishing or receiving data. Because no server or state data is maintained, new clients can be added to a connection fabric without topology changes. This future proofs the connection fabric and allows for easy adds and modifications to client configurations.

## Conclusion: A New Age of Adaptive Edge
From autonomous vehicles to the new energy grid to Industry 4.0, applications increasingly require proximity to activity. Next generation distributed, multi-cloud and edge-autonomous systems need more than what legacy message brokers and streaming data systems can offer. This is growing the demand for adaptive connection fabrics that are lightweight, secure, distributed and environment-agnostic. These fabrics can fit multiple deployment paradigms and adapt and grow with edge applications, often replacing both messaging and streaming systems. These systems save infrastructure costs while increasing security and simplicity. With the adaptive edge, apps become the center of the universe rather than the clouds that or servers that run them. By applying the 14 factors, architects can ensure the messaging and streaming capabilities of their infrastructure meets the needs of adaptive edge applications, today and tomorrow. 


14 Factors of Adaptive Edge: Pt. 2

Adaptive Edge 2

## How the next generation of distributed, multi-cloud and edge-autonomous systems are forcing the evolution of message brokers and streaming data systems into adaptive connection fabrics that are lightweight, secure, distributed and environment-agnostic. 

We have entered a new era of distributed applications and infrastructure driven by performance, security and resilience on the Edge. Makers of distributed applications require new capabilities that allow applications to continue functioning effectively, even in conditions of poor or intermittent connectivity. As performance expectations and service level agreements approach near real-time for a growing array of functionalities, applications require greater persistence and autonomy on far more widely distributed arrays of nodes; the Edge becomes the compute fabric, the memory and the application tier, all rolled into one. The Adaptive Edge is the new construct that fulfills these new and more exacting requirements. Just like web applications forced a rapid evolution for technology architectures in the early Cloud Era, Edge Applications are forcing a similarly rapid evolution in the Edge Era. This paper attempts to define a set of design patterns and principles for the applications and architectures on the Adaptive Edge.

## 14 Factors

Message brokers and event streaming software are critical infrastructure for modern software applications and technology infrastructure. These systems make it possible for applications, systems, and services to exchange information, even if they are not written in the same software language or are deployed on different platforms or environments. Message brokers emerged as software and hardware became more distributed and composed of discrete services. Event streaming software — or event buses — emerged from the message broker world to handle massive volumes of streaming information and ensure that data could be found by users that need it. 

Message brokers and event streaming both relied on one-to-many architectures. Over time, as systems grew still more distributed, the publish-subscribe (pub/sub) model for message brokers and event streaming became more common, allowing end users or services to subscribe to and store streams of data or messages that they needed to consume. 

While the core functionality provided by legacy designs for message brokers and event buses remains crucial, current systems can no longer handle the requirements of modern distributed applications. These applications are rapidly moving to the “Edge” and are requiring local and adaptive processing and persistence capabilities. These capabilities are either not readily available or are complicated and expensive to provide with current message broker or streaming data technologies. This is largely because the paradigm for moving and exchanging data and messages has shifted from a one-to-many to many-to-many. 

Because legacy message and eventing systems are not designed to live on the edge, they often suffer from high-latency and inconsistent delivery of key data to or between edge clients. Because the edge is where data is generated by distributed applications interacting with users and sensors, the question becomes whether to move that data back to the cloud for processing or to process it on the edge with distributed business logic. By extension, this raises the requirement of managing and scaling data aggregation on the edge in a manner that allows for distributed applications to make decisions based on the latest information and not be forced to “trombone” large volumes of data back to cloud data centers.

Further, the requirements of many-to-many architectures require the ability to continuously shape and reshape the topography, taking into account intermittent connectivity, concerns about data transit and storage costs, and the limitations of SLAs and capabilities of cloud computing providers. Enterprises not only want the ability to run applications on the edge but also to allow multiple clients on the edge to act in concert until connectivity is re-established. 

Applications such as IoT or mobile applications today need the ability to run either as a full messaging and connection server on a small form-factor or to function merely as a simple client, bridging the nodes of the application back to more centralized compute and storage. Even architects of more traditional applications that are not required to run on the edge are seeking out adaptive capabilities that require a more flexible connection fabric to handle messaging and streaming; for example, companies are looking to stand up the same application in multiple clouds while maintaining centralized awareness and management of processes and data. These cloud-agnostic designs require a unified service that can provide messaging and eventing within one cloud or across multiple clouds without a dependence on a hosted service in any single cloud. 

Another emergent use case is building an internal SaaS capability that extends an external SaaS service into an organization but allows for an on-premise version that mirrors all capabilities and benefits from product and feature improvements of the external service without exposure to the public Internet. (This configuration is increasingly popular in regulated industries or in at-scale organizations seeking to reduce costs). 

All of these emerging use cases are versions of a similar architecture — what we are calling the Adaptive Edge. The Adaptive Edge Architecture is a flexible deployment topology overlaid atop of legacy architectures based on physical data centers and cloud computing. Adaptive Edge can either integrate with these constructs or function as an independent construct.  that supports one-to-many, many-to-many, and combinations of the two along with sharding of the connection fabric into local clusters that can continue to deliver back-end processing and application functionality even with eventual and discontinued connectivity. 

The Adaptive Edge also requires not only persistence but a lightweight attached data store (such as a key/value or object store)  that can make persistence intelligent to reduce data roundtrips and better enable edge application functionality.  A crucial lynchpin in this Adaptive Edge Architecture is a multi-tenant security model that makes it easier to create tightly partitioned namespaces and sub-groups for data connections, messaging and eventing and other actions. These capabilities make the Adaptive Edge a more flexible paradigm that becomes a general purpose massively distributed control plane. This plane is needed for the next generation of multi-cloud and edge-native applications.

## Designing Systems for the Adaptive Edge
A common pattern among distributed systems has emerged with a central group of applications servicing and receiving data from edge nodes. Usually there is telemetry coming from the edge. Sometimes edge nodes have their own services around command and control and access to local data. In essence, this is a federated edge, loosely coupled and malleable for supporting existing and creating new services. Often identified as an IoT artifact, this pattern also works quite well for other use cases that benefit from an adaptive edge layer and flexible service and message delivery.

Enterprises are executing this architecture by integrating multiple technology systems and hacking them into the Adaptive Edge pattern. These designs require large amounts of resources, staff time, and infrastructure in order to become operational. Yet, in the end, hacked-together Adaptive Edge architectures generate systems that are fragile and break easily, difficult to monitor, and challenging to secure.
  
To build successfully for the Adaptive Edge, a different approach is required, one that looks at creating a federation of equally important or capable nodes or servers, which can reside in any form factor and are just as comfortable in a small Windows PC as in a cloud server. For a truly Adaptive Edge,  communications systems must support and even improve on incumbent paradigms for both one-to-one and one-to-many communications models. But Adaptive Edge systems must also function as a superset of both one-to-one and one-to-many, behaving more like a fabric or mesh that enables almost any shape of communications network topology. These communication fabrics behave more like a graph, allowing connections to form, self-organize and exist as part of the greater whole but remain as independent subsets or connect back to the communication fabric opportunistically and when possible.

Discover more in [Part 2](../blog/adaptive_edge_2)...

14 Factors of Adaptive Edge: Pt. 1

Adaptive Edge 1

As innovation takes off at the Edge, one aspect we’re all trying to avoid is complexity. There is simply no room at the Edge for complexity— or much else for that matter. Simple, fast, robust, small yet powerful and dynamic is the name of the game. These demands are high. Enter ‘cloud native’ tech with its lack of assumptions, microservices architectures, container-based applications and dynamic orchestration of compute, storage, and networking resources and it’s quickly becoming apparent how fit for purpose cloud native is for the Edge.

The team of engineers at [Synadia](../../index.jsx), a SUSE One partner, in conjunction with Community Maintainers, develop and maintain the open source <a href="https://nats.io" target="_blank">NATS project</a>, part of the <a href="https://cncf.io" target="_blank">Cloud Native Computing Foundation</a>. The NATS communications system is used for all kinds of interesting implementations but we’ve invited Synadia to share two compelling Edge use cases from the energy sector demonstrating the power of cloud native solutions at the Edge. ~Bret

> _Bret Dayley has spent 30+ years in high tech roles with various software companies. A fan of open source and all things good, Bret enjoys making things happen and having a good time while doing it. While fascinated with technology, on the weekends you’re likely to find Bret enjoying the great outdoors on a dual sport moto… with some gadgets along for the ride._

## SUSE guest blog authored by Brian Flannery, VP Global Sales, Synadia

As workloads increasingly run at the edge, there is a massive paradigm shift underway to allow location dependent and independent applications to run on any device – securely and efficiently.

Complexity is not something developers want to introduce into edge architectures.  Moreover, edge workloads need to be able to instantly work with cloud, hybrid, and on-premises applicationswithout any re-architecting.  This is where <a href="https://www.suse.com/products/k3s/" target="_blank">K3s</a> and <a href="https://nats.io" target="_blank">NATS</a> work superbly together.

![K3s and NATS](../blog/posts/k3s_nats.png)

NATS is an extremely lightweight, performant, and secure (Zero Trust design) messaging utility. The two together offer edge device and application orchestration (K3s) and communication and data ingestion (NATS), solving two critical design requirements for any developer.

Two interesting, recent real-world examples come from the Energy industry.

## How Duke Energy uses K3s and NATS at the Edge

<a href="https://www.duke-energy.com/our-company/about-us" target="_blank">Duke Energy</a> and its Emerging Technology Office (ETO) are committed to innovation in renewable energy resources, energy storage, distributed intelligent grid architectures, machine to machine interoperability through standards like OpenFMB, and best practice Zero Trust cybersecurity. Consequently, ETO has built a proof-of-concept system named ZTAG, “Zero Trust Applications for the Grid”.

ZTAG represents a multi-year effort to develop a best-in-class architecture for the Distribution Grid. The system has now been deployed at the ETO Mount Holly Microgrid using OpenFMB. Each OpenFMB node is a field device with compute resources like an industrial PC or gateway. The system provisions and deploys nodes, updates node applications, and automates key renewal through distributed PKI. The architecture leverages Docker for application containerization, Kubernetes for container management, SPIFFE/SPIRE with Trusted Platform Module (TPM) identity management for workload attestation, and NATS for secure publish and subscribe communications. The system is scalable and secure from the data center to field devices. The goal is for all field devices to adopt node attributes.

SUSE’s K3s is a critical component in the ZTAG architecture. Multiple distributed OpenFMB nodes form logical clusters that work in small groups to implement distributed intelligent (DI) applications at the electric grid edge. These small “edge” clusters are grouped together in a hierarchy of clusters to perform higher level grid functionality. K3s enables quick, secure workload/application updates providing an environment for the development of secure DI applications.

![Duke Energy](../blog/posts/duke_energy.png)

The ZTAG team also works closely with Synadia’s team, as NATS is also a key component of ZTAG. NATS provides low-latency, multi-path, asymmetric, and event driven data flows from device-to-device, device-to-application, and application-to-application throughout the ZTAG architecture. NATS also provides distributed security allowing actors to only communicate with other authorized actors. These features enable the electric grid to no longer be bound by only north-south data flows from devices in the field to so-called “head end” systems in the back office. This new east-west, peer-to-peer messaging architecture enabled by NATS is foundational for distributed applications in the future electric grid.

## How PowerFlex uses K3s, Rancher, and NATS at the Edge

<a href="http://www.powerflex.com/" target="_blank">PowerFlex</a> is a leading national provider of intelligent onsite energy solutions that support carbon-free electrification and transportation. As a single full-service provider, PowerFlex customizes clean technology solutions to help clients achieve their energy and sustainability goals. Through the comprehensive <a href="https://www.powerflex.com/products/powerflex-x/" target="_blank">PowerFlex X platform</a>, PowerFlex leverages patented smart software to control, monitor, and optimize a client’s distributed energy resources to reduce cost and maximize return on investment.

SUSE’s K3s  is a critical part of how PowerFlex builds and manages highly distributed bare-metal systems on the edge. When PowerFlex added Rancher to the mix, it provided an extra layer of observability while keeping hardware and software secure. PowerFlex developers can hit the ground running with K3S on hardware or in VMs, and they value the access that Rancher gives them to pre-production environments. The PowerFlex Sight Reliability Engineering team has confidence in their ability to address issues in production thanks to the rapid and secure access Rancher provides.

Ted Lee, Principal Software Engineer at PowerFlex, and Robbie Hughes, Full Stack Software Developer on the PowerFlex Cloud X team, are replacing legacy REST services with an event-based architecture with NATS at the center. Ted said, “I’m excited that NATS provides a secure connection between our cloud and edge services where messages are just there.” Robbie echoes his excitement and adds that “the leaf node topology is amazing because it allows our sites to operate independently even when networks go offline.”

![PowerFlex](../blog/posts/powerflex_energy.png)

Interested in getting started at the Edge with NATS? Contact [info@synadia.com](info@synadia.com).

## Additional resources on using K3s and NATS:

 * <a href="https://www.youtube.com/watch?v=zbDjr8rcwZs" target="_blank">NATS & K3S | Extend Your Apps to the Edge in Under 60 Seconds</a>
 * SUSECON 2022 session: <a href="https://susecon.com/sessions.html?search=nats#/session/1649088878519001ApvY" target="_blank">Running a Distributed K3s Control Plane at the Edge with NATS JetStream [TUT-1217]</a>

 ## About the Author

 Brian Flannery is VP Global Sales at Synadia, and has experience scaling the business at multiple technology companies during the blast-off phase of growth. Prior to Synadia he was the CRO for a Salesforce development services partner, and also spent time at Open Source Software providers such as Eucalyptus and rPath prior to their acquisitions.

K3s and NATS

NGS Guides: Battery Spa

## How to Use NATS.io to Set Up a SOX Compliance System

An enterprise customer of ours recently asked for guidance around how their NATS.io enabled system could help them achieve SOX compliance. [Sarbanes Oxley Compliance](https://en.wikipedia.org/wiki/Sarbanes%E2%80%93Oxley_Act) encompasses a very broad set of practices around financial record keeping and reporting, touching nearly every aspect of running a business. While ensuring financial accuracy, undergoing compliance is like going to the dentist - no-one looks forward to doing it but it has to be done.

NATS provides several fundamental features that can help businesses to more easily achieve SOX compliance when properly used. This is most applicable to *Sarbanes–Oxley Section 404: Assessment of internal control*.

These features are centered around NATS advisories and System Events.  **Caveat:** In corner cases, system events and advisories may not be generated, published, or saved in the event of hardware failure, unexpected server shutdown, and other error conditions.

[Exabeam](https://www.exabeam.com) provides a very nice SOX compliance [checklist](https://www.exabeam.com/information-security/sox-compliance-requirements-and-checklist/). There is a lot to SOX compliance; this article is by no means exhaustive but we can run through some of the features of NATS as a good starting place.

## Prevent data tampering

*Implement systems that track logins and detect suspicious login attempts to systems used for financial data.*

This is fairly straightforward to implement for NATS. NATS generates [system events](https://docs.nats.io/running-a-nats-service/configuration/sys_accounts) that can be used to track these and persist them in the NATS [JetStream](https://docs.nats.io/nats-concepts/jetstream) subsystem. [Create a stream](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/streams#creating) and specfify the subjects you'll want to track to store these events.

The system event subjects you’ll want your stream(s) to include are:

- $SYS.ACCOUNT.*.CONNECT (client connects)
- $SYS.ACCOUNT.*.DISCONNECT (client disconnects)
- $SYS.SERVER.*.CLIENT.AUTH.ERR (authentication error)

If you are using leaf nodes, then include these subject:
- $SYS.ACCOUNT.*.LEAFNODE.CONNECT (leaf node connects)
- $SYS.ACCOUNT.*.LEAFNODE.DISCONNECT (leaf node disconnects)

You can then develop an application report and inspect these events to ensure no suspicious users connect, track down the source of authentication errors, and the frequency, time, and duration of connections are in line with normal parameters of your system. The definition of *suspicious* is left to the implementer.

Note that JetStream streams are tamper-proof, with the exception of erasing a message for GDPR purposes (this action can be locked down and audited).

## Record timelines for key activities

*Implement systems that can apply timestamps to all financial or other data relevant to SOX provisions. Store such data at a remote, secure location and encrypt it to prevent tampering.*

You’re covered by adding a bit of process and would want to ensure relevant financial transactions are stored in the NATS JetStream subsystem. JetStream supports [encryption at rest](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/encryption_at_rest) to protect your data. Each message stored in JetStream automatically has a timestamp associated with it, set at the time the message was first stored.  This applies to your event and NATS systems events (if stored in a stream).

```text
$ nats s get transactions 2
Item: transaction#2 received 2022-05-03 17:38:18.114054892 +0000 UTC on Subject payable
```

Data in streams can be [mirrored](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/replication#mirrors) or [sourced](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/replication#sources) to a remote location and/or [backed up](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/disaster_recovery#backup) to files for remote storage.

## Build verifiable controls to track access

*Implement systems that can receive data from practically any organizational source, including files, FTP, and databases, and track who accessed or modified the data.*

NATS has various connecting technologies, but let's focus on tracking of NATS assets.  In a secure NATS system, each connection has a user associated with it.  NATS provides rich monitoring tools to identify which users are accessing the system at any given time. Furthermore, each time there is a CRUD operation on a JetStream asset an [advisory](https://docs.nats.io/running-a-nats-service/nats_admin/monitoring/monitoring_jetstream#advisories) can be sent which includes the identity of the user.  These advisories can be stored in a stream or read by a process to store externally.

## Test, verify and disclose safeguards to auditors

*Implement systems that can report daily to selected officials in the organization that all SOX control measures are working properly. Systems should provide access to auditors using permissions, allowing them to view reports and data without making any changes.*

This is wide ranging.  NATS provides [monitoring](https://docs.nats.io/running-a-nats-service/nats_admin/monitoring#monitoring-nats) and logging in to ensure the system is up, and streams containing metadata of advisories and system events can be replayed anytime to generate a report. Given proper permissions, data stored in NATS can be accessed to generate a report anytime.
	
## Report on the effectiveness of safeguards

*Implement systems that generate reports on data that have streamed through the system, critical messages and alerts, security incidents that occurred and how they were handled.*

Earlier we discussed persisting events correlated to these items to streams. If a user has permissions they can replay the stream containing system events and jetstream audit events to generate a report.


## Detect and disclose security breaches

*Implement security systems that can analyze data, identify signs of a security breach and generate meaningful alerts, automatically updating an incident management system.*

*Implement systems that log security breaches and also allow security staff to record their resolution of each incident. Enable auditors to view reports showing which security incidents occurred, which were successfully mitigated and which were not.*

This would fall on the NATS operators to develop a system that meets the needs of the business. Though the system events generated, NATS provides plenty of data points for an enterprise to quickly build a custom system that does exactly this.

## Summary

SOX compliance is a very broad and loosely defined initiative; this article does not cover every aspect, but can serve as a starting point for enterprises that need to be compliant and are using NATS. We on the NATS team would be certainly open to suggestions as to how to make this easier; if you have ideas or would like to offer contributions in this area please reach out.

## About the Author

Colin Sullivan is VP of Product [Synadia Communications](https://synadia.com). If your enterprise needs assistance with SOX Compliance Synadia, or NATS usage in general Synadia can provide guidance.


Blog