Quantum Myriad – High Performance, Low Latency for Modern Applications

Quantum Myriad provides enterprises with consistently low latency with high bandwidth and IOPS. The all-flash file and object storage software also brings simplicity and adaptability to high-performance unstructured data workloads.

With unstructured data in the enterprise expected to more than double in the next five years, organizations continue to store that data on aging systems along with a habit of throwing hardware at the problem. Expanding and upgrading NAS systems or adding more drives to improve performance has created a bloated and inefficient NAS cluster. Larger enterprises are also adding to the complexity of their infrastructure with a limited headcount budget.

Quantum has addressed this issue with all-new distributed software designed for NVMe Flash. Quantum Myriad™ provides enterprises with consistently low latency for high bandwidth and IOPS. The all-flash file and object storage software also brings simplicity and adaptability to high-performance unstructured data workloads.

Quantum Myriad is a cloud-native, software-defined solution leveraging modern all-flash architecture. Myriad is meticulously designed for specialized applications such as data science, AI, VFX, and animation and focuses on maintaining reliable high performance with minimal latency. Further, Myriad clusters feature a contemporary GUI and integrated inline data services, including deduplication, compression, snapshots, and clones.

Founded in 1980, Quantum has a lengthy history in the storage industry, developing solutions from the ground up rather than attempting to modify existing aging systems. Quantum uses its extensive storage knowledge to develop platforms like Myriad without having to learn as they go.

Myriad leverages application frameworks and design advancements that were unavailable even a few years ago. This modern cloud-native architecture is an easy-to-use solution, overcoming the limitations of typical hardware-centric designs. Customers can adapt to future storage needs while reducing the workload on over-extended IT staff. The new levels of simplicity and adaptability to high-performance workloads are not constrained by investing in specialized hardware.

Built on Containers, Open Hardware

Quantum Myriad is fully containerized and uses familiar, proven cloud technologies like Kubernetes, providing an “always-on” architecture to deliver simplicity, automation, and resilience at even the largest scale. It also offers new features and fixes rapidly with minimal risk. By capitalizing on microservices and Kubernetes, Myriad optimizes deployment, ensuring increased flexibility and streamlined operations. Noteworthy is its hardware-agnostic nature, removing the need for bespoke equipment while allowing adaptability and scalability. This also lets Myriad adopt new technologies as they come to market.

Being containerized simplifies the ongoing orchestration of the microservices running in a Myriad cluster and allows customers to adopt new features and fixes faster and with less risk. In short, Myriad delivers cloud-like simplicity wherever it is deployed.

Everything about Myriad is designed with simplicity in mind. Software automates much of the storage and network management, making even large clusters more straightforward to manage without involving IT. The software automatically detects, deploys, and configures new storage nodes within a cluster so you can scale, modify, and even shrink clusters non-disruptively. NVMe Storage Nodes communicate via an internal, software-managed 100GbE RDMA fabric, eliminating networking complexities typically associated with legacy scale-out NAS systems.

Myriad delivers innovations across the spectrum. Automated storage management means clusters can be scaled or modified without user intervention or the need for advanced IT skills. The self-healing, self-balancing software automatically rebuilds data in the background while rebalancing data as storage clusters expand, shrink, and change. Flash storage costs are reduced with inline data deduplication and compression, and data efficiencies are improved compared to legacy storage systems. Data protection and recovery through built-in snapshots, clones, snapshot recovery tools, and rollback capabilities are made even more straightforward.

Simple Architecture That’s Extremely Resilient

Quantum designed Myriad based on cloud-native principles. Leveraging the containerized microservices architecture orchestrated by Kubernetes provides scalability, flexibility, and simplicity. The Myriad software components work together to provide internal functions and customer-facing services, such as storing and retrieving files and objects.

It’s easier to visualize the components and functions if viewed as layers that parallel the data flow through the system, each designed to minimize latency and maximize parallelism.

At the core of every Myriad system lies a transactional key-value store. This serves as the central repository for all data and metadata. It’s distributed across all storage nodes within a Myriad cluster and seamlessly scales as new nodes are introduced. This store’s redirect-on-write nature naturally facilitates snapshots and rollbacks, and, being lock-free, it effectively avoids a common source of contention encountered in shared storage systems.

The key-value store primarily stores data onto NVMe SSDs using a unique dynamic erasure coding (EC) scheme. This scheme adapts in real-time to various events, ensuring data protection remains intact while optimizing efficiency.

Myriad’s data services include inline deduplication, compression, and support for snapshots and clones. Additionally, the roadmap consists of plans for exciting and empowering services like replication and data analytics.

Cluster Architecture

Despite being a software solution, Quantum Myriad relies on a hardware foundation for execution. While it’s designed for versatility across various environments, including public clouds, the initial deployment of Myriad is facilitated through Quantum-supplied infrastructure. The beauty of this arrangement is its simplicity. It doesn’t demand custom or exotic hardware; standard x86 servers and OCP-compliant switches suffice.

Each node type in a Myriad cluster has a specific role that contributes to data storage, operations, or both. Nodes use common hardware available from a number of suppliers. Although the initial Myriad release will be available on Quantum appliances, customers can provide their hardware in the future.

At the heart of the cluster lie the NVMe storage nodes. Today, that’s a minimum cluster of five easily expandable nodes should customers require something more significant. A pair of Ethernet load balancer nodes are positioned up front to manage network connectivity. These nodes aptly balance incoming traffic from external sources to the Myriad cluster and the internal traffic within the cluster. There’s another crucial element as well: the 1GbE deployment node.

The deployment node takes charge of various cluster-related tasks, including initial setup, capacity expansions, and software upgrades. Its role is pivotal in streamlining the management and maintenance of the Myriad cluster.

Myriad nodes are interconnected internally and externally through high-speed 100Gb Ethernet networks. Unlike traditional storage clusters, which often require extensive customer involvement in configuring and sustaining networking infrastructure, Myriad incorporates this aspect into its design. The result is an automatic and hassle-free configuration, demanding no extra investment or effort from the customer.

System Capacity

External networking is equally straightforward. A Myriad cluster can be accessed and managed via a single IP address, regardless of size. What’s particularly advantageous is that, in most cases, storage administrators can independently expand the system without necessitating the intervention of the network team, offering unmatched flexibility.

Introducing the Nodes

The Main Player: Storage Node

At the core of a Myriad system are the storage nodes. These nodes are compact 1U Linux servers that collectively form a Kubernetes cluster, serving as the foundation for running the essential Myriad software components. From handling presentation layers to managing erasure coding and the user interface, these storage nodes take care of everything.

Storage Node Specification

Looking ahead, Myriad has plans to introduce more versatile storage nodes that can accommodate larger or higher-capacity drives, aligning with market trends and demands. The architecture is designed to be flexible, supporting mixed storage node types and offering scalability to meet diverse requirements. Myriad’s structure supports clusters as small as three storage nodes to scale up to any needed size. For instance, a 40-node Myriad cluster, utilizing today’s 15.36TB drives, can deliver an impressive 4.9PB of raw capacity (not factoring in deduplication and compression), showcasing its scalability potential.

The Balancing Act

In the Myriad architecture, the storage nodes play a vital role in intercommunication and connecting to the customer network. Additionally, the cluster requires an efficient mechanism to distribute incoming connections across all the storage nodes evenly. With traditional NAS clusters, these responsibilities typically fall on the customer’s shoulders and require the management of numerous IP addresses, virtual IPs, and old-fashioned round-robin DNS mappings. Myriad simplifies this aspect by including load balancer nodes as part of its package.

Load Balancer Node Specification

These load balancer nodes are essentially high-speed 100GbE Linux-based switches that are under the complete management of Myriad software. They serve a dual purpose: some ports on each load balancer node are utilized to interconnect the storage nodes within the cluster. In contrast, others are dedicated to external network connectivity.

Two load balancer nodes are sufficient for systems employing up to ten storage nodes. Scaling up to 20 storage nodes requires adding another pair of load balancer nodes to accommodate this expansion. As more storage nodes are introduced, surpassing the 20-node threshold, additional load balancer nodes and associated spine nodes are integrated into the infrastructure. Although the number of connections and load balancer nodes required may evolve with the advent of higher-density switches and faster network speeds, the underlying concepts remain consistent.

Deployment Node? Simple

One of the cornerstones of operational simplicity in Myriad is the deployment node. Like the load balancer node, this node is built on a Linux-based platform but operates at 1GbE. The deployment node plays a pivotal role in the system’s initial setup.

Deployment Node Specification

The deployment node is tasked with installing and configuring all other nodes within the cluster during the initial deployment phase. It acts as the orchestration center, directing new bare-metal nodes to install their operating systems, Kubernetes, and Myriad software components, seamlessly integrating them into the cluster. Furthermore, when the need for software updates arises, the deployment node manages these updates in a rolling fashion to prevent any disruptions or downtime.

It’s worth noting that clusters containing fewer than 20 storage nodes only necessitate a single deployment node. Although this may appear to be a potential “single point of failure,” this is not the case because the deployment node does not sit in the data path. Therefore, even with a deployment node failure, there is no disruption to I/O, so data protection and performance remain unaffected.

Expansion – Zero Clicks Required

Myriad has introduced a groundbreaking feature known as “zero-click” storage expansions to simplify increasing storage capacity. To augment storage capacity, all that’s required is the addition of more storage nodes. The procedure is incredibly streamlined: new nodes are unboxed, mounted in the rack, connected through cables, and powered on.

As outlined above, the deployment node takes on the role of identifying these freshly added nodes. It ensures they receive the requisite software provisioning and seamlessly integrates them into the existing cluster. Notably, this process does not entail connecting to a console port, launching a user interface, or raising a network team ticket for IP address assignment or DNS reconfiguration. The dynamic erasure coding (EC) mechanism immediately commences distributing data across all old and new nodes.

With Myriad, storage expansions are a paradigm of automation, eliminating any need for manual intervention or configuration.

Networking the Myriad Way

The most challenging piece of networking Myriad is finding a free IP address. That’s it.

Myriad uses typical networking elements but leverages the advanced technologies from RDMA and Kubernetes for additional networking requirements. Because Myriad is a fully integrated, self-configuring system, there are zero parts for customers to wire up themselves. The internal cluster networking is configured automatically and is invisible to the customer.

Once the initial cluster configuration is complete, a single IP address is needed for data I/O and management, no matter how large the cluster expands.

Internal Cluster Connectivity

You might wonder how those additional nodes get added without reconfiguring the cluster. Simple. Cluster nodes are interconnected using a 100GbE lossless network with RDMA and a 1GbE network for deployment functions. At installation, the customer provides a non-routable subnet for the internal network or can accept the 172.16/16 default. All cluster node and Kubernetes pod internal interfaces are assigned addresses from that subnet range. Now, when new nodes are added to the cluster, an IP address is automatically assigned.

To isolate the internal cluster network from the rest of the data center network, load balancer nodes run Network Address Translation (NAT). Internal source addresses and port numbers are mapped to external IPs and ports as packets egress from Myriad, and the external source addresses and port numbers of incoming packets are translated to internal IPs and ports.

Load balancer nodes use a technique known as Equal Cost Multipath routing (ECMP) to spread incoming connections across all of the storage nodes. This divides the workload across the cluster without the latency and configuration headaches of a round-robin DNS.

External Cluster Connectivity

Connections to an external cluster still need a subnet, but a much smaller one that supports three IP addresses: one ingress IP used for all data traffic and management and two egress IPs used for NAT (one per load balancer node). Additional IPs are needed if the cluster has more than ten storage nodes. Myriad’s external networking uses Border Gateway Protocol (BGP). BGP has become a favorite for organizations for routing within data centers because of its simplicity and robustness.

BGP is the routing protocol running the Internet and is used by ISPs to exchange routing information. BPG connects an ISP’s autonomous system, represented by a unique public Autonomous System Number (ASN). BGP in a Myriad cluster is an autonomous system with an ASN from the private range assigned by the customer. Customer switches, configured for BGP unnumbered with ECMP, connect to the Myriad-facing ports. ECMP provides load balancing for incoming connections across all Myriad load balancer node interfaces.

With its cloud-native design, Myriad’s architecture makes deploying on public cloud infrastructure such as AWS simple. Cloud deployments can take advantage of underlying cloud platform services, but the core software and experience will be the same as running Myriad in a data center.

Given that Myriad software utilizes containers designed for the Linux OS, it stands to reason that Myriad can be seamlessly integrated with virtually any cloud environment. Many storage vendors are de-coupling their hardware and software to facilitate cloud deployment. Yet, Quantum is already ahead of the curve with Myriad. This positions them for immense flexibility moving forward. Intriguingly—and this is purely our speculation—Myriad might be exceptionally compatible with ARM-based cloud instances, like AWS’s popular Graviton instances. Should Quantum capitalize on this, it could gain a significant edge by operating on the most cost-effective cloud instances.

Users can rely on a modern GUI interface to control and configure Myriad. Additionally, comprehensive API support is in the pipeline to enable tailored automation. For traditional monitoring, SNMP and sFlow are readily available options.

Myriad Deployment

Myriad simplicity starts with making it easy for customers to deploy the cluster. Customers only need to ensure BGP ECMP support on their switch. Most switches offer BGP, but it may require being enabled. Otherwise, the customer needs only to provision an IP for the cluster.

Once the nodes have been racked, stacked, and powered up, it’s time to get to the fun part by setting up the Quantum Myriad cluster through the included intuitive management tool. Quantum provides an easy-to-follow configuration and setup process to quickly get users up and running.

Initializing a small cluster deployment can take less than five minutes. You have entered a nine-step program.

Once the customer reviews the End User License Agreement (EULA), initializing the cluster is next. The fields have been filled in and are presented to verify accuracy. Clicking Next will take the user to the Create Password screen.

Once the password is created and confirmed, the next step is to set up NTP.

Most screens have been prepopulated if the system has the information. Now, it’s time to get the cluster up and running.

Quantum does all the work for this setup. The system has populated this screen based on the information pulled from the nodes. Admins can monitor the initialization process as each node gets configured.

Quantum Myriad provides a list of software available for the cluster. The customer can install all or just some of the software.

Once the software has been installed, it’s easy to either back out the install, go back a screen, or select Next to continue.

If all the information is correct, clicking Complete will apply all changes.

The cluster login screen is next.

If the login is successful, the user is presented with the status of the newly installed cluster.

Myriad Management and Administration

Myriad leverages a modern graphical user interface implemented in other Quantum GUI investments. After logging in successfully, the user is presented with an easy-to-read Dashboard with options to navigate quickly to the desired function or alert.

The image above is a capture of the Quantum Myriad management dashboard. The look and feel are consistent across other Quantum systems.

This view provides the user with an interactive image of the nodes in the system. Although not visible on this capture, the user can scroll up or down to get the status of the cluster’s Storage, Load Balancer, and Deployment nodes.

Navigating around the management GUI is simple. The panel to the left of the screen directs the user to specifics within the cluster, while the options along the top provide a more detailed view, such as the cluster performance detail shown above.

Latency can be viewed from within the Cluster display.

Selecting Drive from the left panel provides options to review individual drive details.

The management tool will provide the overall node component health status.

Closing Thoughts

Quantum Myriad emerges as a beacon of modernity and adaptability in the ever-evolving data management landscape. Its architecture not only scales and safeguards data with remarkable fluidity but also offers a forward-thinking development platform poised for seamless portability to the cloud and diverse underlying hardware systems.

With its vision for the future, Quantum has an ambitious development roadmap. Enhancements in protocol support and data services are on the horizon—including the much-anticipated object support—meaning Quantum is well-placed to redefine industry standards. Although Quantum has a vision and an aggressive roadmap, they’re entirely open to adjusting based on customer needs as adoption scales. They also have a bit of an engineering advantage here, as they can leverage their deep experience in storage and IP. This should allow Myriad to be dynamic as Quantum rapidly delivers more capabilities.

Furthermore, Quantum’s innovative application of modern cloud-like concepts, like leveraging OCP switches in the deployment process, underscores their commitment to speed and user-friendliness. All signs indicate that the future of data management with Myriad is secure and incredibly promising. We think Quantum Myriad is an exciting new entrant into fast file and object storage and are deeply impressed with the system’s simplicity and resiliency.

Learn more from Quantum and set up a demo or POC HERE

Myriad Data Sheet

Myriad Architecture

This report is sponsored by Quantum. All views and opinions expressed in this report are based on our unbiased view of the product(s) under consideration.

Engage with StorageReview 

Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed