Quantum Myriad Software-Defined All-Flash Storage Platform Introduced

Quantum Myriad™ is a new all-flash, scale-out file and object storage software platform for modern enterprise workloads. The Myriad cloud-native architecture is an easy-to-use solution designed to overcome the limitations of hardware-centric designs, allowing customers to adapt to future storage needs while reducing the burden on IT staff.

Quantum Myriad is an entirely new offering from the company, slotting alongside a variety of other storage appliances and archiving solutions. Myriad is different in that it’s a shared-nothing architecture designed for the latest flash technologies to deliver consistent low-latency performance at any scale.

Even though it’s brand new, Quantum is launching with a robust set of inline data services like deduplication, compression, snapshots, clones, and metadata tagging designed to accelerate AI/ML data processing. Operationally, familiar and proven cloud technologies like microservices and Kubernetes are used to deliver cloud simplicity. Ultimately, Quantum has designed Myriad to be a zero-touch storage, cluster, and network management platform that is simple to expand, change, and shrink the system without disruption.

Brian Pawlowski, Quantum’s chief development officer, said:

“To keep pace with data growth, the industry has “thrown hardware” at the problem. Integrating feedback from customers, partners, and industry analysts on what’s missing from current solutions in market, we took a totally different approach with Myriad, and the result is the architecture I’ve wanted to build for 20 years. Myriad is incredibly simple, incredibly adaptable storage software for an unpredictable future.”

Quantum Myriad Software

Myriad software runs on standard x86 storage servers, making it easy for IT teams to adopt the latest hardware. This of course includes the latest in storage form factors, NAND technologies, and interfaces for future needs along with new I/O options and potential accelerator cards in the future.

Typical of most software, Myriad comprises many components that interact to provide internal functions and customer-facing services, in this case, storing and retrieving files and objects. The software components and functions are typical of a layered model, paralleling the flow of data through the system. Quantum depicts the layers as, from top to bottom, clients, data services, file systems, and transactional key/value store. Each layer and component is designed to reduce latency and maximize parallelism.

Looking at each component individually, it’s easy to see that running these in parallel makes sense, delivering better performance and functionality. Let’s break it down starting at the top of the stack.

Client components include the NFS server process, SMB, S3, proprietary services, and GPU-Direct services. The client components interface with an instance of the custom Myriad file system. An arbitrary number of file systems may be defined, each with one or more associated presentation layers. It is possible to have one file system defined with both NFS and S3 presentations enabling access to the same data as both files and objects. With no limit to the number of file systems, they can be provisioned to meet business needs.

The Data Services layer supports things like always-on inline deduplication and compression, snapshots, and clones. The File System layer is fully distributed and composable, where it might be one file system per user or app.

The Transactional key-value store is the central repository for all data and metadata. This is the heart of the system and is distributed across all the storage nodes in a Myriad cluster and linearly as nodes are added to the cluster. The key-value store stores data on the NVMe SSDs using unique-dynamic erasure coding (EC) scheme that adjusts to events in real-time, maintaining protection and maximizing efficiency.

Myriad Cluster Architecture

Myriad is designed as a flexible, self-healing, self-balancing software that allows in-service upgrades, automatically rebuilding and repairing data in the background while rebalancing data while the storage cluster adjusts to customer needs. With the intelligence to automatically detect, deploy, and configure storage nodes within a cluster so the cluster can scale, modify, or shrink without intervention, IT admins can focus on other important functions.

Although Myriad software is designed to run anywhere, the initial release is offered on Quantum iron. The Myriad cluster architecture is illustrated in the image below.

Myriad Management Portal

The secure online portal provides real-time monitoring of system health, performance, and capacity trends by connecting to Quantum cloud-based AI Operations software. There is no need to dedicate a management server to run the Myriad cluster. Myriad delivers simple data security and ransomware recovery with built-in snapshots, clones, snapshot recovery tools, and rollback capabilities.

Myriad Hardware Specifications

Quantum has defined the hardware requirements for each cluster component to make it easy to select the hardware of choice. As mentioned, the initial release will be offered on Quantum-provided appliances. The hardware requirements are straightforward and do not include anything out of the ordinary.

Storage Node Specifications

The Storage node is a standard server that provides compute and storage services. At GA, a cluster is composed of five to ten storage servers. The 1U appliance runs on a 64-core CPU with 512GB RAM, with support for PCIe Gen4. Each server supports ten single-ported TLC drives of 15.36 TB at GA. Networking connectivity is via four 100GbE ports, and management is delivered via one 1GbE RJ-45. The Quantum appliance features dual power and redundant fan assemblies.

The NVMe storage nodes are at the core of the cluster, home to the SSDs, and run Myriad software and services.

Load Balancer Node Specifications

This is a fairly generic switch that connects the cluster to the customer network and balances the inbound traffic to the cluster and within the cluster. The cluster requires two 1U switches with 32 100GbE network ports and one 1GbE RJ-45 management port. The switches from Quantum will have dual power and redundant fans.

Deployment Node Specifications

This node deploys software and configures other nodes in the cluster. The deployment node is not in the data path. Each cluster requires one deployment node. These nodes are configured with 48 1GbE ports, one 1GbE RJ-45 management port, and like the other appliances, has dual power and redundant fans.

The Deployment node is responsible for cluster deployment and maintenance, including initial installation, capacity expansion, and software upgrades.

Myriad Networking

Myriad nodes connect over 100GbE that’s designed to be simple. There is little for the customer to do except physically connect the network. Myriad nodes are automatically configured and require no additional hardware or customer interaction. Myriad provides an intuitive GUI for configuration and control functions, and SNMP and sFlow monitoring is also supported.

Cluster nodes are interconnected via two networks: a 100GbE lossless network with RDMA for data and management traffic and a 1GbE network for deployment tasks. At installation, there’s the choice to provide a non-routable subnet for the internal network or accept the default 172.16/16. The internal interfaces for all cluster nodes and Kubernetes pods are assigned addresses from this range. Any new nodes added to the cluster are automatically assigned an address from the selected subnet.

The Load Balancer nodes use NAT to isolate the internal cluster network from the data center network. Load Balancer nodes use Equal Cost MultiPath (ECMP) routing to spread incoming connections across the storage nodes. Myriad external networking uses Border Gateway Protocol (BGP) routing protocol making connections to an external network simple. Customer switches require Myriad-facing ports to be configured for BGP unnumbered with ECMP.

Myriad In The Cloud

Although the initial release is offered on dedicated appliances, the architecture is designed to deploy Myriad on a public cloud infrastructure like AWS. Cloud deployments for Myriad can take advantage of underlying cloud platform services, but the software and overall experience will be the same.

For cloud deployments, since Myriad software runs in a Kubernetes cluster, Myriad will use cloud Kubernetes services such as AWS EKS. The Myriad load balancer function will use cloud networking and load-balancing services.

Market Impact

Myriad expands Quantum’s portfolio of solutions for unstructured data. It’s suited for emerging use cases that require more performance and scale, including AI/ML, data lakes, rendering of VFX and animation, and other high-bandwidth and high IOPs applications.

While there are probably dozens of solutions that target these workloads, there are very few that have been developed from the ground up in modern times. It would have been easier for Quantum to take one of their existing solutions, apply a new layer of lipstick, like a container storage interface, and call it a new solution. But Quantum opted to take a substantive amount of risk to come to market with an entirely new solution, designed from the ground up. Quantum is pretty happy with the net result, while also acknowledging there’s more work to be done, to bring features to Myriad as customers start to get hands-on and accumulate their feedback.

Speaking of hands-on, we intend to visit a Quantum data center with the Myriad kit very soon, we’ve already scheduled a visit. For those who can’t wait for our full report, Quantum Myriad is available now for early-access customers and is planned for GA in the third quarter of this year.

www.quantum.com

Engage with StorageReview