VMware vSAN 8 Express Storage Architecture Announced

VMware took advantage of VMware Explore to announce the introduction of vSAN 8 with Express Storage Architecture (ESA). This next-generation storage architecture has been significantly overhauled to run high-performance workloads on the latest hardware. ESA is an optional alternative to the current vSAN original storage architecture and will be available to run on qualified hardware ESA-approved vSAN ReadyNodes.

Before getting into the details of ESA, it might be worth just looking at how far things have progressed in the past decade. Ten years ago, spinning disks were the norm, servers were keeping in line with Moore’s Law, and flash drives, although available, were expensive and didn’t have the capacities of HDDs. Cloud storage and computing were in the early stages, causing a stir.

Fast forward to today, and processors are struggling to keep up with the demands of modern applications. Flash technology is vastly more mature and priced lower than ever, with NVMe nearly replacing SATA and SAS. HDDs are now used more for archiving, and cloud storage is part of everyday life for most businesses. To meet the demands of the hardware, software, and cloud evolution, VMware is rolling out technology optimized for next-generation hardware.

VMware vSAN ESA

To keep up with these innovations, VMware developed the next-generation storage architecture that takes advantage of new hardware and software. ESA is optional and straightforward to implement.

The main advantage comes down to moving away from a cache and capacity design and now onto a single-tier architecture that others in the software-defined space have adopted. Performance is now spread out across the entire pool for read and write activity instead of flowing into the cache primarily. There is also the benefit that all SSDs are additive to the pool capacity, whereas previously, cache was only duplicative to buffer write activity.

There are immediate benefits to implementing vSAN 8 with ESA. ESA offers improved CPU efficiency, improved availability with reduced failure domains, simplified storage device provisioning with proactive insights, and RAID-6 data storage with RAID-1 performance.

The vSAN Express Storage Architecture can be incorporated into existing environments to make the transition simple and more transparent. Implementing vSAN ESA requires approved hardware and vSAN Advanced or Enterprise license. Once those requirements are met, VMs can be migrated via vMotion and SvMotion to the new cluster and managed using the same vCenter Server. All cluster types can co-exist in the same environment.

Additionally, vSAN ESA introduces a new patented VM log-structure file system (LFS). LFS is geared toward ingesting large amounts of incoming I/O, wrapping them with the associated metadata, and then storing the data space-efficiently.

Erasure Coding

A significant benefit of vSAN 8 ESA is the ability to deliver RAID-1 performance with the space efficiency of RAID-5/6 erasure coding. If a cluster is large enough, use RAID-6 to gain higher levels of resiliency and performance. Delivering this type of space efficiency with higher performance eliminates the decision process between performance and resiliency that most IT administrators face.

When using RAID-5, vSAN 8 ESA will dynamically adjust the erasure coding to fit the size of the cluster. Now environments with fewer hosts can easily use RAID-5 erasure coding.

Policy-Based Compression

Compression has moved up the stack and occurs at the top with the ability to compress data on a per-sector basis. Theoretically, this capability means a 4K block could be reduced from eight sectors down to a single sector delivering the potential increase in compression and also a more granular compression on a per block basis.

Security

Cluster-based encryption has been included with vSAN ESA. The encryption occurs at the top of the stack eliminating the need to encrypt and decrypt as data traverses the stack. This new functionality improves resource utilization when securing data.

Adaptive Network Traffic Shaping for Resync

With the vast improvement of performance at the device level, there was a greater potential to shift the bottleneck to the network. There needed to be a way to ensure a VM could offer available resources for VM I/O traffic in the event of contention with resynchronization traffic, ensuring data is balanced and adhering to storage policies. This function is dynamically adjusted on a host-by-host basis.

Capture Point-in-Time States

Built into vSAN 8, exclusive to ESA, is a new scalable, high-performance snapshot engine. The improved performance will deliver consistency, faster consolidation, and low stun times. The latest snapshot engine is seamlessly integrated with vSphere and third-party VADP backup solutions. SRM and vSphere replication performance has been improved, and it is now possible to take snapshots of objects in degraded states.

Using the UI, admins get a visual representation of how much data is consumed by a given snapshot.

Health Monitoring

Proactive insights are an improvement that affects both Original Storage Architecture and Express Storage Architecture. In the past, admins could not turn on the Customer Experience Improvement program due to company policy, logistics, or other anomalies. This has been rectified through proactive insights, with Health Check definitions fed through to the new environments. This feature is available for both OSA and ESA.

Buffers

Specific to vSAN OSA, the maximum write buffer size has been increased from 600GB to 1.6TB, nearly three times the original logical limits, allowing a larger number of write operations to be ingested at “burst rate” performance levels. This increase in buffer size will improve performance consistency for environments with larger working sets and reduces I/O demand on the capacity tier. This applies to vSAN OSA all-flash configurations.

This move is especially helpful considering so few SSDs are available in the small capacity vSAN had allowed for cache. Sure, larger drives could be used, but with standard capacities in new SSD releases starting in the 1.6TB-2TB class, the setup wasn’t efficient.

Final Thoughts

StorageReview has been a fan of the VMware vSAN product for years. From our first review of vSAN 6.0 back in 2015, we’ve been thoroughly impressed by the capabilities and ease of use VMware offered IT administrators looking to deploy storage. In that time, though, the market has evolved year over year.

VMware vSAN initially hit the market when HDDs ruled the space, and flash was in the early adoption phase. In those early days, a two-tier storage design made sense since SSDs were far too expensive and lacked the capacity to handle storage needs. Since then, flash has come down in price while also exponentially increasing capacity. Competitors such as Microsoft’s Azure Stack HCI platform have been offering flat architectures for a while now, which VMware has finally been able to match with vSAN 8 Express Storage Architecture.

From a licensing perspective vSAN 8 will require a vSphere 8 license. This is a bit of a letdown as we saw significant improvements in past vSAN releases that came out alongside point releases of vSphere. We saw this with vSAN 6.0 to vSAN 6.2, which had dramatic improvements in all-flash performance without requiring a new license. So if you want to take advantage of vSAN 8, you will need to fully overhaul the licenses in your environment.

Additionally, while these updates are great for vSAN itself and the supporting ReadyNode hardware, fully integrated solutions like Dell VxRail will benefit quickly as well. We’re enthusiastic to see what all of these changes mean when it comes to delivering more performance out of vSAN-based platforms.

VMware vSAN

Engage with StorageReview