EMC VxRack Node powered by ScaleIO Review

In June of 2013, EMC made a $200 million investment in ScaleIO, adding to their portfolio of storage offerings with a webscale software-defined scale-out solution. The ScaleIO stack has grown in flexibility and capabilities since then and now EMC offers ScaleIO in a variety of configurations to solve a wide variety of enterprise and service provider use cases. It's important to understand the wide variety concept as it pertains to ScaleIO. The software can be installed on any hardware or purchased as an engineered solution via VCE (VxRack System or VxRack Node). ScaleIO can run in two-layer mode where it's treated and performs just like a SAN, or it can run hyperconverged, combining compute and storage together. It can also run bare metal OS, integrated with all major hypervisors including VMware, Hyper-V and KVM, or OpenStack. Lastly, ScaleIO supports a wide variety of drive configurations including all-disk, all-flash and hybrid combinations in between, scaling near linearly as the number of nodes grows. The net result is that ScaleIO can deliver top-tier performance that can be consumed in just about every way possible with impressive economics.

EMC is doing its best to ensure ScaleIO is easy to consume. In May of last year EMC took down the barriers to trying ScaleIO; users can download a fully functional install that's good for non-production use without having to register. To switch from development to production is simply a licensing issue that's sold on capacity. The next step up is the VxRack Node which combines ScaleIO with validated commodity hardware and a single vendor contact for support. Nodes are optimized for capacity or performance and come in a few flavors. Our VxRack Nodes are the high-performance 2U 4-node PF100 chassis. EMC VxRack Node powered by ScaleIO takes the Nodes and packages them with Cisco networking and VCE services and support to deliver an easy to consume infrastructure that can scale out as needed to meet enterprise application demands or service provider needs. VxRack System 1000 is a relatively new offering, coming to market in general availability in the second half of last year.

ScaleIO comes with some impressive claims and years of in field implementation to back them up. From a scalability perspective, ScaleIO can scale from just 3 nodes to over 1000. Storage and compute resources can be added modularly as needed and ScaleIO aligns storage growth automatically for applications needs. As the storage increases so does the throughput and IOPS. ScaleIO uses I/O parallelism (every sever in the cluster processes I/O operations) to eliminate bottlenecks. EMC states that performance optimization is automatic and will have minimal to no impact on applications or users. EMC’s internal lab test showed performance of about 31 million IOPS in a 128 node cluster for a 100% Read workload. ScaleIO is also elastic in that resources can automatically be increased or decreased when needs arise. This along with the ability to scale performance with capacity makes it a much more attractive product from an investment perspective as savings increase as it scales up.

ScaleIO provides many critical features for data center workloads such as Read Flash Cache and Recoverpoint. Read Flash Cache is an auto-caching program that leverages PCIe technology to bump up overall performance. Recoverpoint adds a layer of data protection by allowing a restore to a specific point in time. Other benefits to ScaleIO include enabling the user to decide on whether to use thin or thick provisioning to improve storage efficiency. Users can set up protection domains that use a set of SDS as a backup for another set of SDS; this will help protect data in the result of multilevel failure. Through QoS users can set specific bandwidth and storage of a given SDC. Like XtremIO, ScaleIO also enables writable snapshots, snapshots of an existing volume that become a new unmapped volume in the system. These new volumes can be adjusted like any existing volumes on the system. Further data protection aspects include Rack Level High Availability (with only one copy of data written to each rack to protect data in the even of a hardware failure), and data masking that obfuscates volumes for higher protection.

The ScaleIO software consumption models offer a dynamic set of economics. If deployed on commodity hardware, the net cost of the solution either as two-layer or hyperconverged is affordable. The scalability of the solution in any deployment model allows for less hardware to be purchased to anticipate storage demand down the line, nodes of any type fold in and scale out as needed. This works out in the reverse too, as servers can be retired as needed, without requiring a forklift upgrade. In regards to the EMC VxRack Node powered by ScaleIO which we have for review in an 8-node all-flash configuration, pricing is extremely aggressive given the config and performance profile. At this time, the exact pricing is a bit fluid, but with the momentum VxRack is seeing and potential chassis and component savings buyers of the engineered solution stand to benefit. We also speculate that when the Dell acquisition closes, additional savings may be passed down to customers. Generally speaking though, buyers should expect to pay less than $200,000 for a 2U 4-node system like the ones we're reviewing. However, EMC offers many other options designed for different workloads that are far more cost effective.

EMC VxRack Node powered by ScaleIO (Performance Compute All Flash PF100) Specifications

Chassis – # of Node: 2U-4 node
Processors Per Node: Dual Intel E5-2680 V3, 12c, 2.5GHz
Chipset: Intel 610
DDR4 Memory Per Node: 512GB (16x 32GB)
Embedded NIC Per Node: Dual 1-Gbps Ethernet ports + 1 10/100 management port
RAID Controller Per Node: 1x LSI 3008
SSDs Per Node: 4.8TB (6x 2.5-inch 800GB eMLC)
SATADOM Per Node: 32GBSLC
10GbE Port Per Node: 4x 10Gbps ports SFP+
Power Supply: Dual 1600W platinum PSU AC
Router: Cisco Nexus C3164Q-40GE

Management and Usability

ScaleIO has an installed Windows-based (or CLI) management system and also supports vSphere, OpenStack and EMC's ViPR. In order to manage ScaleIO we used both its GUI as well as plugins through VMware's vSphere. The Windows GUI has three main tabs that more or less give admins the ability to monitor the entire system, Dashboard, Backend, and Alerts. The Dashboard gives users an overall view of the system including capacity (total, used, and free), I/O workload (including the read, write, and total of Bandwidth, IOPS, and I/O size), and gives users a readout of Rebalance and Rebuild (if any is currently happening). Along the bottom of Dashboard are tabs for SDCs, Volumes, Protection Domains, SDSs, and Management.

Clicking on the Backend tab gives admins an overall view of the system that is broken down by nodes and individual drives. There is also a readout of various aspects of performance, rebuild and rebalance, and alerts.

Through a dropdown menu admins can take a closer look at just about any aspect within the server including capacity usage, capacity health, rebuild and rebalance, application I/O, overall I/O, I/O bandwidth, state summary, configuration, device details, RAM read cache, as well as advanced function such as planned rebuilds, planned rebalancing, rebuild I/O priority, rebalance I/O priority, and network throttling.

Clicking on any of the items from the drop down screen highlights that aspect of ScaleIO. For instance, clicking on Overall I/O gives users the same screen only the bandwidth, IOPS, and I/O size are highlighted for each node and for each drive if uses expand the view that far.

Another option within Backend is to expand on device information to getting a more in-depth look at its properties. In this case we selected the SSD within the fourth node and were given a readout of capacity (which is further broken down into Thick, Thin, and Snapshot), the health of the drive, the performance, and activity.

The alerts tab is as it sounds, the readout of all alerts. The alerts give the severity, the item where the alert is happening with its name, as well as a readout of what the issue is.

Using the VMware vSphere Web Client gives us some similar views into ScaleIO. Through the Navigator admins can choose ScaleIO and see a version number along with a brief description of the software. Beneath the description are options such as install, deploy, register ScaleIO or more advanced tasks such as advanced setting, sever logs, and showing the SDC upgrade process. The vSphere plugin is one of the main areas to connect different ESXi hosts to ScaleIO storage or provision volumes from available storage pools.

Getting into the system we’re using (StorageReview) we see an overall summary with the name, mode, and state. There is also the ScaleIO Gateway, Primary MDM, Secondary MDM, Tiebreaker MDM, and Management Network Configurations.

Inside the storage pools we can see items we’ve setup up such as protection domains, volumes, and devices.

We can also go deeper into the cluster seeing what nodes are connected along with their capacity, health, whether or not RAM Read Cache is enabled, and its IP address.

This view can be further expanded to capture all of the individual drives as well.

Conclusion

EMC has taken an interesting approach with their ScaleIO software stack compared to other HCI players in the market. At first glance one might look at the offering as a basic HCI kit, with minimal rich data services and an over-simplified management suite. That is one of the biggest mistakes you can make though, as under the covers EMC's ScaleIO offering gives users an incredible amount of flexibility paired with the utmost attention to detail given to minimizing system overhead. Many HCI buyers are used to the traditional HCI deployment model such as VSAN or Nutanix where VMs reside on a given platform, consuming storage and compute off the same hardware. ScaleIO can operate like that, or if buyers want to follow a more traditional two-layered approach, ScaleIO can serve out storage across an IP-based network to virtualized and physical hosts alike. All of this comes at a price… which is surprisingly highly competitive against AFA or Hybrid SANs and other HCI kits.

As we dive into performance in our follow-up segments, one thing is clear; we've never seen any HA shared storage perform as fast as ScaleIO has in our lab. ScaleIO has shattered records in every situation so far, pushing our testing infrastructure to its limits. If previous storage topped out at 12 or 16 high-performance databases, ScaleIO topped out at 32 (we had to stop when we ran out of space). If previous storage systems slowly tapered off as loads increased, ScaleIO would take it into overdrive and show no signs of slowing down. If previous storage slowed down near full capacity, ScaleIO didn't flinch with capacity limits removed and utilization spiked to 99.2%. If those stats weren't crazy enough, system overhead during the aforementioned results frequently hovered around ~10-15% CPU per node with 5-6GB DRAM consumed. Some people might call the enterprise storage market a dull space, but the EMC ScaleIO solution has easily piqued our interest in a way few solutions have. Stay tuned as we take EMC's ScaleIO solution through its paces.

Pros