When we first reviewed the OCZ Z-Drive R4 last year, the PCIe SSD amazed us with it’s class leading performance numbers, as well as its ridiculous IOPS/$ performance. It was so fast in fact, that it made us re-develop our testing platform to better accommodate the burgeoning PCIe SSD space, dedicating a testing platform specifically for enterprise storage solutions. In this re-review, we look at the Z-Drive R4 in our enterprise testing environment to better emulate what enterprise users should expect out of the R4.
When we first reviewed the OCZ Z-Drive R4 last year, the PCIe SSD amazed us with it’s class leading performance numbers, as well as its ridiculous IOPS/$ performance. It was so fast in fact, that it made us re-develop our testing platform to better accommodate the burgeoning PCIe SSD space, dedicating a testing platform specifically for enterprise storage solutions. In this re-review, we look at the Z-Drive R4 in our enterprise testing environment to better emulate what enterprise users should expect out of the R4.
As a bit of a refresher, the OCZ Z-Drive R4 is perhaps the most flexible PCIe SSD storage solution in the market, in that it’s available in several iterations. The full-size card comes in capacities up to 3.2TB; there’s also a half-size option that is available up to 1.2TB. Both cards leverage SandForce SF-2200 controllers (eight controllers on the full card, four on the half card), coupled with OCZ’s SuperScale storage accelerator, which minimizes CPU overhead and power consumption. Other highlights include no need for additional power and the Z-Drive R4 is bootable, removing the need for a system drive. Enterprise buyers can also upgrade to the RM-series which offers SandForce SF-2500 controllers as well as power-protection to flush data to NAND in the event of a power failure. This review is of the 1.6TB full-height CM88 Z-Drive R4.
The big highlight of the Z-Drive R4 and perhaps the key value point is around the NAND used in this PCIe SSD. OCZ uses consumer MLC NAND in the Z-Drive R4, which makes it the lowest cost enterprise PCIe solution in the market. That doesn’t mean it gives up any performance though, the full height card delivers read speeds of up to 2,800 MB/s, writes of 2,800 MB/s and 4k random write IOPS of 410,000 and 8k IOPS of 275,000. The drives also offer plenty of endurance, ranging from 7.5PB in the 300GB half-height card to 80PB in the 3.2TB full height card.
CM88 (Full-Height) Specifications:
- Capacities
- 800GB – ZD4CM88-FH-800G
- Endurance – 20PB
- 1.6TB – ZD4CM88-FH-1.6T
- Endurance – 40PB
- 1490GB Usable
- 3.2TB – ZD4CM88-FH-3.2T
- Endurance – 80PB
- 800GB – ZD4CM88-FH-800G
- Performance
- Max Read up to 2,800 MB/s
- Max Write up to 2,800 MB/s
- Random Write Operations(4kB) 410,000 IOPS
- Random Write Operations(8kB) 275,000 IOPS
- PCI Express Gen. 2 x8
- PCIe full height, 3/4 length compliant
- OCZ SuperScale Storage Controller
- NAND Controller: 8 x SandForce SF-2200 SSD Processors
- Dimensions (L x W x H): 242 x 98.4 x 17.14 mm
- Weight: 283g
- Power Consumption: 23W Idle, 26W Active
CM84 (Half-Height) Specifications
- Capacities
- 300GB – ZD4CM84-HH-300G
- Endurance – 7.5PB
- 600GB – ZD4CM84-HH-600G
- Endurance – 15PB
- 1.2TB – ZD4CM84-HH-1.2T
- Endurance – 30PB
- 300GB – ZD4CM84-HH-300G
- Performance
- Max Read up to 2,000 MB/s
- Max Write up to 2,000 MB/s
- Random Write Operations(4kB) 250,000 IOPS
- Random Write Operations(8kB) 160,000 IOPS
- PCI Express Gen. 2 x8
- PCIe half height, half length compliant
- OCZ SuperScale Storage Controller
- NAND Controller: 4 x SandForce SF-2200 SSD Processors
- Dimensions (L x W x H): 168.55 x 68.91 x 17.14 mm
- Weight: 131g
- Power Consumption: 14.5W Idle, 16W Active
Additional Specifications
- Synchronous Consumer MLC NAND Flash
- OCZ VCA 2.0 Architecture
- TRIM/SCSI unmap (requires OS support)
- Power Fail Protection with DataWrite Assurance Technology
- Encryption: 128-bit & 256-bit AES-compliant
- ECC Recovery
- SMART support w/ enterprise attributes
- MTBF: 2,000,000 hours
- 3-Year Warranty
- Compatible with Windows 7, Windows Server 2008, Linux Red Hat Enterprise 6.1
- Operating Temp: 0°C ~ 70°C
- Storage Temp: -45°C ~ 85°C
Synthetic Benchmarks
The OCZ Z-Drive R4 uses Intel’s 25nm standard MLC NAND, eight SandForce SF-2282 controllers and a PCIe 8x interface; our review unit is 1.6TB. The comparables used for this review include the following recently tested enterprise PCIe SSDs: LSI WarpDrive SLP-300 (300GB, Six SandForce SF-1564 controllers, Micron 34nm SLC NAND, PCIe 8x) and Fusion-io ioDrive Duo (640GB, Two Xilinx Virtex 5 controllers, Samsung 3xnm MLC NAND, PCIe 8x). All enterprise SSDs are benchmarked on our enterprise testing platform based on a Lenovo ThinkServer RD240. All IOMeter figures are represented as binary figures for MB/s speeds.
Given the higher performance capabilities of a PCIe SSD like the OCZ Z-Drive R4, we slightly modified our standard testing methods beyond how we stress-test single enterprise SATA/SAS SSDs. To fully saturate the card we had to increase the I/O load through multiple managers and workers in IOMeter, otherwise we would not see the full potential of this drive. Our method allowed us to still work with the drive in an unformatted setup with two managers and two workers interfaced with the same 5GB LBA segment.
We split the synthetic IOMeter testing portion of this review into two parts. The first is our standard low queue-depth tests, which are performed at a QD=1 level on single drives and on PCI-e SSDs at a level of QD=4 given the manager/worker thread count. The initial tests are more in line with single-user environments, whereas higher queue depth ranges in the second half are more like what the card would see in a server with I/O requests stacked up.
To see how well it performed in straight-line performance we used IOMeter with a 4K-aligned sequential 2MB transfer test, with an effective queue depth rate of 4. OCZ lists a peak sequential transfer speed of 2,800MB/s read and 2,800MB/s write for the full-height Z-Drive R4.
In a straight line measurement the 1.6TB Z-Drive R4 measured 2.863MB/s read and 2,557.2MB/s write.
Our next test looks at random large-block transfers, but still keeping the 2MB transfer size.
Switching to a random large-block transfer the performance didn’t let up that much, with read speeds dropping to 2,822MB/s and write speeds leveling off at 2,493MB/s.
Next we look at both low queue-depth 4K random read/write as well as peak queue-depth figures for all of the PCI-e SSDs we have tested in our lab.
At a low queue depth the OCZ Z-Drive R4 slips behind the LSI WarpDrive in 4K read speed, but surpasses it in write speed. Later in the review where we start to look at scaled performance at higher queue depths it is clear that these multi-controller drives need a multi-threaded environment to stretch their legs.
With additional controllers under the belt, the QD=4 performance of the OCZ Z-Drive R4 showed stronger 4K write performance than the LSI WarpDrive, with equates to faster 4K average latency times. The differences in peak latency can probably be chalked up to NAND and controller differences, especially with the Z-Drive R4 having eight SandForce SF-2200 processors to sync together compared to just two controllers on the ioDrive Duo or six on the LSI WarpDrive. Max latency topped out at 35.38ms in our test.
The next half of our synthetic benchmarks are ramped tests, covering performance from early queue depth levels to either a max of 64 (QD=256) or 128 (QD=512). This section also includes our server profile tests, which from the start are designed to show how well enterprise products perform under demanding mixed server loads.
Our first ramped test looks at random 4K read performance as it scales from an effective queue depth of 4 to 256.
Starting off the Z-Drive R4 trails slightly behind the LSI WarpDrive until a queue depth of 8 (effective 32) where it then leaps above, peaking at 314,000 IOPS.
Looking at the same test with random 4K write activity, we measure performance again from an effective queue depth of 4 to 256.
The OCZ Z-Drive R4 maintained a small lead over the WarpDrive through QD2 (effective 8) before it quickly reached its peak at 355,000 IOPS.
Our last group of standard synthetic benchmarks looks at scaled performance using our server profiles in IOMeter. These tests measure performance from a low queue depth to a max of 128 (QD=512). This section is designed to show how well enterprise products perform under different demanding mixed workloads. The OCZ Z-Drive R4 easily dominated this area, leveraging eight SF-2000 controllers, versus six SF-1500 controllers in the LSI WarpDrive.
Enterprise Synthetic Benchmarks
Flash performance changes the longer you write to a drive and speeds taper down until the drive hits its steady state speed. In an enterprise setting, initial burst is hardly relevant if after one hour of use the drive won’t see that speed again. This is where steady state benchmarking comes in, showing how the drive performs when under a 24/7 load. For this reason, all of the following benchmarks were pre-conditioned and recorded in a steady state mode.
We used our StorageReview Enterprise Testing Environment to benchmark the OCZ Z-Drive R4; accurately representing its capabilities in an enterprise environment. The enterprise testing platform is based around a Lenovo ThinkServer RD240, equipped with dual Intel Xeon X5650 processors, running Windows Server 2008 R2. All IOMeter figures are represented as binary figures for MB/s speeds.
Our first test looks at the speed in a sequential write environment with large block transfers. This particular test uses a 2MB transfer size with IOMeter, with 4k sector alignment and measures performance with a queue depth of 4.
With incompressible data the 1.6TB Z-Drive R4 leveled off with a steady-state large-block sequential transfer speed of 2,200MB/s read and 1,443MB/s write.
Moving to a random access profile, but still maintaining a large block transfer size of 2MB, we start to see how performance varies in a multi-user environment. This test keeps the same queue depth level of 4 that we used in the prior sequential transfer benchmark.
Maintaining a large-block transfer but switching to random access from sequential, the OCZ Z-Drive R4 maintained a read speed of 2,223MB/s and a write speed of 426MB/s
Our next test looks at 4K random write performance at a static queue depth of 32 and results are recorded and averaged once the drives have reached steady-state. While IOPS performance is a good metric to measure steady-state performance, another key area of interest is around average and peak latency. Higher peak latency figures can mean certain requests can backed up under heavy continuous access.
In this test we include both single-manager, single-worker random 4K performance at QD32, as well as 1 manager 4 worker results. Single-worker speeds measured 48,129 IOPS and 188MB/s, while four worker speeds measured 59,904 IOPS and 234MB/s.
Our final section of enterprise synthetic benchmarks covers steady-state performance in our server profile tests. They have a strong preference towards read activity, ranging from 67% read with our database profile to 100% read in our web server profile. Since the Z-Drive R4 utilizes SandForce processors that can compress data for faster speeds, we measured steady-state performance with 0% and 90% compressibility. This shows the two polar sides of real-world conditions where some data in a given environment can be repeating and compressed. Since the OCZ Z-Drive R4 showed much higher performance levels at increased queue depth loads in our burst tests, we included both single-manager/single-worker and single-manager/four-worker benchmarks for our steady-state tests.
Our first sever profile covers database conditions, with a 67% read and 33% write workload mix primarily centered on 8K transfer sizes.
With a lower-intensity load, the OCZ Z-Drive R4 performed roughly on par with the LSI WarpDrive in our Database profile. To show its full potential, as we found in our burst measurements, we increased the load by a factor of four, where the Z-Drive R4 showed stronger performance. At its peak, the R4 measured 63,470 IOPS with incompressible data and 158,902 IOPS with 90% compressible data.
The next profile looks at a file server, with 80% read and 20% write workload spread out over multiple transfer sizes ranging from 512-byte to 64KB.
In our file server profile with a single worker, the OCZ Z-Drive R4 measured 45,211 IOPS with incompressible data and 69,826 IOPS with 90% compressible data. Increasing the load to four workers, speeds picked up substantially to 82,589 IOPS with compressible data and 120,788 IOPS with 90% compressible information.
Our web server profile is read-only with a spread of transfer sizes from 512-byte to 512KB.
In our Web Server scenario with read-only transfers, we measured speeds topping at roughly 50-51,000 IOPS with a single worker load from both compressible and incompressible data-types. Increasing the load to four workers, transfer speeds stayed at 50,961 IOPS with incompressible data, while average latency jumped from the higher queue depths. With 90% compressible information the Z-Drive R4 was able to increase its performance to 83,257 IOPS, and lower its average latency to 1.537ms.
The last profile looks at a workstation, with a 20% write and 80% read mixture using 8K transfers.
In our Workstation profile, the Z-Drive R4 exhibited similar performance to our Database test, where it required higher queue depths to out-perform the LSI WarpDrive at a given queue depth. With a single worker load the OCZ Z-Drive R4 measured 48,077 IOPS with incompressible data, speeding up to 75,273 IOPS with 90% compressible information. With a four-worker load the Z-Drive R4 had transfer speeds measuring 82,079 IOPS with incompressible data and 145,249 IOPS with 90% compressible data.
Real-World Enterprise Benchmarks
Our enterprise trace covers a Microsoft Exchange mail-server environment. We captured the activity of our StorageReview mail server over a period of a few days. This server hardware consists of a Dell PowerEdge 2970 running Windows Server 2003 R2 environment operating off of three 73GB 10k SAS hard drives in RAID5 on the Dell Perc 5/I integrated controller. The trace consists of many small transfer requests, with a strong 95% read load with 5% write traffic.
In our real-world mail server scenario the OCZ Z-Drive R4 measured an average speed of 166,265 IOPS or 1,327MB/s.
Conclusion
Several months later, our re-review has does nothing but increase our admiration for what OCZ has done with the Z-Drive R4. It’s a fantastic general purpose PCIe SSD, delivering fantastic performance, with plenty of endurance, even with standard grade MLC NAND. For users that want a Z-Drive R4 for a specific use, OCZ offers plenty of configuration options, including specialized models. Since the Z-Drive R4 first launched, OCZ announced the Z-Drive R4 CouldServ edition, which is targeted toward cloud computing applications, offering up to a massive 16TB utilizing sixteen SandForce SF-2581 controllers capable of speeds topping 1.4 million IOPS and 6,000 MB/s.
When compared to other PCIe solutions in the market, the Z-Drive R4 does very well when it comes to heavily multi-threaded traffic. Under a significant load, the R4 can surpass other competing options, but if its not being completely utilized there are some scenarios where performance is at or under other models. In areas such as random 4K read speed, the Z-Drive R4 had to be above an effective QD level of 32 to climb above the LSI WarpDrive. In our steady-state Workstation and Database profiles, we found that the R4 couldn’t be used to its fullest potential unless it had a multi-worker environment with certain compressible workloads. It really comes down to analyzing your current workloads and finding the best PCIe solution that fits application requirements. In certain performance envelopes, different PCIe solutions offer better results, which is also consistent with what we’ve seen from the wide variety of 2.5″ form factor enterprise SSDs that have strengths in specific areas.
For applications that can take advantage of the OCZ Z-Drive R4, there are few, if any, products that can compete at the same level. Leveraging consumer MLC NAND in certain configurations, the Z-Drive R4 can match or exceed the performance of other enterprise PCIe solutions, and still offer compelling endurance figures with $/GB and IOPS/$ few can beat. For those that demand a higher level of endurance, requiring SLC-NAND and increased power protection, OCZ has you covered as well with their Z-Drive R4 RS-series. With countless variants the Z-Drive R4 offers class-leading performance with a custom configuration for nearly any application.
Pros
- Incredibly fast, more than twice in some scenarios than the competition
- Priced very competitively at $7/GB, with many configuration options
- Bootable, with Windows and Linux driver support
Cons
- Needs higher levels of multi-threaded traffic to fully utilize eight SandForce controllers
Bottom Line
The OCZ Z-Drive R4 offers a configuration for everyone, and delivers great IOPS/$ out of their standard MLC NAND offering. For the right workloads the Z-Drive R4 is nearly impossible to beat in performance, price or flexibility.
