The Fusion ioMemory SX300 is a third generation PCIe application accelerator with an emphasis on endurance and price-to-performance. The SX300 and its higher-performance PX600 sibling comprise Fusion’s new “Atomic Series” which is essentially one hardware platform with two different NAND overprovisioning schemes that result in different performance profiles for the two drives.
The Fusion ioMemory SX300 is a third generation PCIe application accelerator with an emphasis on endurance and price-to-performance. The SX300 and its higher-performance PX600 sibling comprise Fusion’s new “Atomic Series” which is essentially one hardware platform with two different NAND overprovisioning schemes that result in different performance profiles for the two drives.
The PX600 and SX300 are using the same controller platform and the same raw NAND, the core difference between the two is in the NAND provisioning. The SX300 is tuned for capacity and better cost metrics while the PX600 is tuned for endurance. Both drives offer similar performance profiles. The SX300 comes in 1.25TB, 1.6TB and 3.2TB capacities in low-profile form factors in addition to a 6.4TB full-height, half-length profile. This review features the 3.2TB edition of the SX300. All cards are offered with a PCIe 2.0 x8 interface.
The Fusion-io ioMemory SX300 comes with a five-year warranty up to the maximum endurance used for each card. Our review unit is the 3.2TB capacity card.
Fusion ioMemory SX300 Specifications
- Usable Capacity:
- 1.25TB (Model No: 1300)
- Read Bandwidth (GB/s): 2.6
- Write Bandwidth (GB/s): 1.1
- Ran. Read IOPS (4K): 195,000
- Ran. Write IOPS (4K): 285,000
- Endurance (PBW): 4
- 1.6TB (Model No: 1600)
- Read Bandwidth (GB/s): 2.6
- Write Bandwidth (GB/s): 1.1
- Ran. Read IOPS (4K): 195,000
- Ran. Write IOPS (4K): 285,000
- Endurance (PBW): 5.5
- 3.2TB (Model no: 3200)
- Read Bandwidth (GB/s): 2.6
- Write Bandwidth (GB/s): 1.2
- Ran. Read IOPS (4K): 215,000
- Ran. Write IOPS (4K): 300,000
- Endurance (PBW): 11
- 6.4TB (Model no: 6400)
- Read Bandwidth (GB/s): 2.6
- Write Bandwidth (GB/s): 1.2
- Ran. Read IOPS (4K): 180,000
- Ran. Write IOPS (4K): 285,000
- Endurance (PBW): 22
- 1.25TB (Model No: 1300)
- NAND Type: MLC (Multi Level Cell)
- Read Access Latency: 92μs
- Write Access Latency: 15μs
- Bus Interface: PCI-Express 2.0 x8
- Weight 5.2 ounces 7.25 ounces
- Form Factor: Low Profile Standard Height (1.25TB, 1.6TB, 3.2TB) Half-Length (6.4TB)
- Operating Systems
- Microsoft: Windows Server 2012 R2, 2012, 2008 R2 SP1
- Linux: RHEL 5/6, SLES 11, OEL 5/6, CentOS 5/6, Debian Squeeze, Ubuntu 12/13
- Unix: Solaris 11.1/11 x64, Solaris 10 U11 x64
- Hypervisors: VMware ESXi 5.0/5.1/5.5, Windows Server 2012 Hyper-V, 2012 R2 Hyper-V
- Power Requirements: 25W
- Temperature
- Operational: 0°C – 55°C
- Non-operational: -40°C – 70°C
- Air Flow: 300 (LFM)2
- Humidity: Non-condensing 5-95%
- Altitude
- Operational: -1,000ft to 10,000ft
- Non-operational: -1,000ft to 30,000ft
- Warranty: 3 years or maximum endurance used
Design and Build
The Fusion-io Atomic Series SX300 is a single-controller PCIe Application Accelerator that comes in HHHL and FHHL form-factors. For 1.2-3.2TB versions, the card has the smaller HHHL form-factor, which gives it a near universal fit in servers on the market. The larger capacity 6.4TB model needs a larger height for the additional NAND, although it still fits in most servers on the market, just not all slots.
The new Atomic Series SX300 cards are similar to the previous Application Accelerators from Fusion-io leveraging a FPGA controller, which is able to leverage host resources, which they claim offers lower latency performance being closer to the CPU. One small difference compared to the ioDrive2-series is that none of the newest models use two controllers (which were found in the Duo SLC and MLC products before). This helps save on power consumption, not to mention that it presents the user with a single pool of storage, versus two that they would need to stripe together.
Fusion-io has also done away with any external power connectivity on the SX300 cards, which was seen on the first- and second-generation models. The reason for this is older models could draw more power in higher performance modes, and some servers couldn’t function safely above minimum PCIe power spec. However, the current crop of servers on the market support much higher power demands, so Fusion-io included the ability to enable higher power modes through the slot itself.
Testing Background and Comparables
The Fusion-io ioMemory SX300 a single FPGA controller and Intel MLC NAND with a PCIe 2.0 x8 interface.
Comparables for this review:
- Fusion-io PX600 (2.6TB, 1x FPGA controller, MLC NAND, PCIe 2.0 x8)
- Fusion-io ioDrive2 (1.2TB, 1x FPGA controller, MLC NAND, PCIe 2.0 x4)
- Fusion-io ioDrive2 Duo (2.4TB, 2x FPGA controllers, MLC NAND, PCle 2.0 x8)
- Fusion-io ioDrive2 Duo (1.2TB, 2x FPGA controllers, SLC NAND, PCle 2.0 x8)
- Fusion-io ioScale (3.2TB, 1x FPGA controller, MLC NAND, PCIe 2.0 x4)
- Huawei Tecal ES3000 (2.4TB, 3x FPGA controllers, MLC NAND, PCIe 2.0 x8)
- Intel SSD 910 (800GB, 4x Intel EW29AA31AA1 controllers, MLC NAND, PCIe 2.0 x8)
- LSI Nytro WarpDrive (800GB, 4x LSI SandForce SF-2500 controllers, MLC NAND, PCle 2.0 x8)
- Memblaze PBlaze3H (2.4TB, 2x FPGA controllers, MLC NAND, PCIe 2.1 x8)
- Memblaze PBlaze3L (1.2TB, 1x FPGA controller, MLC NAND, PCIe 2.1 x8)
- Micron P320h (700GB, 1x IDT controller, SLC NAND, PCIe 2.0 x8)
- Micron P420m (1.6TB, 1x IDT controller, MLC NAND, PCIe 2.0 x8)
- OCZ ZD-XL Flash (1.6TB, 8x LSI SandForce SF-2500 controllers, MLC NAND, PCle 2.0 x8)
- Virident FlashMAX II (2.2TB, 2x FPGA controllers, MLC NAND, PCIe 2.0 x8)
All PCIe Application Accelerators are benchmarked on our second-generation enterprise testing platform based on a Lenovo ThinkServer RD630. For synthetic benchmarks, we utilize FIO version 2.0.10 for Linux and version 2.0.12.2 for Windows. In our synthetic testing environment, we use a mainstream server configuration with a clock speed of 2.0GHz, although server configurations with more powerful processors could yield even greater performance.
- 2 x Intel Xeon E5-2620 (2.0GHz, 15MB Cache, 6-cores)
- Intel C602 Chipset
- Memory – 16GB (2 x 8GB) 1333Mhz DDR3 Registered RDIMMs
- Windows Server 2008 R2 SP1 64-bit or CentOS 6.3 64-Bit
- 100GB Micron P400e Boot SSD
- LSI 9211-4i SAS/SATA 6.0Gb/s HBA (For boot SSDs)
- LSI 9207-8i SAS/SATA 6.0Gb/s HBA (For benchmarking SSDs or HDDs)
Application Performance Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our first three benchmarks of the ioMemory SX300 are therefore the MarkLogic NoSQL Database Storage Benchmark, MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload.
Our MarkLogic NoSQL Database environment requires groups of four SSDs with a usable capacity of at least 200GB, since the NoSQL database requires roughly 650GB of space for its four database nodes. Our protocol uses an SCST host and presents each SSD in JBOD, with one allocated per database node. The test repeats itself over 24 intervals, requiring between 30-36 hours total. MarkLogic records total average latency as well as interval latency for each SSD.
The ioMemory SX300 scored an average latency of 1.527ms when overprovisioned for best performance during the NoSQL benchmark, comparable to its sibling the PX600. Both Atomic drives scored among the best accelerators in this large dataset.
During the NoSQL benchmark, the ioMemory SX300 experienced a latency spike up to 13.79ms during a merge read operation early in the protocol, and a smaller spike to 11.84ms during a merge write operation. Neither spike was large enough to significantly affect overall performance during the benchmark.
Our Percona MySQL database test via SysBench measures the performance of OLTP activity. In this testing configuration, we use a group of Lenovo ThinkServer RD630s and load a database environment onto a single SATA, SAS or PCIe drive. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency over a range of 2 to 32 threads. Percona and MariaDB can make use of Fusion-io flash-aware application acceleration APIs in recent releases of their databases, although for the purposes of comparison we test each device in a “legacy” block-storage mode.
The Memblaze PBlaze3 platform has held the top position among PCIe flash drives we have benchmarked to date with the MySQL workload. At up to 16 threads, both of the Atomic ioMemory drives outperform the PBlaze3 comparables. Above 16 threads the PX600 and SX300 are only slightly edged out by the PBlaze3H.
Comparative results are similar for average latencies during the MySQL benchmark, with the PX600 and the SX300 outperforming the PBlaze3 drives under almost every workload.
The Fusion drives handle the worst-case latency scenario better than the PBlaze3 platform.
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Our SQL Server protocol uses a 685GB (3,000 scale) SQL Server database and measures the transactional performance and latency under a load of 30,000 virtual users.
In terms of transactions per second, the SX300 was able to keep pace with the comparable drives in our Microsoft SQL benchmark. Throughput is generally not a limiting factor for current-generation PCIe storage with the 30,000 user SQL Server benchmark.
The more important metric for evaluating performance in the Microsoft SQL Server benchmark is average latency. With a workload of 30,000 virtual users, the ioMemory SX300 showed its Fusion-io colors as part of a cohort of Fusion drives which have held the best PCIe performances to date in the SQL Server benchmark.
Enterprise Synthetic Workload Analysis
Flash performance varies throughout the preconditioning phase of each storage device. Our synthetic enterprise storage benchmark process begins with an analysis of the way the drive performs during a thorough preconditioning phase. Each of the comparable drives are secure erased using the vendor’s tools, preconditioned into steady-state with the same workload the device will be tested with under a heavy load of 16 threads with an outstanding queue of 16 per thread, and then tested in set intervals in multiple thread/queue depth profiles to show performance under light and heavy usage.
- Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Our Enterprise Synthetic Workload Analysis includes two profiles based on real-world tasks. These profiles have been developed to make it easier to compare to our past benchmarks as well as widely-published values such as max 4k read and write speed and 8k 70/30, which is commonly used for enterprise hardware.
- 4k
- 100% Read or 100% Write
- 100% 4k
- 8k 70/30
- 70% Read, 30% Write
- 100% 8k
With a 4k preconditioning workload of 100% write operations in Linux, the SX300 with high performance overprovisioning experienced a competitive burst period. Otherwise, both SX300 overprovisioning schemes experienced lower performance as it neared steady-state.
In a Windows environment, the SX300’s 4k write throughput is at its best in Windows, showing a slightly edge over its Linux performance.
Average 4k write latencies for the stock SX300 overprovisioning scheme eventually rise to the highest among the comparable drives in Linux, but high performance overprovisioning drives the SX300 into the middle of the pack.
In Windows, the high performance overprovisioning also finds the SX300 at the middle of the pack in terms of average latency for write operations during 4k preconditioning.
Despite momentary performance problems, such as a 20ms latency measured during 4k preconditioning with the stock overprovisioning, the SX300 performed competently in terms of average latency with the Linux testbed.
The ioMemory SX300’s maximum latency results during 4k write preconditioning were more dramatic in Windows. The stock configuration’s maximum latencies regularly spiked above 200ms, while the high performance overprovisioning still regularly exceeded 150ms for maximum measured latency.
Plotting standard deviation calculations provides a clearer way to compare the amount of variation between individual latency data points collected during a benchmark. The standard deviation results for the SX300 put it at the middle of the pack of comparables during the write preconditioning for the 4k benchmark in Linux.
In Windows the situation is much the same. The SX300 achieves respectable standard deviation results during 4k preconditioning but nothing as impressive as the Huawei ES3000 or the Micron P430m. The SX300 also falls behind its more write-performance oriented Atomic sibling the PX600.
With preconditioning complete, the ioMemory SX300 reaches fourth-best write performance when overprovisioned for high performance on the 4k Linux benchmark. High performance overprovisioning has less an effect on read performance, which puts the SX300 at or near the bottom of the pack for both configurations.
The SX300 was able to slightly outperform the PX600 in terms of read performance with both drives overprovisioned for high performance during the 4k benchmark in Windows. Write performance for the high performance configuration was a less competitive 138,897IOPS.
The 4k benchmark average latency results in Linux for the SX300 were at or near the highest among the comparables regardless of whether the drive was overprovisioned for additional performance.
Average latencies in Windows were a more competitive for the SX300, particularly with high performance overprovisioning.
High performance overprovisioning only has a notable effect with write operations for the ioMemory SX300 during the 4k benchmark in Linux. Enabling the high performance configuration decreased the maximum latency from 15.36ms to 12.96ms.
In Windows both the SX300 and the PX600 struggled from larger latency spikes than during the Linux 4k benchmark.
Standard deviation calculations continue to show the SX300 as a reliable but unexceptional performer among the comparable drives during the 4k Linux benchmark.
In Windows the SX300 lies in the middle of the pack in terms of the consistency of its latency results; regardless of momentary spikes into the hundreds of milliseconds, the SX300’s overall latency performance is more competitive against its peers in Windows than in Linux.
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. The first set of charts present measurements made during the preconditioning process. In Linux, the burst throughput performance of the SX300 is in the top half of the comparables. After the burst period, the SX300 overprovisioned for high performance is able to maintain a competitive position as it approaches steady state near 140,000IOPS.
During the Windows preconditioning for the 8k 70/30 benchmark the SX300 also performed well in terms of throughput during the burst period in both configurations. The high performance configuration maintained a third place finish as the drives neared steady state.
In terms of average latency, the high performance SX300 configuration trailed the two ioMemory PX600 configurations and the Huawei ES3000 during Linux preconditioning.
In Windows, the high performance SX300 configuration edged out the average latency results of the stock PX600 to approach steady state in the third best position.
The 8k maximum latencies recorded in Linux for the two SX300 configurations, much like the two PX600 configurations we also benchmarked, were better than all other comparables except for the Huawei ES3000 and the Micron P420m.
The Windows preconditioning measurements for the 8k benchmark demonstrate the difficulty that both the SX300 and the PX600 have in handling maximum latency here compared to the Atomic ioMemory platform’s performance in Linux. Both SX300 configurations experience spikes above 235ms.
The standard deviation calculations for latencies measured in Linux during 8k preconditioning place the SX300 in the middle of the road in terms of consistency among the comparable drives, with the SX300 stock overprovisioning scheme and when overprovisioned for high performance.
The Windows standard deviation results for the SX300 show more cycling of values than in Linux but also place the SX300 in the middle of the pack in terms of the consistency of its latency results.
With 8k 70/30 preconditioning complete on the Linux testbed, the high performance SX300 overprovisioning is able to nearly mirror the performance of the stock PX600 in a dead heat for third place across each of the variations in workload.
In Windows, the SX300 also fares well when overprovisioned for higher performance. Under the maximum load of 16 threads and a queue depth of 16, the high performance SX300 configuration held third place with at 129,852IOPS.
Average latency results in Linux during the 8k 70/30 benchmark place the SX300 in a favorable light, particularly configured with high performance overprovisioning. Thus configured, it falls just behind the PX600 among the best of the drives we have benchmarked in this class.
The 8k 70/30 benchmark is less competitive in Windows for the SX300, where the stock overprovisioning only consistently outperforms the stock ioDrive Duo MLC configuration among the comparables.
Maximum latency measurements for the 8k 70/30 benchmark on our Linux platform are impressive, with no notable spikes and overall maximum latency results very similar to the PX600.
In Windows the SX300 and PX600 both struggle with latency spikes that introduce drama to the maximum latency chart for the 8k 70/30 benchmark. The SX300 configuration with stock overprovisioning spiked to 249.35ms of latency with 16 threads and a queue depth of 4.
The standard deviation results for the Linux 8k benchmark show the high performance SX300 overprovisioning moving in tandem with the stock PX600 configuration.
The latency spikes revealed in the Windows maximum latency chart disappear into the averages for the standard deviation results, The SX300 gives a consistent if not noteworthy latency performance with the 8k 70/30 workloads regardless of occasional spikes plotted in the maximum latency results.
Conclusion
The Fusion-io (SanDisk) ioMemory SX300 is meant to be the high-capacity offering of the new, simplified ioMemory Atomic Series lineup. Its performance is consistent if not noteworthy across a range of enterprise storage benchmarks. For a drive engineered for more read-intensive workloads and moderate endurance, its write performance can be more competitive than expected. Fusion-io differentiates the PX600 and the SX300 largely based on over-provisioning which affects endurance and capacity, as well as the performance in some cases. Thanks to a heavier over-provisioning level from the factory, the PX600 is able to maintain greater write performance and endurance while giving up some capacity. The SX300 gets more capacity and a better cost/GB profile but gives up endurance and in some cases performance, to it’s PX600 sibling.
In synthetic benchmarks such as the 4k and 8k 70/30 test, the full capacity of the drive is tested. In stock settings this is 3.2TB for the SX300 and 2.6TB for the PX600. The PX600 has an upper hand when it comes to write performance because the drive has more NAND available to handle background activities. If the SX300 were dropped in capacity using ioSphere to match the PX600, the results would be nearly identical.
In application tests where the dataset being tested is finite and the same size across all products being reviewed, performance is nearly identical for both Atomic Series drives. In the MarkLogic NoSQL test both drives were 0.003ms apart over a 30+ hour test. In the Microsoft SQL Server TPC-C test, both drives tied with 3ms average latency recorded, with a slight TPS lead by the SX300. In Sysbench testing we saw the same thing where both drives overlapped each other’s performance, although the PX600 maintained another slight lead.
When it comes time to make a buying decision, the difference between the cards really comes down to the endurance levels which each is warrantied for rather than performance. The PX600 offers 12, 16, 32 and 64PB write endurance across four capacities, while the SX300 is rated for 4, 5.5, 11, and 22PB. On the smaller cards that’s a 4x difference that narrows to just under 3x for the biggest models. For those that have more read-heavy workloads and can do without the extra endurance, the SX300 brings more capacity and a better value per GB to market.
Pros
- Performance on par with prior gen, despite NAND die shrink challenges
- Excellent drive management software
- Tuned for application performance and reasonable endurance
Cons
- Some peak latency issues in Windows vs Linux
Bottom Line
The Fusion ioMemory SX300 offers an excellent performance profile for latency sensitive workloads that tend to be more read-centric. Customers can take advantage of the better pricing/GB, thanks to the increased capacity over the PX600, while sacrificing little in terms of performance.
