The EchoStreams FlacheSAN1L-D is a storage-dense 1U form factor server that houses up to 20 7mm SATA SSDs. This ultra-high performance flash server is pitched as sustaining 112Gbps of throughput and up to 1 million plus IOPS with 20 SSDs. The platform supports dual Intel Haswell CPUs, up to 512GB of DDR4 RAM, and offers two PCIe Gen3 slots for expansion. The EchoStreams FlacheSAN1L-D has use cases both in the traditional data center and several that couldn’t be further away from those friendly confines. The combination of storage density, performance and cost make this unique platform one of the more flexible available.
The EchoStreams FlacheSAN1L-D is a storage-dense 1U form factor server that houses up to 20 7mm SATA SSDs. This ultra-high performance flash server is pitched as sustaining 112Gbps of throughput and up to 1 million plus IOPS with 20 SSDs. The platform supports dual Intel Haswell CPUs, up to 512GB of DDR4 RAM, and offers two PCIe Gen3 slots for expansion. The EchoStreams FlacheSAN1L-D has use cases both in the traditional data center and several that couldn’t be further away from those friendly confines. The combination of storage density, performance and cost make this unique platform one of the more flexible available.
Since the FlacheSAN1L-D can support 20 SSDs it works well as a Tier-0 storage platform, giving low latency performance to applications that tend to be more read intensive. The read centric all-flash configuration is ideal for many database applications, digital media streaming, content editing and as a data cache. Due to its small form factor the FlacheSAN1L-D can also be used in some unique, more mobile use cases. Something this small could be taken into the field for collecting geological data for industries such as oil and gas where conditions are unfriendly to traditional HDDs and data ingest and field analysis is of utmost concern.
EchoStreams also offers a few other flash server models in their FlacheStreams line: the FlacheSAN1S is a 1U, Intel Xeon UP server model, the FlacheSAN1L is a 1U, Intel Xeon server model, and the FlacheSAN2 is a 2U model that has 48 bays and supports up to 512GB of RAM.
The EchoStreams FlacheSAN1L-D comes with a 1-year parts replacement warranty. For this review we used 20 Toshiba HK3R2 SATA SSDs for storage.
EchoStreams FlacheSAN1L-D specifications:
- From factor: 1U
- Supported CPU: Dual Socket Intel Xeon processor E5-2600 v3 series
- Chipset: Intel C612 chipset
- Supported RAM: Supports DDR4 2133/1866/1600 LR DIMM, Registered, ECC,16 x DIMM slot
- I/O interface: 4 x USB2.0, 1xVGA, 2x1Gb Intel i350, 1xRJ45 MGMT port, and 1 x Serial DB
- Expansion Slots: 1x PCIe Gen3 x16 FHHL, 1x PCIe Gen3 x16 LP
- Storage: 20 bays
- SSD: 2.5″ 7mm 3Gbps/6Gbps SATA
- Compatible OS:
- Microsoft Windows
- Server 2008 R2 SP1 (64 bit)
- Server 2012 (64 bit)
- Server 2012 R2 (64 bit)
- Linux
- RedHat Enterprise Linux Server 5.10/6.5 (32 / 64 bit)
- CentOS 5.10 / 6.5 (32 / 64 bit)
- SUSE Enterprise Linux Server 11 SP3 (32 / 64 bit)
- FreeBSD 9.2 (32 / 64 bit)
- Fedora core 19 (64 bit)
- Ubuntu 12.04.2 (64 bit) / 12.10 (64 bit)
- Hypervisors
- VMware ESXi 5.x
- Microsoft Hyper-V
- Microsoft Windows
- Server Management: IPMI with iKVM MGMT
- Cooling: 6x 40mm easy-swap high-speed fans
- Power: 1+1 400W AC/DC or -48VDC/DC high efficiency redundant power supplies
- Dimensions (LxWxH): 23”x19”x1.75”
- Weight: 40lbs (diskless)
- Environmental:
- Operating Temperature: 0°C to 35°C
- Non-Operating Temperature: -20°C to 70°C
- Humidity: 5% to 95% non-condensing
Design and build
The EchoStreams FlacheSAN1L-D is a 1U form factor, rack mount server. Looking at the front of the system, to the right is the display/control panel with the power button in the upper left and an ID button in the upper right, both with LED indicator lights. There are also LED lights for 4 LAN beneath the power and ID buttons. Beneath the LAN lights are the reset button on the lower left and a system warning light on the lower right. The rest of the front panel is made up of the twenty tool-less drive bays.
Going around to the rear of the device there are two redundant power sources on the left hand side. To the far left is the alarm mute button. Traveling from left to right, there are 2 USB 2.0 ports, a VGA port, a serial DB9 port, the RJ45 MGMT port and two Intel i350 1Gb ports.
To remove the 7mm drive trays simply press the tray latch and pull the drive tray out. To install new SSDs, push the drive lock latch inward, pull the drive hinge up, slide the new SSD in under the hooks (IO connector side goes in first), snug the drive in place and snap the hinge back in place. Slide the tray into the enclosure and lock in place. (The tool-less aspect of the tray is for SSDs only; installing an HDD requires M3 screws).
Testing Background and Comparables
We publish an inventory of our lab environment, an overview of the lab’s networking capabilities, and other details about our testing protocols so that administrators and those responsible for equipment acquisition can fairly gauge the conditions under which we have achieved the published results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing.
We will be comparing the EchoStreams FlacheSAN1L-D to the Infortrend ESDS S16F-G2652-4 with SanDisk Optimus Eco SSDs, Huawei Dorado2100 G2, and the Fujitsu Eternus DX200F. While the initial comparable group might seem odd since they are all arrays versus a storage server, the market this platform is being sold into is looking for the most dense flash storage platform to run applications… internal or via shared storage.
With each platform we test, it is very important to understand how each vendor configures the unit for different workloads as well as the networking interface used for testing.
- Infortrend ESDS S16F-G2652-4:
- Flash: 3.2TB usable (16x 400GB Smart Optimus Eco SSD RAID10)
- Network Interconnect: 4x 16GB Fibre Channel
- Huawei OceanStor Dorado2100 G2:
- Flash: 7.2TB/14.2TB/14.4TB usable (75 x 200GB SLC Huawei SSD RAID10/RAID5/RAID0)
- Network Interconnect: 8 x 8Gb Fibre Channel
- Fujitsu Storage ETERNUS DX200F
- Flash: 9.6TB usable (24x 800GB SAS3 MLC SSDs, 12 x RAID1, Wide-Striped)
- Network Interconnect: 4x 16Gb Fibre Channel
Application Performance Analysis
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Our SQL Server protocol for this review uses a 685GB (3,000 scale) SQL Server database and measures the transactional performance and latency under a load of 30,000 virtual users.
The FlacheSAN1L-D performed at 6,315 TPS when configured in Storage Spaces Mirrored (RAID10) with 30,000 virtual users running with the other top performers.
The average latency results during the 30k user SQL Server benchmark placed the FlacheSAN1L-D at 7ms when configured for RAID10 again running right alongside the top performers. The key difference with the EchoStreams FlacheSAN1 though is it hosted the storage and the application inside the same system, the others serve only storage to a separate compute platform.
Enterprise Synthetic Workload Analysis
For our Enterprise Synthetic testing we looked at the above mentioned flash platforms and the EchoStreams FlacheSAN1L-D under four configurations:
- Raw 1 HBA: 16 SSDs, direct access performance through a single expander with 20GB utilized per SSD
- Raw 2 HBAs: 16 SSDs, expander removed with SSDs directly attached to the onboard LSI 3008 HBA as well as one additional LSI 9300-8i with 20GB utilized per SSD
- RAID10 SMB3: 16 SSDs with 1 HBA presented as a pool in Storage Spaces. Four SMB3 shares utilizing 25GB per share
- RAID10 iSCSI: 16 SSDs with 1 HBA presented as a pool in Storage Spaces. Four iSCSI LUNs utilizing 25GB LUN
Prior to initiating each of the fio synthetic benchmarks, our lab preconditions the device into steady-state under a heavy load of 16 threads with an outstanding queue of 16 per thread. Then the storage is tested in set intervals with multiple thread/queue depth profiles to show performance under light and heavy usage.
- Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregated)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
This synthetic analysis incorporates four profiles which are widely used in manufacturer specifications and benchmarks:
- 4k – 100% Read and 100% Write
- 8k – 100% Read and 100% Write
- 8k – 70% Read/30% Write
- 128k – 100% Read and 100% Write
During the 4k synthetic benchmark, the Raw 2 HBAs was the overall top performer with a whopping 1,037,200 IOPS read performance and 742,662 IOPS write performance (the read performance was almost twice the next best performer and almost 20 times higher than the poorest performer). The RAID10 iSCSI cam in last with read throughput of 52,430 IOPS and a write throughput of 43,063 IOPS.
Moving on to average latency we see similar results to the above. The Raw 2 HBAs configuration had the lowest read latency at 0.24ms and the lowest write latency at 0.34ms. And the RAID10 iSCSI again came in far behind the other configurations at 4.88ms read and 5.94ms write.
With max latency we see almost the opposite of the above. This time the RAID10 iSCSI had the best overall read speed at 8.5ms. The Eternus DX200F had the best write speed at 15.6ms. The Raw 1 HBA had the highest maximum latency with the Raw 2 HBAs coming in with the second highest.
Standard deviation results help identify devices and configurations which experience a greater than average range of latencies during the synthetic benchmarks. This time around we see the RAID10 iSCSI configuration come out as the top read performer with a speed of 0.12ms and the Huawei Dorado 2100 was the top write performer with a speed of 0.59ms.
After reconditioning the array for 8k workloads, we measured the EchoStreams FlacheSAN1L-D throughput with a load of 16 threads and a queue depth of 16 for 100% read and 100% write operations. Here we see similar results to the 4k throughput test above. The Raw 2 HBAs came out on top for both read and write performance with a read throughput of 736,683 IOPS and a write throughput of 568,252 IOPS.
The next results are derived from a protocol composed of 70% read operations and 30% write operations with an 8k workload across a range of thread and queue counts. In terms of throughput, the Raw 2 HBAs was overall the best performer peaking at 535,166 IOPS.
Average latency calculations during the 8k 70/30 benchmark saw both of the Raw HBA configurations run neck and neck throughout with the Raw 2 HBAs having slightly lower latency throughout.
With max latency we see the Fujitsu Eternus DX200F had the lowest latency overall. The RAID10 iSCSI had the best latency of the FacheSAN1L-D configurations tested.
Standard deviation showed the Eternus and the Dorado2100 battle it out for the top performer. Of the EchoStreams FlachSAN1L-D configurations tested the Raw 1 HBA was the best performer.
Our final synthetic benchmark made use of 128k transfers and a workload of 100% read and 100% write operations. Here we see the Raw 2 HBAs as the top performer with a read speed of 8.3GB/s and a write speed of 7.6GB/s.
Conclusion
The EchoStreams FlacheSAN1L-D is a 1U form factor flash server that can house up to 20 2.5″ 7mm SATA SSDs. It supports dual socket 2011 Intel Xeon E5 2600 V3 (Haswell) processors, up to 512GB of DDR4 RAM and has two x16 PCIe 3.0 slots for expansion. It comes equipped with an onboard LSI 3008-based HBA and uses a patented tool-less SSD tray. The flash server has a wide variety of use cases including a Tier-0 storage, read-intensive database, content cache and so on.
Our benchmarks tested the performance of the EchoStreams FlacheSAN1L-D configured in Raw access as well as Mirror mode (e.g. RAID10) with iSCSI and SMB connectivity. In synthetic enterprise benchmarks, the FlacheSAN1L-D ran along the top of the pack in our SQL Server OLTP testing with 6,315 TPS and a latency of 7ms. In our 4k enterprise synthetic workloads the FlacheSAN1L-D had a read throughput of 1,037,200 IOPS and a maximum read latency of 8.5ms. In our 8k test the FlacheSAN1L-D had a read throughput of 736,683 IOPS. And looking at large block sequential testing, the FlacheSAN1L-D was topped out at 8.3GB/s.
Ultimately what EchoStreams has done is create one of the most dense 1U servers on the market with support for 20 commonly available 2.5″ SATA SSDs. Pairing the FlacheSAN1L-D with the current top-in-class read-centric SATA SSDs from Toshiba (HK3R2) has devastating effects, in a good way. The combination rivaled and beat much more expensive all flash arrays configured as shared storage for SQL Server, and when storage doesn’t have to be shared over the network, it did even better with an amazing 1,000,000 + 4K IOPS internally, again, with relatively low cost storage. There are many use cases where an application can take advantage of local storage and that’s really where this server shines. Of course it’s not intended to take over for a SAN, SANs still have many features and data services the FlacheSAN and Windows don’t. However, in those use cases when the form factor can be leveraged well, the FlacheSAN1 is fantastic and delivers tremendous performance and flexibility that most other 1U servers simply can’t aspire to.
Pros
- 20 SSD bays in 1U form factor
- Tool-Less SSD tray
- Wide variety of use cases
Cons
- Lacks full scale KVM suite with monitoring capabilities
The Bottom Line
The EchoStreams FlacheSAN1L-D is a highly dense 1U server that serves multiple use cases by packing a tremendous amount of storage and performance in a small form factor.
EchoStreams FlacheSAN1L-D Page
