The EchoStreams FlacheSAN2 is an ultra-dense 48-bay server that is extremely space-efficient, fitting into just 2U of rack space. As with other solutions from EchoStreams, the FlacheSAN2 is also extremely customizable with hardware configurations available to specifically integrate this product for the organizations’ precise needs. That makes the FlacheSAN2 an excellent option for an enterprise that already has specific hardware in mind. In our particular configuration, we populated the 48 bays with Toshiba HK3R2 SATA SSDs and ensured robust performance with four PCIe 3.0 Adaptec 71605 RAID cards and four Mellanox ConnectX-3 56Gb/s dual-port InfiniBand adapters. With certain configurations built to support upwards of 25GB/s of bandwidth and over 3.8 million IOPS, the FlachSAN2 is designed to compete where few others can.
The EchoStreams FlacheSAN2 is an ultra-dense 48-bay server that is extremely space-efficient, fitting into just 2U of rack space. As with other solutions from EchoStreams, the FlacheSAN2 is also extremely customizable with hardware configurations available to specifically integrate this product for the organizations’ precise needs. That makes the FlacheSAN2 an excellent option for an enterprise that already has specific hardware in mind. In our particular configuration, we populated the 48 bays with Toshiba HK3R2 SATA SSDs and ensured robust performance with four PCIe 3.0 Adaptec 71605 RAID cards and four Mellanox ConnectX-3 56Gb/s dual-port InfiniBand adapters. With certain configurations built to support upwards of 25GB/s of bandwidth and over 3.8 million IOPS, the FlachSAN2 is designed to compete where few others can.
In the high-performance storage market segment, there are pre-built and more common systems shipping in order to cut back on the planning and deployment process, but there are also plenty of organizations that have unique IT infrastructure arrangements in mind. In the case of the FlacheSAN2, some organizations may have the budget to leverage a majority of the 48 bays for high-capacity small form factor (SFF) 2.5″ SSDs (giving the FlacheSAN2 the highest density for SSD in 2U), while others might be interested in gaining the raw SSD transfer rates, but with less concern over capacity. The FlacheSAN2 can support these and other configurations with its flexibility and capability of supporting both 7mm z-height SAS and SATA drives. The FlacheSAN2 also supports any SSDs, enabling customers to leverage even more savings vs. offerings from standard server vendors. Another area where savings can be had is in power efficiency. The FlacheSAN2 only consumes 500W at its max power consumption, giving it one of the highest performance to watts ratios out there. The FlacheSAN2 leverages Intel motherboards giving it great reliability and high compatibility. It maximizes these motherboard resources giving balance between the disk side and controller side I/Os.
The EchoStreams FlacheSAN2 comes with a 1-year parts replacement warranty and is available now with a starting price of $7,800.
EchoStreams FlacheSAN2 Specifications
- Capable of supporting 3.8+ million IOPS or up to 25GB/s of bandwidth
- Intel proprietary size dual-socket Sandy Bridge EP
- Up to two socket 2011 Intel® Xeon® E5 2600 V2 Ivy Bridge processors
- 16 DDR3 four channel/CPU 800/1066/1333/1600 RDIMM/UDIMM/LRDIMM
- 48 x 6G SAS/SATA 2.5”x7mm hot-swap SSD bays + 2 x internal 2.5” x 7mm SSD/HDD for OS
- Quad 1GbE ports Intel i350 onboard
- 7x PCIe Gen3 X8 LP + 1x PCIe Gen2 X4 LP slots + 1x Intel PCIe Gen3
- Power 800W AC/DC 1+1 80Plus high-efficiency redundant power supply; 700W -48VDC option available
- 4x quick-swap 80mm PWM fans
- Dimensions (WxHxD): 16.8” x 3.5” x 26”
- Weight: 58lbs fully loaded
Design and Build
The FlacheSAN2 has a really sleek design that appears to be both premium and industrial at the same time. Starting with the front of the server, there are front-panel LED indicators for power, network and system status as well as standard power, NMI and reset buttons. The most prominate feature of the entire unit is the ultra-dense 48 bays designed for 7mm SSDs. From a serviceability aspect the FlacheSAN2 is designed to be simple to build to your specifications, effortless to rack and easy to interact with once put in production and also be fairly easy. EchoStreams goes to great lengths to make this possible, with very detailed illustrations to walk their customers through the process.
When it comes time to install SSDs to the front of the FlacheSAN2, no tools are required. The top portion of each bay has a latch that enables quick and simple access. The drive-trays themselves don’t require any tools to install SSDs, using a unique gripping mechanism to keep the SSDs secure without requiring screws which are sometimes overused in JBODs and storage servers. In our unit we have the front fully populated with 48 Toshiba HK3R2 SSDs.
EchoStreams has also included tool-less ball-bearing slide rails to simplify the installation process. For those who have experience installing rails, that makes this process even easier. With the rails installed, you can position the FlacheSAN2 into place. From there, it can be slid back and forth with ease, drawing out from the front of the rack several inches to enable maintenance.
To understand the unique nature of the motherboard layout inside the FlacheSAN2, EchoStreams includes a guide detailing the PCIe layout. From this buyers can understand how to best leverage certain PCIe devices to maximize performance, as well as take advantage of unique positioning when traditional PCIe slots are taken. One of the more unique items is the ROC Module Slot, which EchoStreams uses a custom adapter to mount a remote PCIe slot inside the chassis for an additional RAID card.
Inside the FlacheSAN2 you can see the great attention to detail EchoStreams put into the 2U chassis to support four to six RAID cards or HBAs, as well as the supporting NICs to share the storage on the network. EchoStreams uses custom SAS cabling to connect the front drive backplanes to the storage cards, which helps keep airflow disruption to a minimum. This approach helps cooling, where every bit helps in a high-performance build. Another unique item is a custom PCIe cage added to the chassis to improve hardware density where every last PCIe card needs to be leveraged inside the system.
Moving to the back side of the FlacheSAN2, on the left side there are the two power supplies as well as exhaust ports for the four PWM fans located inside the chassis. Then we get to all of the connectivity including four 1GbE ports (shared with management), as well as four USB ports, VGA and a legacy COM port. Next to those are the eight PCIe slots, which in our production build includes three Adaptec 71605 RAID cards (the fourth is located inside) as well four dual-port Mellanox ConnectX-3 56Gb/s IB cards. This leaves one open slot for another network interface or additional storage product.
Data Fusion Performance Testing
We’ve partnered with Stream Financial to replicate their DataFusion performance test in our lab. The introduction of this test was part of our Violin WFA-64 review, where once completed we ported the data over to the EchoStreams FlacheSAN2 and presented it in a similar fashion to the same Dell R920 compute server.
DataFusion in its simplest form is designed to demonstrate the processing and aggregating of more than one trillion rows of risk data, containing 13 trillion points of data, one risk point per row. The test looks at a very real big data use case, where decision-making can be hampered by the time it takes to process the data. The test mimics a trading environment with risk data containing risk buckets for delta, gamma, vega and theta for trading books over a 12-year period. To simulate a typical business view, the data was aggregated using SQL ‘where’, ‘like’ and ‘group by’ queries to show bucketed risk exposure by risk type, currency and counterparty. The overall highly-compressed database footprint is a little over 8TB, expanded exceeds 100TB. For the purposes of this test, the database is run without indexing, forcing the server and storage to process all data in real-time.
With the EchoStreams FlacheSAN2 loaded with 48 Toshiba HK3R2 SSDs backing the storage in the benchmark, we measured a time of 60 minutes and 32 seconds. This compared to 56 minutes and 16 seconds that we measured using the Violin WFA-64 for storage.
Enterprise Synthetic Workload Analysis
Our enterprise benchmark process preconditions each storage device into steady-state with the same workload the device will be tested with under a heavy load of 16 threads with an outstanding queue of 16 per thread, and then tested in set intervals in multiple thread/queue depth profiles to show performance under light and heavy usage.
EchoStreams FlacheSAN2 Hardware Configuration
- 2 x Intel Xeon E5-2697 v2 (2.7GHz, 30MB Cache, 12-cores)
- Intel C602 Chipset
- Memory – 64GB (8 x 8GB) 1333Mhz DDR3 Registered RDIMMs
- Windows Server 2012 Standard
- 100GB Micron P400e Boot SSD internal
- 4 x Adaptec 71605 in HBA mode
- 48 x Toshiba HK3R2 SSDs
- Local Storage Configuration: Each SSD addressed individually
- Remote Storage Configuration: 4 x 12 SSD Storage Spaces Mirror Pools (12 SSDs per HBA). 2 25GB SMB3 shares per pool.
EchoStreams FlacheSAN1 Hardware Configuration
- 2 x Intel Xeon E5-2697 v3 (2.6GHz, 35MB Cache, 14-cores)
- Memory – 256GB (16 x 16GB) DDR4 Registered RDIMMs
- Windows Server 2012 Standard
- 2 x LSI 9300-8i
- 24 x Toshiba HK3R2 SSDs
- Local Storage Configuration: Each SSD addressed individually
- Remote Storage Configuration: 2 x 12 SSD Storage Spaces Mirror Pools (12 SSDs per HBA). 4 25GB SMB3 shares per pool.
Dell PowerEdge R920 Infiniband FIO LoadGen
- Four Intel E7-4870 v2 CPUs (2.3GHz, 15-cores, 30MB Cache)
- 512GB RAM (8GB x 64 DDR3, 128GB per CPU)
- 2 x 300GB 10K SAS RAID1 Boot
- 4 x Mellanox ConnectX-3 Dual-Port InfiniBand Adapters
Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Our Enterprise Synthetic Workload Analysis includes profiles based on real-world tasks. These profiles have been developed to make it easier to compare to our past benchmarks as well as widely-published values such as 4K, 8k 70/30, and 128K which are commonly used for enterprise products.
- 4k Profile
- 100% Read and 100% Write
- 8k 70/30
- 70% Read, 30% Write
- 100% 8k
- 128K (Sequential)
- 100% Read or 100% Write
- 100% 128K
In our 100% 4K random read and write test, we measured the performance of 16 KK3R2 SSDs in RAID10 or Raw as well as the performance of 48 KK3R2 SSDs in RAID10 or Raw. The 4K random read results showed that the 48 Raw configuration boasted almost 2 million IOPS read (1,906,010 IOPS) with 1,765,088 IOPS write, both of which were very impressive. The second closest configuration was the 16 Raw; however, both results were nearly 1 million IOPS in reads and writes with 1,037,200 IOPS and 742,662 IOPS, respectively.
When we looked at average latency portion, we saw that the 16 Raw configuration had the lowest latency with 0.245ms read and 0.341ms write. The EchoSteams using the 48x RAID10 configuration had the next best results, posting 0.41ms read and 0.49ms write.
Moving on to max latency performance, the 16 Raw configuration boasted opposite performance compared to the last test as we recorded latency that was five times higher than the next closest configuration. That being said, the 48 RAID10 had the lowest latency with 8.56ms read and 31.358ms write with the 16 RAID10 configuration fairly close behind with reads and writes of 9.4ms and 44.6ms, respectively.
Our standard deviation results showed that the 48 RAID10 posted the lowest read latency at just 0.08ms, while the 48 Raw had the lowest write latency at 0.30ms. The 16 RAID10 configuration showed the weakest results with 0.46ms read and 1.73ms write.
Our next benchmark uses a sequential workload composed of 100% read operations and then 100% write operations with an 8k transfer size. Again we see the 48 Raw configuration crush the other configurations in throughput. For read throughput, the 48 Raw boasted a blistering 1,823,704 IOPS while writes hit 1,503,920 IOPS. Both of these results were over a million IOPS higher than the next closest configuration.
Our next series of workloads are composed of a mixture of 8k read (70%) and write (30%) operations up to a 16 Threads 16 Queue, our first being throughput. It was no real surprise here to see the 48 Raw configuration start out and stay on top throughout the test. The 48 Raw peaked at 1,489,979 IOPS in terminal queue. Again, this impressive configuration placed roughly 1 million IOPS higher than the next closest comparable.
Looking at average latency, we see results that mirror our 4K test of the same benchmark. Here, the 16 Raw had the lowest latency throughout; however, it barely edged out the 48 RAID10 at the end with 0.47ms vs 0.48ms.
With max latency, the two 48 configurations were neck and neck from beginning to end for the lowest latency, as neither drive went above 30ms or below 10ms. That being said, we did see some rather large spikes from the 16 RAID10 (one as high as 2,133ms) that went off our charts.
Our next chart plots the standard deviation calculation for latencies during the 8k 70% read and 30% write operations. The 48 RAID10 had the lead at the start of the benchmark and maintained the until the end. Once again, however, we saw big spikes from the 16 RAID10, though certainly not as large as with max latency above.
Our final synthetic benchmark made use of sequential 128k transfers and a workload of 100% read and 100% write operations. Here the 48 Raw configuration was the clear winner once again in throughput, as it boasted top-class speeds of 15,840MB/s read and 16,005MB/s write. The next best configuration was the 48 RAID10, which had read and write results of 13,715MB/s and 6,258MB/s, respectively.
Conclusion
The EchoStreams FlacheSAN2 provides organizations with a customizable solution with its 48-bays for 7mm SSDs that fits into just 2U of rack space. EchoStreams is more than happy to build the system to an organizations’ specs by installing a range of hardware to meet their exact needs. That makes the FlacheSAN2 ideal for organizations who already have an IT infrastructure plan in mind and don’t need a pre-configured/turnkey solution. In our test system, we loaded up the FlacheSAN2 with four Adaptec 71605 RAID cards, four Mellanox ConnectX-3 56Gb/s dual-port InfiniBand adapters and 48 Toshiba HK3R2 SSDs for exceptional shared storage performance. The SSDs in particular give us one of the best performance/$ blends available with enterprise SSDs. Additionally, our FlacheSAN2 features a dual-processor Intel motherboard with twin Intel E5-2697 v2 CPUs to help drive plenty of I/O and bandwidth over the wire.
In our tests we measured the performance recorded locally (raw) as well as in StorageSpaces Mirror over SMB3 to another Windows Server 2012 server. These two configurations show how much storage potential either locally operated or remote applications have on tap. Locally we measured peak I/O in excess of 1.8 million IOPS read and 1.7 million IOPS write with a 4K random workload. Looking at sequential throughput, those figures measured about 16GB/s read and write. Presenting the storage in a mirrored configuration using Storage Spaces, we measured peak I/O at 629k IOPS read and 421k IOPS write, while sequential throughput measured 13.7GB/s read and 6.3GB/s write.
From a 2U platform, there aren’t many competitors that can meet or exceed those numbers in this particular footprint. PCIe or NVMe SSD-equipped servers might come close to catching up, but you lose the storage capacity and cost benefits of the SATA SSDs the FlacheSAN2 can leverage. The FlacheSAN2 is also effectively a single-controller server, so HA objectives require more than one chassis and some degree of coordination. Overall though we were quite impressed with the platform, from both its high performance and its user-configurable design that covers multiple industry segments. It’s truly the definition of flexible, high-performance storage and many will find the extra configuration steps well worth the investment considering the payoff. For those who need a little more performance, Echostreams has recently announced the FlacheSAN2-D3, which supports Haswell and offers 4 more drive bays in the rear of the chassis for 52 total.
Pros
- Very dense 48-bay 2U storage server design (SATA or SAS)
- Incredible peak I/O and bandwidth offered
- Flexible design allows for high level of customization
Cons
- Not HA in a single chassis
Bottom Line
Paired with the right components, the EchoStreams FlacheSAN2 provides a unique blend of performance and cost effectiveness with its 48 bays in just a 2U footprint. The platform is extremely flexible, meaning it can be configured to meet the most demanding application needs with a comparatively modest investment.
Discuss this review
Sign up for the StorageReview newsletter
