Dell PowerEdge R660 E3.S Backplane Review

The Dell PowerEdge R660 is a 1U rackmount server designed to support the latest and greatest in server technology, including E3.S Gen5 storage, DDR5 memory, and configurations with liquid CPU cooling. It features support for two processors (dual sockets), 32 RDIMMs, and up to three graphics cards.

The Dell PowerEdge R660 is a 1U rackmount server designed to support the latest and greatest in server technology, including E3.S Gen5 storage, DDR5 memory, and configurations with liquid CPU cooling. It features support for two processors (dual sockets), 32 RDIMMs, and up to three graphics cards.

Although it’s a compact 1U form factor, this new Dell server can be outfitted with up to 16 SSDs. The PowerEdge R660 is the successor to the R650 and is designed for a range of different applications—from general-purpose usage to HCI/HPC.

Dell PowerEdge R660 E3.S Backplane Review Build

We previously tested the Dell PowerEdge R660, configured with five 800GB SAS3 SSDs (RAID5), 512GB DDR5 RAM, and dual Intel Xeon Platinum 8452Y CPUs.

For this review, we are changing things up by equipping it with two Intel 8460Y Platinum CPUs (a decent upgrade), sixteen 16GB 4800MHz DDR5 RAM modules, and eight Samsung PM1743 7.68GB E3.S SSDs. This configuration will undoubtedly yield different performance results, especially with the Gen5 E3.S SSDs and higher-end CPUs.

For example, the Intel 8452Y used in the previous review features 36 cores and 72 threads, with a base frequency of 2.00GHz (and 3.70GHz max turbo frequency) up to 3.8 GHz in turbo clock. The Intel 8460Y, on the other hand, has a total of 40 cores and 80 threads, with a base frequency of 2.00GHz (and 3.70GHz max turbo frequency). This increase in core and threads will provide several noticeable benefits, including improved performance and multitasking (it can handle more requests, processes, and users simultaneously), increased scalability, and improved energy efficiency.

The use of E3.S SSDs will also affect the overall performance and functionality of the R660. The PM1743 uses the most recent EDSFF E3.S form factor, which has improved signal integrity and better performance while boasting a storage capacity of up to 15.36TB (we are using 8x of the 7.68GB model in this review). Its slim 7.5mm profile also allows for greater storage density in enterprise servers, which can improve performance and power efficiency per rack unit.

Most important, however, is the dual-port configuration of the PM1743, which ensures uninterrupted and continuous computing, even if one port fails. With the ability to seamlessly shift workloads to the second port, it can help to maintain business continuity and prevent any disruptions caused by a port failure. This is an important feature for all types of businesses that rely on 24/7 operations.

Dell PowerEdge R660 vs R650

Compared to the last-gen R650, notable updates include support for Intel “Sapphire Rapids” fourth-generation Xeon Scalable processors. As mentioned above, this review leverages the Intel 8460Y; these new chips can scale up to 56 cores (in addition to DDR5 memory and support Gen5 storage). They also feature improved AI and cryptography acceleration. With the added support for PCIe Gen5, the new servers can provide a higher bandwidth to handle data-intensive workloads.

Storage configurations are also one of its biggest strengths. Conventional storage scales up to ten 2.5-inch SAS/SATA drives, but it can also use Gen5 NVMe SSDs. As mentioned earlier, the new backplane option also includes support for up to 16x E3.S (7.5mm) drives. There are also rear-drive options for boot, including the new NVMe BOSS drive. The NVMe BOSS drive can free up valuable drive bays while providing fast boot and recovery times.

The server offers a range of power supply options to fit different needs, including AC Platinum (800W, 1400W) and AC Titanium (700W, 1100W, 1800W) power supplies. These power supplies are highly efficient and can help reduce overall energy consumption and costs. The server also supports a 1100W LVDC @-48VDC input for data center environments with different power requirements.

Dell PowerEdge R660 Specifications

Dell PowerEdge R660 E3.S Design and Build

Since we covered the design and build in detail in our first review of the R660, we’ll only briefly go over it here.

On the front panel, it features four clusters of E3.S bays with the center stack unpopulated to improve airflow to the dual Xeon Scalable processors.

On the rear of the unit, it has dual 1GbE, one USB 2.0, one USB 3.0, VGA out, and an additional Ethernet port for Dell’s iDRAC remote management. Optional serial port support is also available, and extra networking ports can be added via the server’s OCP 3.0 slot.

The unit comes with dual 1,400-watt hot-swap power supplies, and the dual M.2 NVMe SSDs are accessible at the left rear for use as boot drives, though this system isn’t equipped with BOSS. Other configurations (like this review) support Gen5 E3.S drives.

Inside, you’ll see several cooling fans that draw air through the drive bays, passing over the dual CPU heatsinks and DDR5 RDIMM slots. The server also supports CPU liquid-cooling solutions; however, this functionality was not included in our review unit. The unit’s expansion slots are located towards the rear and include up to three PCIe slots, with the riser allowing full-height cards to be used.

The Dell PowerEdge R660 offers a wide range of configuration options, even across a unique system such as our E3.S build. This setup comes in three specific configurations that we will outline below, including 14 or 14+2 direct-attached NVMe builds or a 16 HWRAID NVMe build. The newest generation of PowerEdge servers offers similar configurations allowing users to leverage hardware RAID for current-generation SSDs.

The first configuration shown is the 16-bay HWRAID setup. Here the drives are split into two groups of 8 drives, each flowing to a dedicated PERC12 RAID card. With any of these configurations, Dell has balanced the storage and I/O needs to be evenly distributed across the CPUs.

Next is a model similar to the previous review, with 14 front-mount E3.S bays directly attached to the motherboard. They are split into groups of eight and six across CPU1 and CPU2.

The final server configuration is for customers needing as many E3.S SSDs as possible while also consuming one PCIe bay in the rear providing a 14+2 configuration with all 16 SSDs direct attached to the motherboard.

Dell PowerEdge R660 iDRAC9 Remote Management

For the PowerEdge R660 server, we utilized Dell’s iDRAC9 Enterprise remote management software. Integrated Dell Remote Access Controller (iDRAC) is a proprietary Dell technology that allows users to remote access and manage Dell servers. It provides system administrators with a web-based interface to monitor and manage server hardware, such as system health, configuration settings, and firmware updates. With iDRAC, you don’t need to be physically on-site to manage the server.

If you’re interested in learning more about how the Dell R660 server is managed by iDRAC9, be sure to check out our previous review, where we go into more detail.

Dell PowerEdge R660 CPU Benchmarks

Here is our Dell PowerEdge R660 configuration for this review:

• 2 x Intel Platinum Xeon 8460Y CPUs
• 16 x 16GB 4800 DDR5
• 8 x Samsung PM1743 7.68GB E3.S SSDs

Our first test is Cinebench R23,  a widely used benchmarking tool that measures the performance of CPUs and GPUs using Maxon Cinema 4D for rendering and provides a score that can be used to compare the performance of different systems and components.

y-cruncher is a multi-threaded and scalable program that can compute Pi and other mathematical constants to trillions of digits. Since its launch in 2009, it has become a popular benchmarking and stress-testing application for overclockers and hardware enthusiasts.

Dell PowerEdge R660 Storage Performance

Leveraging the eight 7.68TB Samsung PM1743 NVMe Gen5 E3.S SSDs, we focused on both application performance with Sysbench and synthetic I/O performance with Vdbench.

Sysbench Performance

The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test also measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency.

Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system resource perspective, we configured each VM with eight vCPUs, and 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.

Sysbench Testing Configuration (per VM)

• CentOS 6.3 64-bit
• Percona XtraDB 5.5.30-rel30.1
  • Database Tables: 100
  • Database Size: 10,000,000
  • Database Threads: 32
• RAM Buffer: 24GB
• Test Length: 3 hours
  • 2 hours preconditioning 32 threads
  • 1 hour 32 threads

For our Sysbench test, we used eight VMs, two of which were positioned on each SSD VMware datastore. With this eight VM load, we could scale up to 29,086TPS with an average of 3,636TPS per VM.

We measured average latency across the Sysbench VM group between 8.77ms and 8.80ms.

Measuring 99th percentile latency in the Sysbench 8VM group, the aggregate average measured 15.87ms.

VDBench Workload Analysis

When benchmarking storage devices, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests help baseline storage devices with a repeatability factor that makes it easy to make apples-to-apples comparisons between competing solutions. These workloads offer a range of different testing profiles ranging from “four corners” tests, to common database transfer size tests, to trace captures from different VDI environments.

All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices. Our testing process for these benchmarks fills the entire drive surface with data and then partitions a drive section equal to 25% of the drive capacity to simulate how the drive might respond to application workloads. This differs from full entropy tests, which use 100% of the drive and takes them into a steady state. As a result, these figures will reflect higher-sustained write speeds.

Profiles:

• 4k Random Read: 100% Read, 128 threads, 0-120% iorate
• 4k Random Write: 100% Write, 128 threads, 0-120% iorate
• 64k Sequential Read: 100% Read, 32 threads, 0-120% iorate
• 64k Sequential Write: 100% Write, 16 threads, 0-120% iorate
• 128k Sequential Read: 100% Read, 32 threads, 0-120% iorate
• 128k Sequential Write: 100% Write, 16 threads, 0-120% iorate
• 70R/30W and 90R/10W Random Mix, 64 threads, 0-120% iorate
• Synthetic Database: SQL and Oracle
• VDI Full Clone and Linked Clone Traces

Starting in random 4k reads, the Dell PowerEdge R660 showed a peak performance of 4.5 million IOPS at 223.2µs in latency when using eight Samsung PM1743 7.68GB E3.S SSDs.

In writes, the Dell PowerEdge R660 peaked at 5.16 million IOPS at 184.1µs.

Moving on to 64k sequential performance, the Dell PowerEdge R660 peaked at 53.1GB/s (849K IOPS) read with a latency of 267.6µs with the Samsung drives (before taking a dip in performance at the very end).

For 64k sequential writes, the Dell PowerEdge R660 recorded a peak performance of 23.6GB/s (roughly 377K IOPS) with 301µs, with another slight spike in performance at the end.

In 128k sequential performance, the Dell PowerEdge R660 posted 50GB/s (400K IOPS) and 492.7µs in latency for reads.

In 128k sequential writes, the Dell PowerEdge R660 peaked at 23.2GB/s (186K IOPS) with 631.2µs in latency.

Next up is our mixed workloads, starting with our mixed 70/30 4k profile (70% read, 30% write). Here, the Dell PowerEdge R660 had a very stable performance, posting a peak of 4.88 million IOPS at 97.8µs in latency.

In our mixed 90/10 4k profile, the Dell PowerEdge R660 yielded a peak of 4.5 million IOPS at 101.5µs in latency when populated with the Samsung E3.S drives.

Our next set of tests is our SQL workloads: SQL, SQL 90-10, and SQL 80-20. Starting with SQL, the R660 server posted a peak performance of 2.6 million IOPS with a latency of just 96.5µs via the G3n5 Samsung E3.S drives.

In SQL 90-10, the Dell PowerEdge R660 hit a peak of 2.5 million IOPS and a latency of 101µs before taking a slight dip in IOPS at the very end of the test.

With SQL 80-20, the Dell PowerEdge R660 peaked at 2.4 million IOPS with a latency of just 104µs.

Next up are our Oracle workloads: Oracle, Oracle 90-10, and Oracle 80-20. Starting with the general Oracle workload, the R660 had a peak performance of 2.2 million IOPS at 108.4µs.

Looking at Oracle 90-10, the Dell PowerEdge R660 posted a peak performance of 1.64 million IOPS at 104.3µs (with a slight spike in performance at the end).

Next up is Oracle 80-20, where the Dell PowerEdge R660 peaked at 1.56 million IOPS at 109µs.

Next, we switched to our VDI clone test, Full and Linked, where the 9400 continued its excellent performance. For VDI Full Clone (FC) Boot, Dell PowerEdge R660 at 1.97 million IOPS with a latency of 126.7µs with the Samsung drives.

Moving on to VDI FC Initial Login, the Dell PowerEdge R660 peaked at 1.21 million IOPS with a latency of 143.8µs.

With VDI FC Monday Login, the Dell PowerEdge R660 posted a peak of 809K IOPS at a latency of 119.8µs.

For VDI Linked Clone (LC) Boot, the Dell PowerEdge R660 showed steady performance, peaking at 876K IOPS with 118.8µs.

In VDI LC Initial Login, the Dell PowerEdge R660 peaked at 375K IOPS with 123.4µs.

For VDI LC Monday Login, the Dell PowerEdge R660 showed a peak of 609K IOPS with a latency of 155.2µs.

Conclusion

As we mentioned before, Dell’s PowerEdge R660 represents a significant step forward for their server portfolio, offering enhanced cooling capabilities to accommodate up to two 4th-gen Xeon Scalable processors. These processors support DDR5 memory and Gen5 storage, allowing the server to demonstrate high-performance and density capabilities. The PowerEdge R660 also showcases Dell’s commitment to innovation, offering a unique platform that opens up new performance and density possibilities for businesses and enterprises.

Dell’s experience with servers is evident across the careful designing of the internal components of the chassis mixed with the rich iDRAC 9 remote management system. The internal chassis layout enables a wide range of configuration options while maintaining uncompromising cooling performance to keep those components operating reliably and at full performance. iDRAC 9 allows for comprehensive server health, settings, maintenance, and support, putting control and oversight at users’ fingertips. That said, one of the R660’s coolest features is its variety of storage drive configurations, which includes the use of E3.S SSDs for maximum storage density.

In our previous review of the PowerEdge R660, we utilized five 800GB SAS3 SSDs and Intel Xeon Platinum 8452Y CPUs. However, in this new build, we wanted to push the platform’s capabilities even further, employing Gen5 E3.S SSDs – specifically, eight Samsung PM1743 7.68GB SSDs. We also leveraged the higher-end dual Intel Platinum Xeon 8460Y CPUs. As you saw from our benchmarking results, these upgrades helped showcase the more significant potential of this unique platform.

While it’s still early days for E3.S SSDs, they’re clearly the next step after U.2/U.3 drives, every flash vendor is dumping immense resources into this form factor. Dell has done well with the R660 to include a balanced application of the drives, without limiting configurations on PCIe slots in the back for additional I/O or accelerators. And it’s our guess that should customers want some other configuration, like 20 SSDs, and are willing to trade out the PCIe slots, Dell could easily transition to something like that. With these Intel systems, it’s all about lane allocation, unfortunately, there aren’t more to play with.

Overall, the PowerEdge R660 is an impressive server platform that sets a new standard for performance, I/O and storage density. Its advanced cooling capabilities (with liquid options), support for fourth-generation Xeon Scalable processors, DDR5 memory, and Gen5 storage, combined with the innovative use of E3.S SSDs, make it an excellent choice for a wide range of applications and it’s one of our Best of 2023 winners.

Dell PowerEdge Servers

Engage with StorageReview

Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed