Comino Grando Review – Liquid-Cooled GPU Powerhouses

The sheer power of the Comino Grando liquid-coooled systems ensures that GPU performance remains uncompromised even under full load.

Comino, for the uninitiated, is a company that has carved a niche in liquid-cooled servers, creating versatile and efficient workstations and servers. In this review, we will examine the Comino Grando Server and Workstation, their flagship systems. These systems are designed to function as a 19-inch rack-mountable unit and as a standalone desktop, catering to a wide array of high-performance computing needs. Whether it’s data analysis, graphic design, gaming, or other demanding tasks, these servers are built to deliver performance, reliability, and cooling efficiency.

It’s impossible to overstate the “cool factor” of these Grando systems. They exude power and performance, seamlessly integrating state-of-the-art liquid cooling technologies to maintain optimal temperatures. This ensures peak performance, even under strenuous workloads, making coolness a literal and figurative attribute. The cooling platform design also enables GPU densities that wouldn’t otherwise be possible without liquid cooling. On the Grando Server, for example, Comino can fit six NVIDIA 4090 GPUs side by side, which would be impossible air-cooled. The liquid-cooled design is one slot width wide, while the stock air-cooled part is double width.

The Server and Workstation models are highly customizable, offering users an array of choices, including powerful multi-core processors (AMD Threadripper PRO), memory configurations up to 1TB, and massive storage capacities. Users also have the option of adding up to six top-tier professional GPUs, including NVIDIA RTX A6000 (48GB), NVIDIA RTX 6000 ADA (46GB), NVIDIA H100 (80GB), NVIDIA A100 (80GB), NVIDIA L40s (48GB) and the NVIDIA RTX 4090 (24GB). This makes the Comino Grando a solid choice for professionals and businesses searching for robust computing solutions that promise power and reliability.

We had the opportunity to test two of Comino’s liquid-cooled systems. Starting with the Grando Server, our configuration is outfitted with six NVIDIA RTX 4090s. It also comes equipped with AMD’s Threadripper PRO 5995WX CPU (a 64-core processor that we saw inside the Lenovo ThinkStation P620), 512GB DDR5 DRAM, a 2TB NVMe SSD, and four 1600W PSUs for redundancy to enhance reliability against power failures.

The second system we received from Comino, the Workstation iteration, is highlighted by four NVIDIA A100 professional GPUs, a Threadripper Pro 3975WX processor (36 cores, 64 threads, turbo speed up to 4.2 GHz), 512GB DDR5 DRAM and 2TB NVMe SSD. This one is designed for near-silent operations; shocking considering the GPUs inside.

As indicated above, the main distinction between the two models lies in their GPU and CPU capabilities, which means their targeted application scenarios also differ. The 5995WX CPU in the Server model boasts a higher core count, making it well-suited for parallel processing tasks. It significantly enhances performance in server environments where handling multiple tasks simultaneously is crucial.

In contrast, while still powerful, the Workstation’s 3975WX offers fewer cores, aligning with workstation use where balanced performance across various applications is needed. The GPU selection further differentiates the two; RTX 4090s are cutting-edge in graphics rendering and ideal for tasks requiring intense graphical computation, whereas the NVIDIA A100s (each equipped with 40GB of memory, are tailored for deep learning and scientific computing.

Comino Grando Components and Build

Despite an initial shipping mishap (where the Server suffered a fall off the delivery truck–not by us, I swear!), Comino’s thorough packaging ensured the system’s integrity remained intact. This resilience is partly due to its compact, water-cooled design, which inherently reduces the risk of damage during transport. This system has been showcased at conventions and analyzed by technology experts across the U.S., so it is no stranger to the wear and tear of travel.

Unboxing the Grando Server model, we discovered a well-organized interior, complete with power cables, rails, and a notice about the pre-filled cooling system, including tools and resources for maintenance. The front of the server boasts an impressive lineup of six RTX 4090s, each stripped down to a single-slot form factor thanks to the water blocks for a sleek, efficient design. Considering the power these compact units possess, this setup is certainly interesting.

Both models use almost identical cases; internally, they look virtually the same aside from the cooling configuration. Unless otherwise noted, we will focus on the Server iteration for the layout portion. The front panel connectivity comprises additional I/O options, including audio in/out jacks for direct sound system integration, multiple USB ports for peripheral connectivity, and networking ports to allow connections to a wide range of devices. Both models emphasize easy access for maintenance and user interaction, consistent with Comino’s design philosophy for high-performance computing hardware.

Located at the far left of the server’s front panel is an LED display, which provides a range of helpful telemetry data, including information on the pumps, air temperature, coolant temperature, and fan speed. Navigating the menu is facilitated by illuminated buttons on the cooling module, with short presses (under 2 seconds) to scroll through the information. A long press on PB2 (over 2 seconds) opens access to additional menu branches, including Commands, Service (Settings), and Event Log, adding further control over settings and system diagnostics. Overall, the interface is intuitive, with easily navigable menus enhanced by backlit buttons for easy operation.

Opening the top panel, the first thing that caught our eye was of course the liquid tubing, showing a sense of sophistication and uniformity. We were also drawn to the support bar that spans the areas housing the GPUs and SSDs within the chassis, which will significantly reduce any movement of sensitive components during shipping, ensuring they arrive in good condition.

At the center of the system is a large water distribution block, providing both cool liquid and a return path. These are all connected with dripless quick-disconnect couplings, exuding Comino’s high-quality engineering prowess.

Encased in proprietary Comino cold plates, the six NVIDIA RTX 4090 GPUs inside the Grando Server are arranged in a very compact formation to maximize the space within the server while ensuring each unit can be cooled effectively. The Workstation model utilizes four NVIDIA A100 GPUs.

Both are securely fastened within the server using a retention brace, which runs perpendicularly across them. This helps stabilize and align the GPUs and maintains their structural integrity during moves.

Removing a GPU to get a better look, each is an off-the-shelf Gigabyte NVIDIA RTX 4090 with Comino’s custom cold plate design. The cards weigh a significant amount, no doubt in part from the massive heatsink assembly sandwiched onto the GPU.

The fittings on each card that go back to the water distribution block are dripless connections, which allow the cards to be removed for service without leaking liquid everywhere. They have a minimal amount of residue on the mating surface, but that is it after disconnecting it from the system.

The same fittings are on the distribution block as well.

The CPU and voltage regulators have a cold plate directly connected to coolant hoses to keep them cool and avoid bottlenecks under intense workloads. Flanking the CPU on both sides are fully populated DRAM slots for a total of 512GB.

While both systems have a massive radiator in the back to offload the heat, the biggest difference between the two platforms is the fan config and overall noise. While both systems are very quiet, the Workstation config is quieter and has a less aggressive fan setup.

In terms of design, the only real complaint we have is placing I/O ports at the front and power inputs at the rear. In a rackmount data center environment, this design poses a challenge. By using an off-the-shelf motherboard, Comino only has so many options. The net result is a minor annoyance, in an otherwise extremely well-thought-out design.

Comino Grando Remote Management

Grafana is leveraged for managing the Comino Grando Server, serving as a powerful open-source analytics and monitoring solution providing real-time data visualization through dashboards. It integrates with various data sources and offers features like alerting, multi-tenant support, and API access, making it practical for tracking and diagnosing system performance.

Looking at the General information dashboard, Grafana displays real-time metrics on the Comino Grando, such as coolant flow rates for thermal management, ambient humidity levels, temperatures of critical components like System Management (STM), Voltage Regulator Module (VRM), and Printed Circuit Boards (PCB).

It also monitors energy consumption, displaying individual component usage like CPU and GPU power draw, which is essential for managing a server of this caliber. The interface efficiently consolidates all this critical information displayed in an easily consumable manner. This ensures the server’s performance and reliability are maintained at optimal levels while providing the administrators with actionable insights to address any potential issues preemptively.

Comino Grando Specs

Comino Grando Performance

To compare the performance between two configurations of Comino Grando systems—a server and a workstation—we utilized a range of benchmarks that assess different aspects of computing performance, including rendering, computation, and data processing capabilities. The benchmarks include Blender 4.0, Luxmark, OctaneBench, Blackmagic RAW Speed Test, 7-zip Compression, and Y-Cruncher, each chosen for its relevance in evaluating the specific strengths of these systems in various computing environments.

Blender 4.0

Blender is a comprehensive open-source 3D creation suite for modeling, animation, simulation, and rendering projects. Blender benchmarks evaluate a system’s performance in rendering complex scenes, a crucial aspect for professionals in visual effects, animation, and game development. This benchmark measures CPU and GPU rendering capabilities, which are relevant for servers and workstations designed for high-end graphics processing and computational tasks.

The results highlight the Grando Server’s robust performance in rendering, which can be attributed to its high-powered CPU and superior GPU setup. While the Grando Workstation also performs admirably, especially in professional settings requiring detailed 3D modeling and rendering, the server’s configuration is designed for even more demanding graphics processing tasks. The performance differences reflect the server’s readiness for high-end rendering jobs, offering a valuable tool for projects with tight deadlines or complex rendering needs.

Luxmark

Luxmark is a GPU benchmark that uses LuxRender, an open-source ray tracing renderer, to evaluate a system’s performance in handling highly detailed 3D scenes. This benchmark is pertinent for assessing the graphical rendering prowess of servers and workstations, especially for visual effects and architectural visualization applications, where accurate light simulation is critical.

The Grando Server’s performance showcases its exceptional ray tracing capabilities, thanks to its advanced 6x RTX 4090 GPU setup. Although the Grando Workstation demonstrates substantial capabilities, particularly beneficial for tasks requiring detailed architectural visualization, the Server model is geared towards more intensive rendering tasks. This distinction emphasizes the server’s suitability for projects where maximizing ray tracing performance is essential, ensuring efficiency and realism in rendered scenes.

OctaneBench

OctaneBench assesses the rendering performance of GPUs using OctaneRender, a powerful and popular GPU-accelerated ray tracing renderer. This benchmark is critical for evaluating the ability of servers and workstations to perform in content creation, architectural visualization, and visual effects, where rendering speed directly impacts productivity and project timelines.

The Grando Server shines in the OctaneBench tests, indicating its exceptional suitability for demanding rendering workflows that benefit from GPU acceleration. While offering significant rendering capabilities, the Grando Workstation is optimized to balance performance and versatility in professional environments. This performance distinction underscores the server’s design for specialized tasks requiring unparalleled rendering speeds, supporting more efficient project completions.

Blackmagic RAW Speed Test

The Blackmagic RAW Speed Test measures the processing speed for high-quality video formats, an essential aspect for servers and workstations in video production and editing. It evaluates how systems manage RAW video files, affecting workflow efficiency and productivity in media production environments.

The Grando Server and Workstation demonstrate solid performance in the Blackmagic RAW Speed Test, with the server slightly edging out in CUDA-accelerated tasks. This means that the Server model offers slightly better performance for workflows involving heavy video editing and processing (especially those benefiting from GPU acceleration). However, the Workstation still provides substantial capabilities for video processing tasks, illustrating its well-rounded build for media production.

7-zip Compression

The 7-zip Compression benchmark tests a system’s efficiency in handling data compression and decompression, which is crucial for managing large datasets and optimizing storage. This benchmark reflects the performance of servers and workstations in data-intensive operations, where speed and efficiency in data manipulation are vital.

The results from the 7-zip Compression benchmark show the Grando Server and Workstation as capable performers in handling data compression and decompression tasks. Their performance indicates that the systems can efficiently manage large volumes of data, with slight differences reflecting their respective CPU configurations. These capabilities highlight the systems’ suitability for significant data management tasks, ensuring efficient storage and retrieval processes.

Y-Cruncher

Y-Cruncher is a computational benchmark that tests a system’s ability to handle complex mathematical operations, precisely calculating Pi to trillions of digits. This benchmark indicates the computational power of servers and workstations, particularly for use in scientific research and simulations requiring intensive number-crunching.

The Y-Cruncher benchmark reveals the Grando Server’s slightly superior computational capabilities, tailored for highly demanding tasks. While the Grando Workstation offers substantial computational power, suitable for a wide range of scientific and analytical applications, the server is optimized explicitly for scenarios requiring maximum computational performance. This emphasizes the Server’s role in supporting complex simulations and analyses, facilitating faster and more efficient processing of computational tasks.

Geekbench 6

Geekbench 6 measures the computational performance of CPUs and GPUs, spanning single-core and multi-core capabilities and graphical processing power. This benchmark is essential for assessing the overall computing efficiency of servers and workstations across various tasks, including simulations, data analysis, and graphics rendering.

The Geekbench 6 results illustrate a near parity in CPU single-core performance between the Grando Server and Workstation iterations, indicating comparable efficiency in tasks that rely on single-threaded processing. However, the Grando Server demonstrates superior performance in multi-core and especially GPU tests, reflecting its ability to handle more complex, parallel processing tasks and more intensive graphical computations. This suggests the server’s suitability for environments requiring robust multi-threaded applications and high-end graphics processing, enhancing productivity in compute and graphics-intensive workflows.

Cinebench R23

Cinebench R23 measures the CPU’s rendering capability, focusing on single-core and multi-core performance. It’s an essential benchmark for evaluating how well a server or workstation can perform in content creation, 3D rendering, and other CPU-intensive tasks. The MP Ratio (multi-core performance ratio) further provides insight into how effectively a system utilizes its multiple cores.

Here, the Grando Server outperforms the Workstation in multi-core tests, showcasing its superior capability to handle demanding rendering tasks that benefit from parallel processing. The slight difference in single-core results indicates similar efficiencies in tasks not requiring multiple cores. The higher MP Ratio of the Server model demonstrates its proficiency in multi-core utilization, making it adept at handling complex, multi-threaded applications in rendering and simulation projects.

Cinebench 2024

Cinebench 2024 extends the scope of its 2023 counterpart by incorporating GPU performance alongside traditional CPU benchmarks. This comprehensive strategy is essential for evaluating a system’s proficiency in managing the rigorous demands of current 3D content creation and rendering activities. It offers an all-encompassing perspective on its computational and graphical rendering capabilities.

The Grando Server shows impressive performance in GPU and CPU multi-core benchmarks, indicating its strong suit in handling complex rendering tasks that utilize extensive resources. Conversely, the Grando Workstation demonstrates commendable efficiency, especially considering its architecture and configuration compared to the Server model. The close CPU single-core scores between the two systems suggest a comparable efficiency for tasks that do not heavily rely on multi-core processing. Lastly, the MP Ratio score of 49.49x underscores the Server model’s adeptness at maximizing its multi-core configuration, making it ideal for demanding projects requiring high CPU and GPU throughput to achieve optimal performance and speed.

Virtualized Desktop Performance

In addition to the benchmarks looking at the overall system performance, we wanted to virtualize the Grando Server system to share out six virtual desktops, each with a dedicated 4090 GPU. We deployed Windows Server 2022 Standard and set up six virtual machines (VMs), one for each GPU, allowing us to distribute tasks efficiently and test the system’s true capabilities.

By setting up a VM for each RTX 4090 GPU, we create an isolated environment that mimics a dedicated workstation, allowing for a focused evaluation of each GPU’s performance. Below is the powershell process of passthrough process.

This setup is also handy for identifying the strengths and potential bottlenecks in a multi-GPU system, as it ensures that the performance of each card can be assessed independently without interference from shared resources.

LuxMark

The LuxMark benchmark results showcase a solid range of scores from 38,599 to 40,797 in Hallbench. This demonstrates a remarkable consistency in performance, with slight variations likely due to operational fluctuations or VM-specific configurations. The highest scores (VM1 and VM4) highlight the peak performance capabilities of the RTX 4090 GPUs for demanding ray tracing tasks, likewise for the Food category.

Overall, these results show the Comino server’s capability for high-end computing tasks using a single GPU, offering both power and precision across multiple, simultaneous workloads. When you compare the aggregate score against the standalone client system performance, the 6 individual systems did very well. We measured 241,183 in Hallbench and 103,399 in Food in aggregate against the standalone result of 240,303 and 92,156 respectively.

Cinebench 2024

The Cinebench 2024 benchmark allows for an assessment of how each GPU handles complex rendering tasks under a virtualized environment. The focus in this workload is really the GPU, as we didn’t fully provision out the available host CPU resources to each VM. Here we measured an aggregate GPU score of 183,348 pts, against the client system which measured 140,842.

The GPU scores range from 29,406 to 31,178 points, which demonstrates a high degree of performance consistency. The CPU multi-core scores are tightly grouped between 384 and 394 points, with corresponding single-core scores of 81 to 82 points, showcasing the server’s ability to handle intensive tasks effectively across all VMs.

The multi-core to single-core performance ratio (MP ratio) varies slightly from 4.74x to 4.83x, reflecting a balanced distribution of processing power that benefits multi-threaded rendering tasks. As for the scores themselves, overall, these results showcase the Comino Grando Server’s solid performance in a virtualized environment, making it well-suited for demanding 3D rendering and computational tasks where efficiency and speed are paramount.

Blender 4.0

During the Blender benchmark, each VM demonstrated consistent performance across three different scenes: Monster, Junkshop, and Classroom. VM6 stands out with the highest score in the Monster scene at 5,949, indicating a slight variance in performance possibly due to VM resource allocation or slight differences in workload handling. The Junkshop scene scores are relatively close, with VM5 showing the lowest score at 2,656. Classroom scene results are more varied, with VM2 achieving the highest score at 2,868, pointing towards differences in handling geometry-rich scenes.

When we compare these results to the non-virtualized Server, which wasn’t able to scale the workload in Blender, we see much greater performance and utilization. The client system topped out at 5,881 in Monster, 2,809 in Junkshop and 2,896 in Classroom, whereas in aggregate in our Windows HyperV deployment, we measured 34,729 in Monster, 16,270 in Junkshop and 16,836 in Classroom.

Conclusion

The Comino Grando Server and Workstation are prime examples of doing GPU-driven performance systems right, catering to the high-demand realms of data analysis, AI research, and graphic-intensive tasks. The Server model, configured with six NVIDIA RTX 4090s, is a powerhouse designed for heavy computational work, complemented by AMD’s Threadripper PRO 5995WX CPU and 512GB of DDR5 memory. This configuration makes it a fantastic choice for server environments requiring parallel processing and high-speed data manipulation.

In contrast, though slightly less core-abundant, the Workstation model balances the equation with four NVIDIA A100 GPUs, demonstrating its prowess in detailed 3D modeling and deep learning tasks. This system design is slightly different, allowing for more silent operation. To be fair though, they’re both extremely quiet in our lab compared to any air-cooled server.

Performance-wise, the benchmark results paint a clear picture: the Grando Server excels in GPU-intensive tests like OctaneBench and Blender 4.0, highlighting its capability to breeze through high-end rendering jobs. With its tailored CPU-GPU balance, the workstation version offers versatility for various professional applications. The speed and efficiency of both systems in managing large datasets, as evidenced by the 7-zip Compression and Y-Cruncher benchmarks, underscore their capacity for handling data-intensive operations, a critical advantage in today’s data-driven landscape.

The flexibility of the systems was further underscored when we turned the 6x 4090 system into a VDI box. We saw consistent performance there too, which means an organization could effectively divide up any of these systems to support multiple remote desktops with dedicated GPU access. This use case is entirely enabled by the liquid cooling plates, which is the only way to get this kind of GPU density into a single box.

In their entirety, these Comino Grando systems are some of the most impressive we’ve seen. Yes, there are plenty of enterprise server alternatives that are liquid-cooled, but that’s not the goal here. These Comino systems are about extreme flexibility in configuration (client and enterprise GPU options) and rack density/efficiency. The overall execution on these fronts is extremely impressive, making the Comino Grando systems one of our Best of 2024 award winners.

Comino Grando Systems

Engage with StorageReview

Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed