CNA Performance Testing: Your Mileage May Vary
When deploying FCoE in virtualized server environments, not stomping on the CPU is important
By Deni Connor and James E. Bagley
Senior Analysts
Storage Strategies NOW
April 2010
When configuring a server that will be virtualized and placed in a Fibre Channel over Ethernet (FCoE) environment, it’s important to tune the server for optimal CPU utilization and balanced IOPs performance. Typically a server reaching the upper end of its CPU utilization adversely impacts the applications running on the server as they become sluggish or unresponsive. Servers that are deployed without virtualization are normally underutilized. By virtualizing the server, utilization can increase to somewhere north of 50% and leave adequate headroom for processing IO between the server and the converged Fibre Channel and Ethernet fabric.
Test results from Emulex and QLogic for their converged network adapters (CNAs) attempt to do just that – show how a server will perform in a virtual environment and run applications optimally on its virtual machines.
Both companies tested their converged network adapters using IOMETER and showed decidedly different results. QLogic’s tests focused on server utilization and balanced IOPs which are both important, while Emulex’s report — http://www.emulex.com/emulexlabs/?p=417 – focused only on IOPs and paid little if any attention to processor utilization.
Emulex’s results show a dramatic difference in the performance of its adapter in simulated 512B and 1KB block environments compared to QLogic’s QLE8152 converged network adapter (CNA). In real-world environments such as Microsoft Exchange, which uses 4KB to 8KB blocks, the IOPs performance is essentially equal.
FCoE implementation differences account for this performance variance at smaller block sizes. QLogic, which was the first to release an integrated single chip controller and firmware operating on the adapter to perform all FCoE functions, off-loads the work from the server and subsequently preserves CPU cycles for applications.
Emulex had to port their existing FC firmware and driver to a 3rd party owned 10GbE controller, which resulted in making architectural tradeoff’s, forcing Emulex to move some of the functionality to the driver from the firmware. When the company introduced the OCe10102 CNA, they did a ‘pay as you go’ release — first providing a base 10Gbit Ethernet adapter and later providing an FCoE upgrade, which required the user to shut down the server, update the firmware image in the flash on the card and load the new FCoE driver.
While this was an expeditious way to distribute the upgrade (and get revenue from the 10Gbit Ethernet product), leveraging IP from a 3rd party and trying to port an existing firmware/driver to accommodate FCoE comes at the expense of server utilization. Server CPU cycles should be reserved for processing workloads, not to doing housekeeping typically done in the adapter.
QLogic’s test results in an unconstrained environment using real world storage suggests that while Emulex’s OCe10102 beat out the QLogic adapter in sheer IOPs, it was at the expense of very high CPU utilization. All data points shown below are for a sequential read workload.
In tests using 512B to 2KB blocks, the Emulex CNA consumed 50 to 81% of the system CPU (8-core Nehalem server), as shown below. The data shows that in a real world deployment, the server CPU has little headroom to process applications if using the Emulex UCNA. QLogic noted that in some tests with less powerful CPUs akin to existing enterprise datacenters today, the UCNA consumed over 90-95% of the system CPU. Based on this data, Emulex’s UCNA appears to be a server choke point in a typical enterprise server environment. Furthermore we need to consider how little head room would be available for real applications in a deployment scenario where the end user is running TCP/IP, FCoE and iSCSI traffic through the UCNA?
To provide an apples to apples comparison, IOPS should be divided by the percent CPU utilization. In this measurement, QLogic outperforms Emulex at all block sizes, implying that if server administrators were to allocate a fixed amount of CPU/memory resources for IO to both CNAs, QLogic would be able to push higher traffic through the 10GbE network.
SSG-NOW Assessment
Performance test results need to be compared on more than a single dimension. While the QLogic testing confirms Emulex’s UCNA pumps out a bit more IOPs on tiny blocks, when CPU utilization is examined it is clear that these data rates cannot be sustained in a real world environment. It is our opinion that these tests confirm the overall benefit of the QLogic implementation – optimal server CPU utilization balanced against IOPs, which is required for the deployment of virtual machines in both current and next generation data centers. As data centers strive for higher server consolidation with virtualization, the number of available processors impacts the assignment of the virtualized hosts and their applications. When a server consumes a significant amount of its processors’ CPU cycles, it creates a bottleneck for server consolidation in the data center while adversely affecting the normal operation of the server applications.
Clearly, the ability to own its FCoE ASIC, Fibre Channel IP and Ethernet IP gives QLogic complete control over its technology roadmap and feature introduction, whereas some of Emulex’s performance disadvantages can be attributed to the fact that the company relies upon Server Engines for its FCoE ASIC and Ethernet stack. For virtualized environments looking to network convergence as the next step of data center efficiency, QLogic has the optimal solution today.
QLogic will soon have the complete data set and system configuration used in these benchmarks posted to their website.
One Response to “CNA Performance Testing: Your Mileage May Vary”
Leave a Reply
[...] This post was mentioned on Twitter by John Furrier and QLogic Global, dconnor. dconnor said: Converged Network Adapter Performace Testing: You Mileage may vary http://bit.ly/9ovhLX [...]