Titan: Fair Packet Scheduling for Commodity Multiqueue NICs

Abstract

The performance of an OS’s networking stack can be measured by its achieved throughput, CPU utilization, latency, and per-flow fairness. To be able to drive increasing line-rates at 10Gbps and beyond, modern OS networking stacks rely on a number of important hardware and software optimizations, including but not limited to using multiple transmit and receive queues and segmentation offloading. Unfortunately, it not clear how best to leverage these optimizations to extract performance. The first contribution of this paper is a detailed empirical study of the impact of different OS and NIC configurations on this four-dimensional trade-off space. We find that enabling certain specific features is crucial for latency, CPU utilization, and throughput. However, substantial flow-level unfairness still remains. The second contribution of this paper is Titan, an extension to the Linux networking stack that systematically addresses unfairness arising in different operating conditions, while minimally impacting CPU utilization, latency, and throughput.