This series adds net/dynthread, a loadable module that adapts the NAPI processing and the RX queue IRQ placement of a set of net devices to the scheduling behaviour of a target application at runtime. Motivation ---------- Interrupt and softirq processing of network RX competes with latency sensitive applications (e.g. mysqld) for CPU time. When such an application runs for a long time on a CPU without blocking, the softirq work of the NIC completion queues can preempt it on the same CPU and inflate its tail latency. The usual remedies are to steer the RX queue IRQs to other CPUs and/or to move NAPI processing into kernel threads bound to spare CPUs, but a static configuration cannot follow the application: once the target goes idle, is scheduled in short bursts, or is displaced, the reserved CPUs are simply wasted. Approach -------- The module periodically samples the scheduling statistics of all threads matching the appname module parameter and classifies them into one of five statuses (DATA_LOSS, NO_TASK, NOT_RUNNING, FRAGMENTATION, LONG_RUNNING_WITH_INTERRUPTS). Acting on that status it: - restores the RX queue IRQs to their original affinity when the target is gone or barely running; - spreads the IRQs over the NUMA node of their queues when processing in softirq context is acceptable again; - switches the NAPI processing of the configured net devices to kernel threads and binds the NAPI threads (and, optionally, the queue IRQs) to the CPUs of their NUMA node that the target tasks are not allowed to use; - scales the number of CPUs used by those threads with their measured load while the target stays in the long-running status. An unusable sample (counters moved backwards: DATA_LOSS) keeps the previous status, so a single bad sample cannot toggle the mode. Everything is controlled through module parameters, and the original IRQ affinity and NAPI mode are restored when the target disappears or the module is unloaded. The module is buildable only as a loadable module; the Kconfig entry currently allows ARM64. Patch breakdown --------------- 1/6 net: dynthread: Add dynamic threaded NAPI scheduler framework Kconfig/Makefile and the module skeleton: record the RX queue IRQs of the configured net devices and their original affinity, and count the NUMA nodes those queues span. 2/6 net: dynthread: Monitor the load of the target tasks The sampling loop and the status judgement. 3/6 net: dynthread: Steer the RX queue IRQs away from the target tasks Manage the affinity of the recorded IRQs based on the status and the placement of the target tasks. 4/6 net: dynthread: Switch NAPI to threads on spare CPUs Enable threaded NAPI and bind the NAPI threads (and optionally the queue IRQs) to the CPUs not used by the target tasks; switch back to softirq mode in the non-long-running statuses. 5/6 net: dynthread: Scale the number of NAPI thread CPUs Adapt the number of CPUs used by the NAPI threads to their load. 6/6 net: dynthread: Add debug switch and instrumentation Add the dyn_threaded_debug module parameter and the ftrace based instrumentation of the sampling, the status transitions and the IRQ/NAPI placement decisions. Amritha Nambiar (2): net: Add queue and napi association net: Add NAPI IRQ support Cai Xinchen (6): net: dynthread: Add dynamic threaded NAPI scheduler framework net: dynthread: Monitor the load of the target tasks net: dynthread: Steer the RX queue IRQs away from the target tasks net: dynthread: Switch NAPI to threads on spare CPUs net: dynthread: Scale the number of NAPI thread CPUs net: dynthread: Add debug switch and instrumentation Carolina Jubran (1): net/mlx5e: Acquire RTNL lock before RQs/SQs activation/deactivation Joe Damato (1): net/mlx5e: link NAPI instances to queues and IRQs .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 5 +- .../mellanox/mlx5/core/en/reporter_tx.c | 7 + .../net/ethernet/mellanox/mlx5/core/en_main.c | 7 + include/linux/netdevice.h | 20 +- include/net/netdev_rx_queue.h | 2 +- net/core/dev.c | 38 + net/dynthread/Kconfig | 13 + net/dynthread/Makefile | 2 + net/dynthread/net_dynthread.c | 1258 +++++++++++++++++ 9 files changed, 1349 insertions(+), 3 deletions(-) create mode 100644 net/dynthread/Kconfig create mode 100644 net/dynthread/Makefile create mode 100644 net/dynthread/net_dynthread.c -- 2.18.0.huawei.25