July 2025 - Kernel - mailweb.openeuler.org

[PATCH OLK-6.6] drm: phytium: fix NULL dereference issue in drm_gem_object_free
by Li Chen 18 Jul '25

18 Jul '25

From: Li Chen <chenl311(a)chinatelecom.cn> driver inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/ICGG2A ---------------------------------------------------------------------- Syzkaller crashed kernel in drm path. The root cause is that phytium_drm_gem_object_funcs is not assigned before phytium_gem_create_object enters the failed_dma_alloc label. Let's fix this issue by assigning the function earlier. Below is the crash log: ``` [ 9042.703078] [drm:phytium_gem_create_object [phytium_dc_drm]] *ERROR* fail to allocate vram buffer with size 3de4000 [ 9042.717862] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 9042.727173] fuse: Unknown parameter '0xffffffffffffffff<r00000000000000000000' [ 9042.730383] Mem abort info: [ 9042.745443] ESR = 0x0000000096000006 [ 9042.745446] EC = 0x25: DABT (current EL), IL = 32 bits [ 9042.745448] SET = 0, FnV = 0 [ 9042.745450] EA = 0, S1PTW = 0 [ 9042.745451] FSC = 0x06: level 2 translation fault [ 9042.745453] Data abort info: [ 9042.745455] ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000 [ 9042.745457] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 9042.745459] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 [ 9042.745462] user pgtable: 4k pages, 48-bit VAs, pgdp=00003000894ca000 [ 9042.745464] [0000000000000000] pgd=08003000894cb403, p4d=08003000894cb403, pud=0800300085320403, pmd=0000000000000000 [ 9042.830042] Internal error: Oops: 0000000096000006 [#1] SMP [ 9042.838310] Modules linked in: cramfs camellia_generic serpent_generic blowfish_generic blowfish_common cast5_generic cast_common des_generic libdes rmd160 tcp_bic unix_diag ansi_cprng tcp_dctcp ppp_synctty ip_set_hash_ip n_hdlc cmac pps_ldisc n_gsm nfnetlink_log slcan tcp_diag nfnetlink_cthelper atm nfsd auth_rpcgss nfs_acl twofish_generic twofish_common ccm md4 ppp_async msdos nfs lockd grace fscache crc32_generic netfs smc_diag tcp_westwood smc nfnetlink_osf vfio_iommu_type1 vfio vhost_vsock iommufd squashfs ib_core gfs2 snd_timer snd soundcore uhid nfnetlink_cttimeout pppoe ip_vs cuse can_bcm loop can_raw can vsock_loopback inet_diag vmw_vsock_virtio_transport_common vhost_net vhost ieee802154_socket vsock cfg80211 uinput ieee802154 vhost_iotlb pptp crypto_user l2tp_ppp pppox sctp ppp_generic slhc af_key ip6_vti ip_vti ipip sit geneve macvtap tap ipvlan macvlan hsr xfrm_interface xfrm6_tunnel tunnel4 wireguard libchacha20poly1305 chacha_neon poly1305_neon libcurve25519_generic libchacha nlmon team vcan can_dev tun [ 9042.838608] xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables overlay nfnetlink_queue authenc echainiv cls_matchall esp6 l2tp_ip6 l2tp_eth l2tp_ip l2tp_netlink l2tp_core br_netfilter sch_etf sch_fq dccp_ipv6 dccp_ipv4 dccp sch_ingress act_mirred cls_basic veth bonding tls esp4_offload esp4 psample macsec vxlan ip6_udp_tunnel udp_tunnel vrf 8021q garp mrp ip6_gre ip6_tunnel tunnel6 ip_gre ip_tunnel gre cls_u32 sch_htb dummy binfmt_misc bridge stp llc rfkill ip_set libcrc32c sunrpc vfat fat ipmi_si ipmi_devintf phytium_dc_drm ses enclosure ipmi_msghandler drm_display_helper scsi_transport_sas cec drm_kms_helper cppc_cpufreq sg drm fuse nfnetlink ext4 mbcache jbd2 sd_mod t10_pi crc64_rocksoft_generic crc64_rocksoft crc64 crct10dif_ce ghash_ce sm4_ce_gcm sm4_ce_ccm sm4_ce sm4_ce_cipher sm4 sm3_ce sha3_ce sha512_ce ahci sha512_arm64 sha2_ce libahci sha256_arm64 ice sha1_ce igb sbsa_gwdt libata megaraid_sas i2c_algo_bit i2c_core [ 9042.958095] dm_mirror dm_region_hash dm_log dm_multipath dm_mod aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher [last unloaded: nf_tables] [ 9043.094331] CPU: 39 PID: 127275 Comm: syz-executor.3 Kdump: loaded Not tainted 6.6.0-0001.rc3.ctl4.aarch64 #1 [ 9043.108430] Hardware name: vclusters VSFT5000 B/VSFT5000 B, BIOS KL4.2A.RC.D.170.240314.D.DX 03/14/2024 18:01:48 [ 9043.122832] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 9043.132983] pc : drm_gem_object_free+0xc/0x40 [drm] [ 9043.140480] lr : phytium_gem_create_object+0x2ac/0x338 [phytium_dc_drm] [ 9043.150238] sp : ffff80009cd83bd0 [ 9043.155639] x29: ffff80009cd83bd0 x28: 00000000000000b2 x27: ffff80009cd83ce8 [ 9043.166023] x26: 0000000000000020 x25: 0000000000000020 x24: ffff80007b4b2d78 [ 9043.176292] x23: ffff00ff8de5b000 x22: 0000000003de4000 x21: 0000000000000000 [ 9043.186548] x20: ffff00ff8df94c80 x19: fffffffffffffff4 x18: ffffffffffffffff [ 9043.196768] x17: 6f74206c69616620 x16: 2a524f5252452a20 x15: 5d5d6d72645f6364 [ 9043.207193] x14: 5f6d756974796870 x13: 205d353732373231 x12: 545b5d3837303330 [ 9043.217647] x11: 00000000ffff7fff x10: ffff80008223b900 x9 : ffff80007b5cdcb4 [ 9043.228070] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 00000000002bffa8 [ 9043.238522] x5 : ffff80008345bd08 x4 : 0000000000000000 x3 : 0000000000000000 [ 9043.248718] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff200014f5b800 [ 9043.258874] Call trace: [ 9043.263132] drm_gem_object_free+0xc/0x40 [drm] [ 9043.270239] phytium_gem_dumb_create+0x60/0x160 [phytium_dc_drm] [ 9043.279194] drm_mode_create_dumb_ioctl+0x98/0xc0 [drm] [ 9043.287188] drm_ioctl_kernel+0xdc/0x188 [drm] [ 9043.294188] drm_ioctl+0x274/0x540 [drm] [ 9043.300456] __arm64_sys_ioctl+0xb4/0x100 [ 9043.306777] invoke_syscall+0x50/0x128 [ 9043.312774] el0_svc_common.constprop.0+0xc8/0xf0 [ 9043.319953] do_el0_svc+0x24/0x38 [ 9043.325324] el0_svc+0x44/0x1b8 [ 9043.330517] el0t_64_sync_handler+0x100/0x130 [ 9043.337369] el0t_64_sync+0x188/0x190 [ 9043.343315] Code: ffff00ff aa1e03e9 952fe875 f940a001 (f9400021) [ 9043.352351] SMP: stopping secondary CPUs [ 9043.412513] Starting crashdump kernel... [ 9044.001381] Bye! ``` Disassembler drm_gem_object_free: ``` 0xffff80007b437398 <drm_gem_object_free>: mov x9, x30 0xffff80007b43739c <drm_gem_object_free+4>: bl 0xffff800080031570 <ftrace_caller> 0xffff80007b4373a0 <drm_gem_object_free+8>: ldr x1, [x0, #320] 0xffff80007b4373a4 <drm_gem_object_free+12>: ldr x1, [x1] 0xffff80007b4373a8 <drm_gem_object_free+16>: cbz x1, 0xffff80007b4373c8 <drm_gem_object_free+48> 0xffff80007b4373ac <drm_gem_object_free+20>: paciasp 0xffff80007b4373b0 <drm_gem_object_free+24>: stp x29, x30, [sp, #-16]! 0xffff80007b4373b4 <drm_gem_object_free+28>: mov x29, sp 0xffff80007b4373b8 <drm_gem_object_free+32>: blr x1 0xffff80007b4373bc <drm_gem_object_free+36>: ldp x29, x30, [sp], #16 0xffff80007b4373c0 <drm_gem_object_free+40>: autiasp 0xffff80007b4373c4 <drm_gem_object_free+44>: ret 0xffff80007b4373c8 <drm_gem_object_free+48>: brk #0x800 0xffff80007b4373cc <drm_gem_object_free+52>: ret 0xffff80007b4373d0 <drm_gem_object_free+56>: .inst 0x865baea8 ; undefined 0xffff80007b4373d4 <drm_gem_object_free+60>: .inst 0xffff00ff ; undefined ``` ldr x1, [x1] <-- trapping instruction Signed-off-by: Li Chen <chenl311(a)chinatelecom.cn> Reviewed-by: Bin Lai <laib2(a)chinatelecom.cn> Reviewed-by: Shuo Li <lishuo(a)phytium.com.cn> --- drivers/gpu/drm/phytium/phytium_gem.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/phytium/phytium_gem.c b/drivers/gpu/drm/phytium/phytium_gem.c index 2cbbcd9fbd112..95b6b0360c065 100644 --- a/drivers/gpu/drm/phytium/phytium_gem.c +++ b/drivers/gpu/drm/phytium/phytium_gem.c @@ -432,6 +432,8 @@ struct phytium_gem_object *phytium_gem_create_object(struct drm_device *dev, uns goto failed_object_init; } + phytium_gem_obj->base.funcs = &phytium_drm_gem_object_funcs; + if (priv->support_memory_type & (MEMORY_TYPE_VRAM_WC | MEMORY_TYPE_VRAM_DEVICE)) { ret = phytium_memory_pool_alloc(priv, &phytium_gem_obj->vaddr, &phytium_gem_obj->phys_addr, size); @@ -475,8 +477,6 @@ struct phytium_gem_object *phytium_gem_create_object(struct drm_device *dev, uns goto failed_dma_alloc; } - phytium_gem_obj->base.funcs = &phytium_drm_gem_object_funcs; - phytium_gem_obj->size = size; list_add_tail(&phytium_gem_obj->list, &priv->gem_list_head); DRM_DEBUG_KMS("phytium_gem_obj iova:0x%pa size:0x%lx\n", -- 2.49.0

1 0

[PATCH openEuler-1.0-LTS 0/2] Fix the race of lock/unlock and open for nfs
by Li Lingfeng 18 Jul '25

18 Jul '25

Fix the race of lock/unlock and open for nfs. Li Lingfeng (2): nfs: fix the race of lock/unlock and open nfs: do not unlock when no lock is held fs/nfs/delegation.c | 5 ++++- fs/nfs/nfs4proc.c | 8 +++++++- 2 files changed, 11 insertions(+), 2 deletions(-) -- 2.31.1

2 3

[openeuler:OLK-6.6 2527/2527] drivers/media/v4l2-core/v4l2-subdev.c:2253:10: error: call to undeclared function 'v4l2_subdev_get_locked_active_state'; ISO C99 and later do not support implicit function declarations
by kernel test robot 18 Jul '25

18 Jul '25

tree: https://gitee.com/openeuler/kernel.git OLK-6.6 head: cbc572af8c3daa87a87df19c570fb3fd3ed99022 commit: c1be6d998806425f28600e867a8e9a2dd9bac741 [2527/2527] media: subdev: Add v4l2_subdev_is_streaming() config: x86_64-buildonly-randconfig-004-20250718 (https://download.01.org/0day-ci/archive/20250718/202507181525.PYgxRhvq-lkp@…) compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250718/202507181525.PYgxRhvq-lkp@…) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202507181525.PYgxRhvq-lkp@intel.com/ All errors (new ones prefixed by >>): >> drivers/media/v4l2-core/v4l2-subdev.c:2253:10: error: call to undeclared function 'v4l2_subdev_get_locked_active_state'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] 2253 | state = v4l2_subdev_get_locked_active_state(sd); | ^ >> drivers/media/v4l2-core/v4l2-subdev.c:2253:8: error: incompatible integer to pointer conversion assigning to 'struct v4l2_subdev_state *' from 'int' [-Wint-conversion] 2253 | state = v4l2_subdev_get_locked_active_state(sd); | ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2 errors generated. vim +/v4l2_subdev_get_locked_active_state +2253 drivers/media/v4l2-core/v4l2-subdev.c 2242 2243 bool v4l2_subdev_is_streaming(struct v4l2_subdev *sd) 2244 { 2245 struct v4l2_subdev_state *state; 2246 2247 if (!v4l2_subdev_has_op(sd, pad, enable_streams)) 2248 return sd->s_stream_enabled; 2249 2250 if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) 2251 return !!sd->enabled_pads; 2252 > 2253 state = v4l2_subdev_get_locked_active_state(sd); 2254 2255 for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) { 2256 const struct v4l2_subdev_stream_config *cfg; 2257 2258 cfg = &state->stream_configs.configs[i]; 2259 2260 if (cfg->enabled) 2261 return true; 2262 } 2263 2264 return false; 2265 } 2266 EXPORT_SYMBOL_GPL(v4l2_subdev_is_streaming); 2267 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[PATCH openEuler-1.0-LTS] net/sched: Abort __tc_modify_qdisc if parent class does not exist
by Dong Chenchen 18 Jul '25

18 Jul '25

From: Victor Nogueira <victor(a)mojatatu.com> mainline inclusion from mainline-v6.16-rc6 commit ffdde7bf5a439aaa1955ebd581f5c64ab1533963 category: bugfix bugzilla: 190701 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- Lion's patch [1] revealed an ancient bug in the qdisc API. Whenever a user creates/modifies a qdisc specifying as a parent another qdisc, the qdisc API will, during grafting, detect that the user is not trying to attach to a class and reject. However grafting is performed after qdisc_create (and thus the qdiscs' init callback) is executed. In qdiscs that eventually call qdisc_tree_reduce_backlog during init or change (such as fq, hhf, choke, etc), an issue arises. For example, executing the following commands: sudo tc qdisc add dev lo root handle a: htb default 2 sudo tc qdisc add dev lo parent a: handle beef fq Qdiscs such as fq, hhf, choke, etc unconditionally invoke qdisc_tree_reduce_backlog() in their control path init() or change() which then causes a failure to find the child class; however, that does not stop the unconditional invocation of the assumed child qdisc's qlen_notify with a null class. All these qdiscs make the assumption that class is non-null. The solution is ensure that qdisc_leaf() which looks up the parent class, and is invoked prior to qdisc_create(), should return failure on not finding the class. In this patch, we leverage qdisc_leaf to return ERR_PTRs whenever the parentid doesn't correspond to a class, so that we can detect it earlier on and abort before qdisc_create is called. [1] https://lore.kernel.org/netdev/d912cbd7-193b-4269-9857-525bee8bbb6a@gmail.c… Fixes: 5e50da01d0ce ("[NET_SCHED]: Fix endless loops (part 2): "simple" qdiscs") Reported-by: syzbot+d8b58d7b0ad89a678a16(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/netdev/68663c93.a70a0220.5d25f.0857.GAE@google.com/ Reported-by: syzbot+5eccb463fa89309d8bdc(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/netdev/68663c94.a70a0220.5d25f.0858.GAE@google.com/ Reported-by: syzbot+1261670bbdefc5485a06(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/netdev/686764a5.a00a0220.c7b3.0013.GAE@google.com/ Reported-by: syzbot+15b96fc3aac35468fe77(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/netdev/686764a5.a00a0220.c7b3.0014.GAE@google.com/ Reported-by: syzbot+4dadc5aecf80324d5a51(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/netdev/68679e81.a70a0220.29cf51.0016.GAE@google.com/ Acked-by: Jamal Hadi Salim <jhs(a)mojatatu.com> Reviewed-by: Cong Wang <xiyou.wangcong(a)gmail.com> Signed-off-by: Victor Nogueira <victor(a)mojatatu.com> Link: https://patch.msgid.link/20250707210801.372995-1-victor@mojatatu.com Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Conflicts: net/sched/sch_api.c [commit d636fc5dd692 and 2561f97267d6 is not backport] Signed-off-by: Dong Chenchen <dongchenchen2(a)huawei.com> --- net/sched/sch_api.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index 976908642263..05bed30b0a6a 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -333,18 +333,23 @@ struct Qdisc *qdisc_lookup_rcu(struct net_device *dev, u32 handle) return q; } -static struct Qdisc *qdisc_leaf(struct Qdisc *p, u32 classid) +static struct Qdisc *qdisc_leaf(struct Qdisc *p, u32 classid, + struct netlink_ext_ack *extack) { unsigned long cl; struct Qdisc *leaf; const struct Qdisc_class_ops *cops = p->ops->cl_ops; - if (cops == NULL) - return NULL; + if (cops == NULL) { + NL_SET_ERR_MSG(extack, "Parent qdisc is not classful"); + return ERR_PTR(-EOPNOTSUPP); + } cl = cops->find(p, classid); - if (cl == 0) - return NULL; + if (cl == 0) { + NL_SET_ERR_MSG(extack, "Specified class not found"); + return ERR_PTR(-ENOENT); + } leaf = cops->leaf(p, cl); return leaf; } @@ -1378,7 +1383,7 @@ static int tc_get_qdisc(struct sk_buff *skb, struct nlmsghdr *n, NL_SET_ERR_MSG(extack, "Failed to find qdisc with specified classid"); return -ENOENT; } - q = qdisc_leaf(p, clid); + q = qdisc_leaf(p, clid, extack); } else if (dev_ingress_queue(dev)) { q = dev_ingress_queue(dev)->qdisc_sleeping; } @@ -1389,6 +1394,8 @@ static int tc_get_qdisc(struct sk_buff *skb, struct nlmsghdr *n, NL_SET_ERR_MSG(extack, "Cannot find specified qdisc on specified device"); return -ENOENT; } + if (IS_ERR(q)) + return PTR_ERR(q); if (tcm->tcm_handle && q->handle != tcm->tcm_handle) { NL_SET_ERR_MSG(extack, "Invalid handle"); @@ -1485,7 +1492,9 @@ static int tc_modify_qdisc(struct sk_buff *skb, struct nlmsghdr *n, NL_SET_ERR_MSG(extack, "Failed to find specified qdisc"); return -ENOENT; } - q = qdisc_leaf(p, clid); + q = qdisc_leaf(p, clid, extack); + if (IS_ERR(q)) + return PTR_ERR(q); } else if (dev_ingress_queue_create(dev)) { q = dev_ingress_queue(dev)->qdisc_sleeping; } -- 2.25.1

2 1

[PATCH OLK-6.6 0/2] CVE-2025-38052
by Wang Liang 18 Jul '25

18 Jul '25

Charalampos Mitrodimas (1): net: tipc: fix refcount warning in tipc_aead_encrypt Wang Liang (1): net/tipc: fix slab-use-after-free Read in tipc_aead_encrypt_done net/tipc/crypto.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.34.1

2 3

[PATCH OLK-6.6] ioinf: inflight based weighted relative controller
by Yu Kuai 18 Jul '25

18 Jul '25

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/ICN84T -------------------------------- The demo version 2. Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> --- block/Kconfig | 5 + block/Makefile | 1 + block/blk-ioinf.c | 1280 ++++++++++++++++++++++++++++++++++++++++ block/blk-mq-debugfs.c | 2 + block/blk-rq-qos.h | 1 + include/linux/blk-mq.h | 3 + 6 files changed, 1292 insertions(+) create mode 100644 block/blk-ioinf.c diff --git a/block/Kconfig b/block/Kconfig index 7018fdcaa459..fc57f790d353 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -204,6 +204,11 @@ config BLK_CGROUP_LEGACY_IOCOST If unsure, say N. ++config BLK_CGROUP_IOINFLIGHT ++ bool "Enable support for inflight based cgroup IO controller" ++ help ++ xxxx + config BLK_CGROUP_IOPRIO bool "Cgroup I/O controller for assigning an I/O priority class" depends on BLK_CGROUP diff --git a/block/Makefile b/block/Makefile index bfba1d2afc0e..eeea4092a5c0 100644 --- a/block/Makefile +++ b/block/Makefile @@ -21,6 +21,7 @@ obj-$(CONFIG_BLK_DEV_THROTTLING) += blk-throttle.o obj-$(CONFIG_BLK_CGROUP_IOPRIO) += blk-ioprio.o obj-$(CONFIG_BLK_CGROUP_IOLATENCY) += blk-iolatency.o obj-$(CONFIG_BLK_CGROUP_IOCOST) += blk-iocost.o +obj-$(CONFIG_BLK_CGROUP_IOINFLIGHT) += blk-ioinf.o obj-$(CONFIG_MQ_IOSCHED_DEADLINE) += mq-deadline.o obj-$(CONFIG_MQ_IOSCHED_KYBER) += kyber-iosched.o bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o diff --git a/block/blk-ioinf.c b/block/blk-ioinf.c new file mode 100644 index 000000000000..5b0f422c0310 --- /dev/null +++ b/block/blk-ioinf.c @@ -0,0 +1,1280 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * IO inflight relative controller + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/timer.h> +#include <linux/time64.h> +#include <linux/parser.h> +#include <linux/blk-cgroup.h> + +#include "blk-cgroup.h" +#include "blk-rq-qos.h" +#include "blk-mq.h" + +/* default weight for each cgroup */ +#define IOINF_DFL_WEIGHT 10 +/* default wake up time in jiffies for backgroup job, see ioinf_timer_fn() */ +#define IOINF_TIMER_PERID 500 +/* default time in jiffies that cgroup will idle without any IO */ +#define INFG_DFL_EXPIRE 100 +/* minimal number of samples for congestion control */ +#define MIN_SAMPLES 100 +/* io.inf.qos controls */ +enum { + INF_ENABLE, + INF_INFLIGHT, + + QOS_ENABLE, + QOS_RLAT, + QOS_WLAT, + QOS_RPCT, + QOS_WPCT, + + NR_QOS_CTRL_PARAMS, +}; + +/* ioinf_gq flags */ +enum { + INFG_EXHAUSTED, + INFG_LEND, + INFG_BORROW, + INFG_OFFLINE, +}; + +/* qos control params */ +struct ioinf_params { + bool enabled; + bool qos_enabled; + u32 inflight; + u64 rlat; + u64 wlat; + u32 rpct; + u32 wpct; +}; + +struct ioinf_lat_stat { + u64 rmet; + u64 wmet; + u64 rmissed; + u64 wmissed; +}; + +struct infg_lat_stat { + atomic_long_t latency; +}; + +/* the global conrtol structure */ +struct ioinf { + struct rq_qos rqos; + + struct ioinf_params params; + /* real inflight with consideration of busy_level */ + u32 inflight; + + /* default time for ioinf_timer_fn */ + unsigned long inf_timer_perid; + struct timer_list inf_timer; + + /* default time for infg_expire_fn */ + unsigned long infg_expire_jiffies; + + /* global lock */ + spinlock_t lock; + /* list of active infgs */ + struct list_head active_infgs; + /* list of active infgs that lend inflight budget to other infgs */ + struct list_head lend_infgs; + /* list of active infgs that borrow inflight budget from other infgs */ + struct list_head borrow_infgs; + + /* for offline cgroups */ + u32 offline_hinflight; + struct rq_wait offline_rqw; + + struct ioinf_lat_stat last_stat; + struct ioinf_lat_stat __percpu *stat; + + int busy_level; + int old_scale; +}; + +/* per disk-cgroup pair structure */ +struct ioinf_gq { + struct blkg_policy_data pd; + struct ioinf *inf; + + unsigned long flags; + /* head of the list is inf->active_infgs */ + struct list_head active; + /* head of the list is inf->lend_infgs */ + struct list_head lend; + /* head of the list is inf->borrow_infgs */ + struct list_head borrow; + + /* configured by user */ + u32 weight; + /* normalized weight */ + u32 hweight; + /* normalized inflight budget */ + u32 hinflight; + /* inuse inflight budget */ + u32 hinflight_inuse; + /* IO beyond budget will wait here */ + struct rq_wait rqw; + + struct timer_list expire_timer; + + /* max inflight in current perid */ + u32 max_inflight; + /* max inflight in last perid, will gradual reduction */ + u32 last_max_inflight; + + /* to calculate avgqu size */ + struct infg_lat_stat stat; +}; + +/* per cgroup structure, used to record default weight for all disks */ +struct ioinf_cgrp { + struct blkcg_policy_data cpd; + + /* if def_weight is 0, means it's offline */ + u32 dfl_weight; +}; + +/* scale inflight according to busy_level, from 1/10 to 10 */ +static const u8 scale_table[20] = { + [0] = 100, /* -90 */ + [1] = 90, /* -80 */ + [2] = 80, /* -70 */ + [3] = 70, /* -60 */ + [4] = 60, /* -50 */ + [5] = 50, /* -40 */ + [6] = 40, /* -30 */ + [7] = 30, /* -20 */ + [8] = 20, /* -10 */ + [9] = 10, /* 0 */ + [10] = 9, /* 10 */ + [11] = 8, /* 20 */ + [12] = 7, /* 30 */ + [13] = 6, /* 40 */ + [14] = 5, /* 50 */ + [15] = 4, /* 60 */ + [16] = 3, /* 70 */ + [17] = 2, /* 80 */ + [18] = 1, /* 90 */ +}; + +static struct blkcg_policy blkcg_policy_ioinf; + +static struct ioinf *rqos_to_inf(struct rq_qos *rqos) +{ + return container_of(rqos, struct ioinf, rqos); +} + +static struct ioinf *q_to_inf(struct request_queue *q) +{ + return rqos_to_inf(rq_qos_id(q, RQ_QOS_INFLIGHT)); +} + +static struct ioinf_gq *pd_to_infg(struct blkg_policy_data *pd) +{ + if (!pd) + return NULL; + + return container_of(pd, struct ioinf_gq, pd); +} + +static struct ioinf_gq *blkg_to_infg(struct blkcg_gq *blkg) +{ + return pd_to_infg(blkg_to_pd(blkg, &blkcg_policy_ioinf)); +} + +static struct blkcg_gq *infg_to_blkg(struct ioinf_gq *infg) +{ + return pd_to_blkg(&infg->pd); +} + +static struct ioinf_cgrp *blkcg_to_infcg(struct blkcg *blkcg) +{ + struct blkcg_policy_data *cpd = + blkcg_to_cpd(blkcg, &blkcg_policy_ioinf); + + return container_of(cpd, struct ioinf_cgrp, cpd); +} + +static struct blkcg_gq *ioinf_bio_blkg(struct bio *bio) +{ + struct blkcg_gq *blkg = bio->bi_blkg; + + if (!blkg || !blkg->online) + return NULL; + + if (blkg->blkcg->css.cgroup->level == 0) + return NULL; + + return blkg; +} + +static struct ioinf_gq *ioinf_bio_infg(struct bio *bio) +{ + struct ioinf_gq *infg; + struct blkcg_gq *blkg = ioinf_bio_blkg(bio); + + if (!blkg) + return NULL; + + infg = blkg_to_infg(blkg); + if (!infg) + return NULL; + + return infg; +} + +static u32 infg_weight(struct ioinf_gq *infg) +{ + struct ioinf_cgrp *infcg; + struct blkcg_gq *blkg; + + if (infg->weight) + return infg->weight; + + /* if user doen't set per disk weight, use the cgroup default weight */ + blkg = infg_to_blkg(infg); + infcg = blkcg_to_infcg(blkg->blkcg); + + return infcg->dfl_weight; +} + +static void infg_clear_loan(struct ioinf_gq *infg) +{ + if (!list_empty(&infg->lend)) { + clear_bit(INFG_LEND, &infg->flags); + list_del_init(&infg->lend); + } + + if (!list_empty(&infg->borrow)) { + clear_bit(INFG_BORROW, &infg->flags); + list_del_init(&infg->borrow); + } +} + +/* + * called when infg is activate or deactivate + * TODO: support cgroup hierarchy, each infg is independent for now + */ +static void __propagate_weights(struct ioinf *inf) +{ + struct ioinf_gq *infg; + u32 total = 0; + + if (list_empty(&inf->active_infgs)) + return; + + /* + * TODO: instead of clearing loan and reinitializing everything, it's + * better to keep loan and do minor incremental modification. + */ + list_for_each_entry(infg, &inf->active_infgs, active) { + total += infg_weight(infg); + infg->max_inflight = 0; + infg->last_max_inflight = 0; + infg_clear_loan(infg); + } + + list_for_each_entry(infg, &inf->active_infgs, active) { + u32 weight = infg_weight(infg); + + infg->hweight = weight * 100 / total; + infg->hinflight = infg->inf->inflight * infg->hweight / 100; + if (!infg->hinflight) + infg->hinflight = 1; + infg->hinflight_inuse = infg->hinflight; + } + + mod_timer(&inf->inf_timer, jiffies + inf->inf_timer_perid); +} + +static void propagate_weights(struct ioinf *inf) +{ + spin_lock_irq(&inf->lock); + __propagate_weights(inf); + spin_unlock_irq(&inf->lock); +} + +static void ioinf_active_infg(struct ioinf_gq *infg) +{ + struct ioinf *inf = infg->inf; + + spin_lock_irq(&inf->lock); + if (list_empty(&infg->active)) { + list_add(&infg->active, &inf->active_infgs); + __propagate_weights(inf); + } + spin_unlock_irq(&inf->lock); +} + +static unsigned int atomic_inc_below_return(atomic_t *v, unsigned int below) +{ + unsigned int cur = atomic_read(v); + + for (;;) { + unsigned int old; + + if (cur >= below) + return below + 1; + + old = atomic_cmpxchg(v, cur, cur + 1); + if (old == cur) + break; + cur = old; + } + + return cur + 1; +} + +/* + * Called from io fast path, return false means inflight IO is full, and the + * forground thread will wait inflight IO to be done. + */ +static bool ioinf_inflight_cb(struct rq_wait *rqw, void *private_data) +{ + struct ioinf_gq *infg = private_data; + unsigned int inflight; + unsigned int limit; + +retry: + limit = infg->hinflight_inuse; + inflight = atomic_inc_below_return(&infg->rqw.inflight, limit); + + if (inflight > infg->max_inflight) + infg->max_inflight = inflight; + + if (inflight <= limit) + return true; + + if (infg->hinflight_inuse == limit) { + /* + * This infg want more inflight budget, set INFG_EXHAUSTED, and + * later ioinf_timer_fn() will check, if other infg can lend + * budget. + */ + if (test_bit(INFG_EXHAUSTED, &infg->flags)) + return false; + + set_bit(INFG_EXHAUSTED, &infg->flags); + return false; + } + + /* Stop lend inflight budget to other infgs */ + infg->hinflight_inuse = infg->hinflight; + /* wake up ioinf_timer_fn() immediately to inform other infgs */ + timer_reduce(&infg->inf->inf_timer, jiffies + 1); + goto retry; +} + +void ioinf_done(struct ioinf_gq *infg) +{ + int inflight = atomic_dec_return(&infg->rqw.inflight); + + BUG_ON(inflight < 0); + + if (inflight < infg->hinflight && wq_has_sleeper(&infg->rqw.wait)) + wake_up_all(&infg->rqw.wait); + + /* deactivate infg if there is no IO for infg_expire_jiffies */ + if (inflight == 0) + mod_timer(&infg->expire_timer, + jiffies + infg->inf->infg_expire_jiffies); +} + +void ioinf_offline_done(struct ioinf *inf) +{ + int inflight = atomic_dec_return(&inf->offline_rqw.inflight); + + if (inflight < inf->offline_hinflight && + wq_has_sleeper(&inf->offline_rqw.wait)) + wake_up_all(&inf->offline_rqw.wait); +} + +static void ioinf_cleanup_cb(struct rq_wait *rqw, void *private_data) +{ + ioinf_done(private_data); +} + +static bool infg_offline(struct ioinf_gq *infg) +{ + struct ioinf_cgrp *infcg; + struct blkcg_gq *blkg; + + if (test_bit(INFG_OFFLINE, &infg->flags)) + return true; + + if (infg->weight != 0) + return false; + + /* if user doen't set per disk weight, use the cgroup default weight */ + blkg = infg_to_blkg(infg); + infcg = blkcg_to_infcg(blkg->blkcg); + + return infcg->dfl_weight == 0; +} + +static bool ioinf_offline_inflight_cb(struct rq_wait *rqw, void *private_data) +{ + struct ioinf *inf = private_data; + + return rq_wait_inc_below(rqw, inf->offline_hinflight); +} + +static void ioinf_offline_cleanup_cb(struct rq_wait *rqw, void *private_data) +{ + struct ioinf *inf = private_data; + + ioinf_offline_done(inf); +} + +static void ioinf_throttle_offline(struct ioinf *inf, struct bio *bio) +{ + rq_qos_wait(&inf->offline_rqw, inf, ioinf_offline_inflight_cb, + ioinf_offline_cleanup_cb, NULL); + + /* + * In case no online cgroup is active, daemon will adjuct all the + * budget to offline cgroup. + */ + timer_reduce(&inf->inf_timer, jiffies + inf->inf_timer_perid); +} + +static void ioinf_rqos_throttle(struct rq_qos *rqos, struct bio *bio) +{ + struct ioinf *inf = rqos_to_inf(rqos); + struct ioinf_gq *infg = ioinf_bio_infg(bio); + + if (!inf->params.enabled || !infg) + return; + + if (infg_offline(infg)) { + ioinf_throttle_offline(inf, bio); + return; + } + + if (list_empty_careful(&infg->active)) + ioinf_active_infg(infg); + + rq_qos_wait(&infg->rqw, infg, ioinf_inflight_cb, ioinf_cleanup_cb, + NULL); +} + +static void ioinf_rqos_track(struct rq_qos *rqos, struct request *rq, + struct bio *bio) +{ + struct blkcg_gq *blkg = ioinf_bio_blkg(bio); + + if (!blkg) + return; + + rq->blkg = blkg; +} + +static void ioinf_rqos_cleanup(struct rq_qos *rqos, struct bio *bio) +{ + struct ioinf_gq *infg = ioinf_bio_infg(bio); + + if (!infg || infg->inf->params.enabled || + list_empty_careful(&infg->active)) + return; + + ioinf_done(infg); +} + +static void ioinf_record_lat(struct ioinf_gq *infg, struct request *rq) +{ + struct ioinf *inf = infg->inf; + u64 lat; + + lat = rq->io_end_time_ns ? rq->io_end_time_ns : blk_time_get_ns(); + lat -= rq->alloc_time_ns; + atomic_long_add(lat, &infg->stat.latency); + + if (!inf->params.qos_enabled) + return; + + switch (req_op(rq)) { + case REQ_OP_READ: + if (lat > inf->params.rlat) + this_cpu_inc(inf->stat->rmissed); + else + this_cpu_inc(inf->stat->rmet); + break; + case REQ_OP_WRITE: + if (lat > inf->params.wlat) + this_cpu_inc(inf->stat->wmissed); + else + this_cpu_inc(inf->stat->wmet); + break; + default: + break; + } +} + +static struct ioinf_lat_stat ioinf_get_lat(struct ioinf *inf) +{ + struct ioinf_lat_stat stat = {0}; + int cpu; + + for_each_possible_cpu(cpu) { + struct ioinf_lat_stat *pstat = per_cpu_ptr(inf->stat, cpu); + + stat.rmet += pstat->rmet; + stat.rmissed += pstat->rmissed; + stat.wmet += pstat->wmet; + stat.wmissed += pstat->wmissed; + } + + return stat; +} + +static void ioinf_rqos_done(struct rq_qos *rqos, struct request *rq) +{ + struct blkcg_gq *blkg = rq->blkg; + struct ioinf_gq *infg; + + if (!blkg) + return; + + infg = blkg_to_infg(blkg); + if (infg_offline(infg)) { + ioinf_offline_done(infg->inf); + } else { + ioinf_done(infg); + ioinf_record_lat(infg, rq); + } + + rq->blkg = NULL; +} + +static void ioinf_rqos_exit(struct rq_qos *rqos) +{ + struct ioinf *inf = rqos_to_inf(rqos); + + blkcg_deactivate_policy(rqos->disk, &blkcg_policy_ioinf); + + del_timer_sync(&inf->inf_timer); + free_percpu(inf->stat); + kfree(inf); +} + +static int ioinf_stat_show(void *data, struct seq_file *m) +{ + struct rq_qos *rqos = data; + struct ioinf *inf = rqos_to_inf(rqos); + struct ioinf_gq *infg; + char path[32]; + + spin_lock_irq(&inf->lock); + + seq_printf(m, "busy_level %d inflight %u->%u\n", inf->busy_level, + inf->params.inflight, inf->inflight); + + list_for_each_entry(infg, &inf->active_infgs, active) { + blkg_path(infg_to_blkg(infg), path, sizeof(path)); + seq_printf(m, "%s: hweight %u", path, infg->hweight); + + if (test_bit(INFG_LEND, &infg->flags)) + seq_puts(m, " lend"); + if (test_bit(INFG_BORROW, &infg->flags)) + seq_puts(m, " borrow"); + if (test_bit(INFG_EXHAUSTED, &infg->flags)) + seq_puts(m, " exhausted"); + + seq_printf(m, " inflight %d/(%d->%d) %u->%u\n", + atomic_read(&infg->rqw.inflight), + infg->hinflight, infg->hinflight_inuse, + infg->last_max_inflight, + infg->max_inflight); + } + + seq_printf(m, "offline inflight %u/%d\n", + atomic_read(&inf->offline_rqw.inflight), + inf->offline_hinflight); + + spin_unlock_irq(&inf->lock); + + return 0; +} + +static int ioinf_lat_show(void *data, struct seq_file *m) +{ + struct rq_qos *rqos = data; + struct ioinf *inf = rqos_to_inf(rqos); + struct ioinf_lat_stat stat = ioinf_get_lat(inf); + struct ioinf_gq *infg; + char path[32]; + + seq_printf(m, "qos lat: %llu %llu %llu %llu\n", + stat.rmet, stat.rmissed, stat.wmet, stat.wmissed); + + spin_lock_irq(&inf->lock); + list_for_each_entry(infg, &inf->active_infgs, active) { + blkg_path(infg_to_blkg(infg), path, sizeof(path)); + seq_printf(m, "%s latency %lu\n", path, + atomic_long_read(&infg->stat.latency)); + } + spin_unlock_irq(&inf->lock); + + return 0; +} + +static const struct blk_mq_debugfs_attr ioinf_debugfs_attrs[] = { + {"stat", 0400, ioinf_stat_show}, + {"lat", 0400, ioinf_lat_show}, + {}, +}; + +static struct rq_qos_ops ioinf_rqos_ops = { + .throttle = ioinf_rqos_throttle, + .done = ioinf_rqos_done, + .track = ioinf_rqos_track, + .cleanup = ioinf_rqos_cleanup, + .exit = ioinf_rqos_exit, + +#ifdef CONFIG_BLK_DEBUG_FS + .debugfs_attrs = ioinf_debugfs_attrs, +#endif +}; + +static void infg_update_inflight(struct ioinf_gq *infg, u32 *exhausted_count) +{ + unsigned int last_max_inflight = infg->last_max_inflight; + + infg->hinflight_inuse = max(last_max_inflight, infg->max_inflight); + + infg->last_max_inflight = max(last_max_inflight >> 1, infg->max_inflight); + infg->max_inflight = infg->max_inflight >> 1; + + if (infg->hinflight_inuse < infg->hinflight) { + clear_bit(INFG_EXHAUSTED, &infg->flags); + clear_bit(INFG_BORROW, &infg->flags); + set_bit(INFG_LEND, &infg->flags); + + if (!list_empty(&infg->borrow)) + list_del_init(&infg->borrow); + if (list_empty(&infg->lend)) + list_add_tail(&infg->lend, &infg->inf->lend_infgs); + } else if (test_bit(INFG_EXHAUSTED, &infg->flags)) { + (*exhausted_count)++; + + if (list_empty(&infg->borrow)) { + set_bit(INFG_BORROW, &infg->flags); + list_add_tail(&infg->borrow, &infg->inf->borrow_infgs); + } + } +} + +static void ioinf_set_offline_inflight(struct ioinf *inf, u32 inflight) +{ + inf->offline_hinflight = inflight; + + if (wq_has_sleeper(&inf->offline_rqw.wait)) + wake_up_all(&inf->offline_rqw.wait); +} + +static bool ioinf_online_busy(struct ioinf *inf) +{ + struct ioinf_lat_stat stat; + u32 met, missed; + bool ret = false; + + if (!inf->params.qos_enabled) + return false; + + stat = ioinf_get_lat(inf); + met = stat.rmet - inf->last_stat.rmet; + missed = stat.rmissed - inf->last_stat.rmissed; + + if (met + missed >= MIN_SAMPLES && + met * 100 < (met + missed) * inf->params.rpct) { + ret = true; + goto out; + } + + met = stat.wmet - inf->last_stat.wmet; + missed = stat.wmissed - inf->last_stat.wmissed; + + if (met + missed >= MIN_SAMPLES && + met * 100 < (met + missed) * inf->params.wpct) + ret = true; + +out: + inf->last_stat = stat; + return ret; +} + +static void inf_clear_exhausted(struct ioinf *inf, u32 borrow) +{ + struct ioinf_gq *infg; + + list_for_each_entry(infg, &inf->borrow_infgs, borrow) + if (test_and_clear_bit(INFG_EXHAUSTED, &infg->flags)) + infg->hinflight_inuse += borrow; +} + +static void ioinf_adjust_busy_level(struct ioinf *inf, u32 old_busy_level) +{ + struct ioinf_gq *infg; + int scale; + + inf->busy_level = clamp(inf->busy_level, -90, 90); + + if (inf->busy_level == old_busy_level) + return; + + scale = (inf->busy_level + 90) / 10; + if (scale == inf->old_scale) + return; + + scale = scale_table[scale]; + inf->old_scale = scale; + + inf->inflight = inf->params.inflight * scale / 10; + + list_for_each_entry(infg, &inf->active_infgs, active) { + infg->hinflight = inf->inflight * infg->hweight / 100; + if (!infg->hinflight) + infg->hinflight = 1; + if (infg->hinflight_inuse < infg->hinflight) { + infg->hinflight_inuse = infg->hinflight; + if (wq_has_sleeper(&infg->rqw.wait)) + wake_up_all(&infg->rqw.wait); + } + } +} + +static void infg_clear_borrow(struct ioinf *inf, int borrow) +{ + struct ioinf_gq *infg; + int count = 0; + +retry: + list_for_each_entry(infg, &inf->borrow_infgs, borrow) { + if (infg->hinflight_inuse > infg->hinflight) { + infg->hinflight_inuse--; + count++; + } + } + + borrow -= count; + if (count && borrow > 0) + goto retry; +} + +static void ioinf_timer_fn(struct timer_list *timer) +{ + struct ioinf *inf = container_of(timer, struct ioinf, inf_timer); + int exhausted_count = 0, lend_total = 0, borrow_total = 0; + u32 old_busy_level = inf->busy_level; + bool busy = ioinf_online_busy(inf); + struct ioinf_gq *infg; + unsigned long flags; + + /* only offline cgroup is active */ + if (list_empty_careful(&inf->active_infgs)) { + ioinf_set_offline_inflight(inf, inf->inflight); + return; + } + + spin_lock_irqsave(&inf->lock, flags); + + list_for_each_entry(infg, &inf->active_infgs, active) + infg_update_inflight(infg, &exhausted_count); + + list_for_each_entry(infg, &inf->lend_infgs, lend) + lend_total += infg->hinflight - infg->hinflight_inuse; + + list_for_each_entry(infg, &inf->borrow_infgs, borrow) + borrow_total += infg->hinflight_inuse - infg->hinflight; + + if (lend_total < 0) + lend_total = 0; + if (borrow_total < 0) + borrow_total = 0; + if (lend_total >= borrow_total) { + lend_total -= borrow_total; + } else { + borrow_total -= lend_total; + lend_total = 0; + infg_clear_borrow(inf, borrow_total); + } + /* + * TODO: handle loan gracefully, equal division for now. + */ + if (exhausted_count) { + u32 borrow; + + if (inf->offline_hinflight > 1) { + ioinf_set_offline_inflight(inf, 1); + goto unlock; + } + + borrow = lend_total / exhausted_count; + if (borrow > 0) + inf_clear_exhausted(inf, borrow); + else if (busy) /* slow down */ + inf->busy_level += exhausted_count; + else if (inf->params.qos_enabled) + inf->busy_level--; + } else { + if (busy) { /* too much budget */ + if (inf->offline_hinflight > 1) + inf->offline_hinflight <<= 1; + inf->busy_level++; + } else { /* everything is fine, upgrade offline */ + if (inf->params.qos_enabled) + inf->busy_level--; + if (lend_total > inf->offline_hinflight) + ioinf_set_offline_inflight(inf, lend_total); + } + } + + ioinf_adjust_busy_level(inf, old_busy_level); + +unlock: + spin_unlock_irqrestore(&inf->lock, flags); + + if (!list_empty_careful(&inf->active_infgs)) + mod_timer(&inf->inf_timer, jiffies + inf->inf_timer_perid); +} + +static u32 ioinf_default_inflight(struct gendisk *disk) +{ + return max(disk->queue->nr_requests / 10, 30); +} + +static int blk_ioinf_init(struct gendisk *disk) +{ + struct ioinf *inf; + int ret; + + inf = kzalloc(sizeof(*inf), GFP_KERNEL); + if (!inf) + return -ENOMEM; + + inf->stat = alloc_percpu(struct ioinf_lat_stat); + if (!inf->stat) { + kfree(inf); + return -ENOMEM; + } + + spin_lock_init(&inf->lock); + inf->params.inflight = ioinf_default_inflight(disk); + inf->inflight = inf->params.inflight; + inf->infg_expire_jiffies = INFG_DFL_EXPIRE; + inf->inf_timer_perid = IOINF_TIMER_PERID; + inf->offline_hinflight = 1; + inf->old_scale = 9; + rq_wait_init(&inf->offline_rqw); + + INIT_LIST_HEAD(&inf->active_infgs); + INIT_LIST_HEAD(&inf->lend_infgs); + INIT_LIST_HEAD(&inf->borrow_infgs); + timer_setup(&inf->inf_timer, ioinf_timer_fn, 0); + + ret = rq_qos_add(&inf->rqos, disk, RQ_QOS_INFLIGHT, &ioinf_rqos_ops); + if (ret) + goto err_free_inf; + + ret = blkcg_activate_policy(disk, &blkcg_policy_ioinf); + if (ret) + goto err_del_qos; + return 0; + +err_del_qos: + rq_qos_del(&inf->rqos); +err_free_inf: + free_percpu(inf->stat); + kfree(inf); + return ret; +} + +static struct blkcg_policy_data *ioinf_cpd_alloc(gfp_t gfp) +{ + struct ioinf_cgrp *infcg = kzalloc(sizeof(*infcg), gfp); + + if (!infcg) + return NULL; + + infcg->dfl_weight = IOINF_DFL_WEIGHT; + return &infcg->cpd; +} + +static void ioinf_cpd_free(struct blkcg_policy_data *cpd) +{ + kfree(container_of(cpd, struct ioinf_cgrp, cpd)); +} + +static struct blkg_policy_data *ioinf_pd_alloc(struct gendisk *disk, + struct blkcg *blkcg, gfp_t gfp) +{ + struct ioinf_gq *infg = kzalloc_node(sizeof(*infg), gfp, disk->node_id); + + if (!infg) + return NULL; + + return &infg->pd; +} + +static void infg_expire_fn(struct timer_list *timer) +{ + struct ioinf_gq *infg = + container_of(timer, struct ioinf_gq, expire_timer); + struct ioinf *inf = infg->inf; + unsigned long flags; + + if (atomic_read(&infg->rqw.inflight) > 0) + return; + + spin_lock_irqsave(&inf->lock, flags); + if (atomic_read(&infg->rqw.inflight) == 0) { + list_del_init(&infg->active); + if (atomic_read(&infg->rqw.inflight) == 0) { + infg_clear_loan(infg); + __propagate_weights(inf); + } else { + list_add(&infg->active, &inf->active_infgs); + } + } + spin_unlock_irqrestore(&inf->lock, flags); +} + +static void ioinf_pd_init(struct blkg_policy_data *pd) +{ + struct ioinf_gq *infg = pd_to_infg(pd); + struct blkcg_gq *blkg = pd_to_blkg(pd); + + INIT_LIST_HEAD(&infg->active); + INIT_LIST_HEAD(&infg->lend); + INIT_LIST_HEAD(&infg->borrow); + infg->inf = q_to_inf(blkg->q); + rq_wait_init(&infg->rqw); + timer_setup(&infg->expire_timer, infg_expire_fn, 0); +} + +static void ioinf_pd_offline(struct blkg_policy_data *pd) +{ + struct ioinf_gq *infg = pd_to_infg(pd); + struct ioinf *inf = infg->inf; + + if (list_empty_careful(&infg->active)) + return; + + del_timer_sync(&infg->expire_timer); + + spin_lock_irq(&inf->lock); + + if (!list_empty(&infg->lend)) + list_del_init(&infg->lend); + + if (!list_empty(&infg->borrow)) + list_del_init(&infg->borrow); + + if (!list_empty(&infg->active)) { + list_del_init(&infg->active); + __propagate_weights(inf); + } + + spin_unlock_irq(&inf->lock); +} + +static void ioinf_pd_free(struct blkg_policy_data *pd) +{ + struct ioinf_gq *infg = pd_to_infg(pd); + + kfree(infg); +} + +static u64 ioinf_weight_prfill(struct seq_file *sf, struct blkg_policy_data *pd, + int off) +{ + const char *dname = blkg_dev_name(pd->blkg); + struct ioinf_gq *infg = pd_to_infg(pd); + + if (dname && infg->weight) + seq_printf(sf, "%s %u\n", dname, infg->weight); + + return 0; +} + +static int ioinf_weight_show(struct seq_file *sf, void *v) +{ + struct blkcg *blkcg = css_to_blkcg(seq_css(sf)); + struct ioinf_cgrp *infcg = blkcg_to_infcg(blkcg); + + seq_printf(sf, "default %u\n", infcg->dfl_weight); + blkcg_print_blkgs(sf, blkcg, ioinf_weight_prfill, &blkcg_policy_ioinf, + seq_cft(sf)->private, false); + + return 0; +} + +static ssize_t ioinf_weight_write(struct kernfs_open_file *of, char *buf, + size_t nbytes, loff_t off) +{ + struct blkcg *blkcg = css_to_blkcg(of_css(of)); + struct ioinf_cgrp *infcg = blkcg_to_infcg(blkcg); + struct blkg_conf_ctx ctx; + struct ioinf_gq *infg; + int ret; + u32 v; + + if (!strchr(buf, ':')) { + if (!sscanf(buf, "default %u", &v) && !sscanf(buf, "%u", &v)) + return -EINVAL; + if (v < 0) + return -EINVAL; + + infcg->dfl_weight = v; + + return nbytes; + } + + blkg_conf_init(&ctx, buf); + ret = blkg_conf_prep(blkcg, &blkcg_policy_ioinf, &ctx); + if (ret) + return ret; + + infg = blkg_to_infg(ctx.blkg); + if (!strncmp(ctx.body, "default", 7)) { + v = IOINF_DFL_WEIGHT; + } else if (!sscanf(ctx.body, "%u", &v) || + v < 0 || v > CGROUP_WEIGHT_MAX) { + blkg_conf_exit(&ctx); + return -EINVAL; + } + + infg->weight = v; + if (v == 0) + set_bit(INFG_OFFLINE, &infg->flags); + else + clear_bit(INFG_OFFLINE, &infg->flags); + blkg_conf_exit(&ctx); + propagate_weights(infg->inf); + return nbytes; +} + +static u64 ioinf_qos_prfill(struct seq_file *sf, struct blkg_policy_data *pd, + int off) +{ + const char *dname = blkg_dev_name(pd->blkg); + struct ioinf *inf = q_to_inf(pd->blkg->q); + struct ioinf_params params; + + if (!dname) + return 0; + + params = inf->params; + seq_printf(sf, "%s enable=%d inflight=%u qos_enable=%d", dname, + params.enabled, params.inflight, params.qos_enabled); + + if (inf->params.qos_enabled) + seq_printf(sf, " rlat=%llu rpct=%u wlat=%llu wpct=%u", + params.rlat, params.rpct, params.wlat, params.wpct); + + seq_putc(sf, '\n'); + return 0; +} + +static int ioinf_qos_show(struct seq_file *sf, void *v) +{ + struct blkcg *blkcg = css_to_blkcg(seq_css(sf)); + + blkcg_print_blkgs(sf, blkcg, ioinf_qos_prfill, + &blkcg_policy_ioinf, seq_cft(sf)->private, false); + return 0; +} + +static const match_table_t qos_ctrl_tokens = { + { INF_ENABLE, "enable=%u" }, + { INF_INFLIGHT, "inflight=%u" }, + { QOS_ENABLE, "qos_enable=%u" }, + { QOS_RLAT, "rlat=%u" }, + { QOS_WLAT, "wlat=%u" }, + { QOS_RPCT, "rpct=%u" }, + { QOS_WPCT, "wpct=%u" }, + { NR_QOS_CTRL_PARAMS, NULL }, +}; + +static ssize_t ioinf_qos_write(struct kernfs_open_file *of, char *input, + size_t nbytes, loff_t off) +{ + struct blkg_conf_ctx ctx; + struct gendisk *disk; + struct ioinf *inf; + struct ioinf_params params; + char *body, *p; + int ret; + + blkg_conf_init(&ctx, input); + + ret = blkg_conf_open_bdev(&ctx); + if (ret) + goto err; + + body = ctx.body; + disk = ctx.bdev->bd_disk; + if (!queue_is_mq(disk->queue)) { + ret = -EOPNOTSUPP; + goto err; + } + + inf = q_to_inf(disk->queue); + if (!inf) { + ret = blk_ioinf_init(disk); + if (ret) + goto err; + + inf = q_to_inf(disk->queue); + } + + params = inf->params; + + while ((p = strsep(&body, " \t\n"))) { + substring_t args[MAX_OPT_ARGS]; + s64 v; + + if (!*p) + continue; + + switch (match_token(p, qos_ctrl_tokens, args)) { + case INF_ENABLE: + if (match_u64(&args[0], &v)) + goto einval; + params.enabled = !!v; + continue; + case QOS_ENABLE: + if (match_u64(&args[0], &v)) + goto einval; + params.qos_enabled = !!v; + continue; + case INF_INFLIGHT: + if (match_u64(&args[0], &v)) + goto einval; + params.inflight = v; + continue; + case QOS_RLAT: + if (match_u64(&args[0], &v) || v == 0) + goto einval; + params.rlat = v; + continue; + case QOS_WLAT: + if (match_u64(&args[0], &v) || v == 0) + goto einval; + params.wlat = v; + continue; + case QOS_RPCT: + if (match_u64(&args[0], &v) || v == 0 || v >= 100) + goto einval; + params.rpct = v; + continue; + case QOS_WPCT: + if (match_u64(&args[0], &v) || v == 0 || v >= 100) + goto einval; + params.wpct = v; + continue; + default: + goto einval; + } + } + + if (params.qos_enabled && + (params.rlat == 0 || params.wlat == 0 || + params.rpct == 0 || params.rpct >= 100 || + params.wpct == 0 || params.wpct >= 100)) + goto einval; + + if (params.inflight == 0) + params.inflight = ioinf_default_inflight(disk); + + if (params.qos_enabled && !inf->params.qos_enabled) { + blk_stat_enable_accounting(disk->queue); + blk_queue_flag_set(QUEUE_FLAG_RQ_ALLOC_TIME, disk->queue); + } else if (!params.qos_enabled && inf->params.qos_enabled) { + blk_stat_disable_accounting(disk->queue); + blk_queue_flag_clear(QUEUE_FLAG_RQ_ALLOC_TIME, disk->queue); + } + + inf->params = params; + inf->busy_level = 0; + inf->old_scale = 9; + inf->inflight = params.inflight; + propagate_weights(inf); + + blkg_conf_exit(&ctx); + return nbytes; + +einval: + ret = -EINVAL; +err: + blkg_conf_exit(&ctx); + return ret; +} + +static struct cftype ioinf_files[] = { + { + .name = "inf.weight", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = ioinf_weight_show, + .write = ioinf_weight_write, + }, + { + .name = "inf.qos", + .flags = CFTYPE_ONLY_ON_ROOT, + .seq_show = ioinf_qos_show, + .write = ioinf_qos_write, + }, + {} +}; + +static struct cftype ioinf_legacy_files[] = { + { + .name = "inf.weight", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = ioinf_weight_show, + .write = ioinf_weight_write, + }, + { + .name = "inf.qos", + .flags = CFTYPE_ONLY_ON_ROOT, + .seq_show = ioinf_qos_show, + .write = ioinf_qos_write, + }, + {} +}; + +static struct blkcg_policy blkcg_policy_ioinf = { + .dfl_cftypes = ioinf_files, + .legacy_cftypes = ioinf_legacy_files, + + .cpd_alloc_fn = ioinf_cpd_alloc, + .cpd_free_fn = ioinf_cpd_free, + + .pd_alloc_fn = ioinf_pd_alloc, + .pd_init_fn = ioinf_pd_init, + .pd_offline_fn = ioinf_pd_offline, + .pd_free_fn = ioinf_pd_free, +}; + +static int __init ioinf_init(void) +{ + return blkcg_policy_register(&blkcg_policy_ioinf); +} + +static void __exit ioinf_exit(void) +{ + blkcg_policy_unregister(&blkcg_policy_ioinf); +} + +module_init(ioinf_init); +module_exit(ioinf_exit); diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index efe99cfae51d..b5af47bf99d4 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -798,6 +798,8 @@ static const char *rq_qos_id_to_name(enum rq_qos_id id) return "latency"; case RQ_QOS_COST: return "cost"; + case RQ_QOS_INFLIGHT: + return "inflight"; } return "unknown"; } diff --git a/block/blk-rq-qos.h b/block/blk-rq-qos.h index 93d1ba692973..d504a302ca0f 100644 --- a/block/blk-rq-qos.h +++ b/block/blk-rq-qos.h @@ -17,6 +17,7 @@ enum rq_qos_id { RQ_QOS_WBT, RQ_QOS_LATENCY, RQ_QOS_COST, + RQ_QOS_INFLIGHT, }; struct rq_wait { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 4c4416fd2df7..81a733e1bef9 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -115,6 +115,9 @@ struct request { #ifdef CONFIG_BLK_WBT unsigned short wbt_flags; +#endif +#ifdef CONFIG_BLK_CGROUP_IOINFLIGHT + struct blkcg_gq *blkg; #endif /* * rq sectors used for blk stats. It has the same value -- 2.39.2

2 1

[PATCH OLK-6.6 0/2] CVE-2025-22105
by Wang Liang 18 Jul '25

18 Jul '25

Hangbin Liu (1): bonding: return detailed error when loading native XDP fails Wang Liang (1): bonding: check xdp prog when set bond mode drivers/net/bonding/bond_main.c | 11 +++++++---- drivers/net/bonding/bond_options.c | 3 +++ include/net/bonding.h | 1 + 3 files changed, 11 insertions(+), 4 deletions(-) -- 2.34.1

2 3

[PATCH OLK-6.6] net_sched: sch_sfq: fix a potential crash on gso_skb handling
by Wang Liang 18 Jul '25

18 Jul '25

From: Eric Dumazet <edumazet(a)google.com> stable inclusion from stable-v6.6.94 commit b4e9bab6011b9559b7c157b16b91ae46d4d8c533 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/ICJTA4 CVE: CVE-2025-38115 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 82ffbe7776d0ac084031f114167712269bf3d832 ] SFQ has an assumption of always being able to queue at least one packet. However, after the blamed commit, sch->q.len can be inflated by packets in sch->gso_skb, and an enqueue() on an empty SFQ qdisc can be followed by an immediate drop. Fix sfq_drop() to properly clear q->tail in this situation. Tested: ip netns add lb ip link add dev to-lb type veth peer name in-lb netns lb ethtool -K to-lb tso off # force qdisc to requeue gso_skb ip netns exec lb ethtool -K in-lb gro on # enable NAPI ip link set dev to-lb up ip -netns lb link set dev in-lb up ip addr add dev to-lb 192.168.20.1/24 ip -netns lb addr add dev in-lb 192.168.20.2/24 tc qdisc replace dev to-lb root sfq limit 100 ip netns exec lb netserver netperf -H 192.168.20.2 -l 100 & netperf -H 192.168.20.2 -l 100 & netperf -H 192.168.20.2 -l 100 & netperf -H 192.168.20.2 -l 100 & Fixes: a53851e2c321 ("net: sched: explicit locking in gso_cpu fallback") Reported-by: Marcus Wichelmann <marcus.wichelmann(a)hetzner-cloud.de> Closes: https://lore.kernel.org/netdev/9da42688-bfaa-4364-8797-e9271f3bdaef@hetzner… Signed-off-by: Eric Dumazet <edumazet(a)google.com> Reviewed-by: Toke Høiland-Jørgensen <toke(a)redhat.com> Link: https://patch.msgid.link/20250606165127.3629486-1-edumazet@google.com Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Wang Liang <wangliang74(a)huawei.com> --- net/sched/sch_sfq.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c index 60754f366ab7..2f51def0d5d9 100644 --- a/net/sched/sch_sfq.c +++ b/net/sched/sch_sfq.c @@ -310,7 +310,10 @@ static unsigned int sfq_drop(struct Qdisc *sch, struct sk_buff **to_free) /* It is difficult to believe, but ALL THE SLOTS HAVE LENGTH 1. */ x = q->tail->next; slot = &q->slots[x]; - q->tail->next = slot->next; + if (slot->next == x) + q->tail = NULL; /* no more active slots */ + else + q->tail->next = slot->next; q->ht[slot->hash] = SFQ_EMPTY_SLOT; goto drop; } -- 2.34.1

2 1

[PATCH OLK-6.6] netfilter: nf_set_pipapo_avx2: fix initial map fill
by Wang Liang 18 Jul '25

18 Jul '25

From: Florian Westphal <fw(a)strlen.de> stable inclusion from stable-v6.6.94 commit 39bab2d3517b5b50c609b4f8c66129bf619fffa0 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/ICJT9V CVE: CVE-2025-38120 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit ea77c397bff8b6d59f6d83dae1425b08f465e8b5 ] If the first field doesn't cover the entire start map, then we must zero out the remainder, else we leak those bits into the next match round map. The early fix was incomplete and did only fix up the generic C implementation. A followup patch adds a test case to nft_concat_range.sh. Fixes: 791a615b7ad2 ("netfilter: nf_set_pipapo: fix initial map fill") Signed-off-by: Florian Westphal <fw(a)strlen.de> Reviewed-by: Stefano Brivio <sbrivio(a)redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo(a)netfilter.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Wang Liang <wangliang74(a)huawei.com> --- net/netfilter/nft_set_pipapo_avx2.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/net/netfilter/nft_set_pipapo_avx2.c b/net/netfilter/nft_set_pipapo_avx2.c index b8d3c3213efe..da535b1834dc 100644 --- a/net/netfilter/nft_set_pipapo_avx2.c +++ b/net/netfilter/nft_set_pipapo_avx2.c @@ -1112,6 +1112,25 @@ bool nft_pipapo_avx2_estimate(const struct nft_set_desc *desc, u32 features, return true; } +/** + * pipapo_resmap_init_avx2() - Initialise result map before first use + * @m: Matching data, including mapping table + * @res_map: Result map + * + * Like pipapo_resmap_init() but do not set start map bits covered by the first field. + */ +static inline void pipapo_resmap_init_avx2(const struct nft_pipapo_match *m, unsigned long *res_map) +{ + const struct nft_pipapo_field *f = m->f; + int i; + + /* Starting map doesn't need to be set to all-ones for this implementation, + * but we do need to zero the remaining bits, if any. + */ + for (i = f->bsize; i < m->bsize_max; i++) + res_map[i] = 0ul; +} + /** * nft_pipapo_avx2_lookup() - Lookup function for AVX2 implementation * @net: Network namespace @@ -1170,7 +1189,7 @@ bool nft_pipapo_avx2_lookup(const struct net *net, const struct nft_set *set, res = scratch->map + (map_index ? m->bsize_max : 0); fill = scratch->map + (map_index ? 0 : m->bsize_max); - /* Starting map doesn't need to be set for this implementation */ + pipapo_resmap_init_avx2(m, res); nft_pipapo_avx2_prepare(); -- 2.34.1

2 1

[PATCH OLK-6.6] net: fix udp gso skb_segment after pull from frag_list
by Wang Liang 18 Jul '25

18 Jul '25

From: Shiming Cheng <shiming.cheng(a)mediatek.com> stable inclusion from stable-v6.6.94 commit 85eef1748c024da1a191aed56b30a3a65958c50c category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/ICJTGK CVE: CVE-2025-38124 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 3382a1ed7f778db841063f5d7e317ac55f9e7f72 ] Commit a1e40ac5b5e9 ("net: gso: fix udp gso fraglist segmentation after pull from frag_list") detected invalid geometry in frag_list skbs and redirects them from skb_segment_list to more robust skb_segment. But some packets with modified geometry can also hit bugs in that code. We don't know how many such cases exist. Addressing each one by one also requires touching the complex skb_segment code, which risks introducing bugs for other types of skbs. Instead, linearize all these packets that fail the basic invariants on gso fraglist skbs. That is more robust. If only part of the fraglist payload is pulled into head_skb, it will always cause exception when splitting skbs by skb_segment. For detailed call stack information, see below. Valid SKB_GSO_FRAGLIST skbs - consist of two or more segments - the head_skb holds the protocol headers plus first gso_size - one or more frag_list skbs hold exactly one segment - all but the last must be gso_size Optional datapath hooks such as NAT and BPF (bpf_skb_pull_data) can modify fraglist skbs, breaking these invariants. In extreme cases they pull one part of data into skb linear. For UDP, this causes three payloads with lengths of (11,11,10) bytes were pulled tail to become (12,10,10) bytes. The skbs no longer meets the above SKB_GSO_FRAGLIST conditions because payload was pulled into head_skb, it needs to be linearized before pass to regular skb_segment. skb_segment+0xcd0/0xd14 __udp_gso_segment+0x334/0x5f4 udp4_ufo_fragment+0x118/0x15c inet_gso_segment+0x164/0x338 skb_mac_gso_segment+0xc4/0x13c __skb_gso_segment+0xc4/0x124 validate_xmit_skb+0x9c/0x2c0 validate_xmit_skb_list+0x4c/0x80 sch_direct_xmit+0x70/0x404 __dev_queue_xmit+0x64c/0xe5c neigh_resolve_output+0x178/0x1c4 ip_finish_output2+0x37c/0x47c __ip_finish_output+0x194/0x240 ip_finish_output+0x20/0xf4 ip_output+0x100/0x1a0 NF_HOOK+0xc4/0x16c ip_forward+0x314/0x32c ip_rcv+0x90/0x118 __netif_receive_skb+0x74/0x124 process_backlog+0xe8/0x1a4 __napi_poll+0x5c/0x1f8 net_rx_action+0x154/0x314 handle_softirqs+0x154/0x4b8 [118.376811] [C201134] rxq0_pus: [name:bug&]kernel BUG at net/core/skbuff.c:4278! [118.376829] [C201134] rxq0_pus: [name:traps&]Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP [118.470774] [C201134] rxq0_pus: [name:mrdump&]Kernel Offset: 0x178cc00000 from 0xffffffc008000000 [118.470810] [C201134] rxq0_pus: [name:mrdump&]PHYS_OFFSET: 0x40000000 [118.470827] [C201134] rxq0_pus: [name:mrdump&]pstate: 60400005 (nZCv daif +PAN -UAO) [118.470848] [C201134] rxq0_pus: [name:mrdump&]pc : [0xffffffd79598aefc] skb_segment+0xcd0/0xd14 [118.470900] [C201134] rxq0_pus: [name:mrdump&]lr : [0xffffffd79598a5e8] skb_segment+0x3bc/0xd14 [118.470928] [C201134] rxq0_pus: [name:mrdump&]sp : ffffffc008013770 Fixes: a1e40ac5b5e9 ("gso: fix udp gso fraglist segmentation after pull from frag_list") Signed-off-by: Shiming Cheng <shiming.cheng(a)mediatek.com> Reviewed-by: Willem de Bruijn <willemb(a)google.com> Signed-off-by: David S. Miller <davem(a)davemloft.net> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Wang Liang <wangliang74(a)huawei.com> --- net/ipv4/udp_offload.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 2ab16139c197..653b190f55e3 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -273,6 +273,7 @@ struct sk_buff *__udp_gso_segment(struct sk_buff *gso_skb, bool copy_dtor; __sum16 check; __be16 newlen; + int ret = 0; mss = skb_shinfo(gso_skb)->gso_size; if (gso_skb->len <= sizeof(*uh) + mss) @@ -295,6 +296,10 @@ struct sk_buff *__udp_gso_segment(struct sk_buff *gso_skb, if (skb_pagelen(gso_skb) - sizeof(*uh) == skb_shinfo(gso_skb)->gso_size) return __udp_gso_segment_list(gso_skb, features, is_ipv6); + ret = __skb_linearize(gso_skb); + if (ret) + return ERR_PTR(ret); + /* Setup csum, as fraglist skips this in udp4_gro_receive. */ gso_skb->csum_start = skb_transport_header(gso_skb) - gso_skb->head; gso_skb->csum_offset = offsetof(struct udphdr, check); -- 2.34.1

2 1