January 2024 - Kernel - mailweb.openeuler.org

[PATCH v4 OLK-6.6 00/21] kabi reserve patches
by Lin Yujun 31 Jan '24

31 Jan '24

*** BLURB HERE *** Chen Jiahao (1): KABI: reserve space for struct input_dev Jialin Zhang (1): kabi: reserve space for power management related structure Lin Ruizhe (5): msi: Add kabi_reserve in msi.h interrupt: Add kabi_reserve in interrupt.h irq: Add kabi_reserve in irq irq_desc: Add kabi_reserve in irq_desc irqdomain: Add kabi_reserve in irqdomain Lin Yujun (13): drm: drm_atomic.h: Add kabi_reserve drm: drm_client.h: Add kabi_reserve drm: drm_connector.h: Add kabi_reserve drm: drm_crtc.h: Add kabi_reserve drm: drm_device.h: Add kabi_reserve drm: drm_drv.h: Add kabi_reserve drm: drm_fb_helper.h: Add kabi_reserve drm: drm_file.h: Add kabi_reserve drm: drm_fourcc.h: Add kabi_reserve drm: drm_gem.h: Add kabi_reserve pm: pm_domain.h: Add kabi_reserve pm: pm.h: Add kabi_reserve energy_model: Add kabi_reserve Yang Yingliang (1): kabi: reserve space for struct cpu_stop_work include/drm/drm_atomic.h | 6 ++++++ include/drm/drm_client.h | 7 +++++++ include/drm/drm_connector.h | 18 ++++++++++++++++++ include/drm/drm_crtc.h | 7 +++++++ include/drm/drm_device.h | 4 ++++ include/drm/drm_drv.h | 5 +++++ include/drm/drm_fb_helper.h | 8 ++++++++ include/drm/drm_file.h | 8 ++++++++ include/drm/drm_fourcc.h | 5 +++++ include/drm/drm_gem.h | 8 ++++++++ include/linux/energy_model.h | 5 +++++ include/linux/input.h | 4 ++++ include/linux/interrupt.h | 6 ++++++ include/linux/irq.h | 8 ++++++++ include/linux/irqdesc.h | 4 +++- include/linux/irqdomain.h | 3 +++ include/linux/msi.h | 7 +++++++ include/linux/pm.h | 14 ++++++++++++++ include/linux/pm_domain.h | 10 ++++++++++ include/linux/pm_qos.h | 7 +++++++ include/linux/pm_wakeup.h | 4 ++++ include/linux/stop_machine.h | 3 +++ 22 files changed, 150 insertions(+), 1 deletion(-) -- 2.34.1

1 21

[PATCH openEuler-1.0-LTS] mm/filemap: avoid buffered read/write race to read inconsistent data
by Baokun Li 31 Jan '24

31 Jan '24

mainline inclusion from mainline-v6.7-rc8 commit e2c27b803bb664748e090d99042ac128b3f88d92 category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I8RRAT Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The following concurrency may cause the data read to be inconsistent with the data on disk: cpu1 cpu2 ------------------------------|------------------------------ // Buffered write 2048 from 0 ext4_buffered_write_iter generic_perform_write copy_page_from_iter_atomic ext4_da_write_end ext4_da_do_write_end block_write_end __block_commit_write folio_mark_uptodate // Buffered read 4096 from 0 smp_wmb() ext4_file_read_iter set_bit(PG_uptodate, folio_flags) generic_file_read_iter i_size_write // 2048 filemap_read unlock_page(page) filemap_get_pages filemap_get_read_batch folio_test_uptodate(folio) ret = test_bit(PG_uptodate, folio_flags) if (ret) smp_rmb(); // Ensure that the data in page 0-2048 is up-to-date. // New buffered write 2048 from 2048 ext4_buffered_write_iter generic_perform_write copy_page_from_iter_atomic ext4_da_write_end ext4_da_do_write_end block_write_end __block_commit_write folio_mark_uptodate smp_wmb() set_bit(PG_uptodate, folio_flags) i_size_write // 4096 unlock_page(page) isize = i_size_read(inode) // 4096 // Read the latest isize 4096, but without smp_rmb(), there may be // Load-Load disorder resulting in the data in the 2048-4096 range // in the page is not up-to-date. copy_page_to_iter // copyout 4096 In the concurrency above, we read the updated i_size, but there is no read barrier to ensure that the data in the page is the same as the i_size at this point, so we may copy the unsynchronized page out. Hence adding the missing read memory barrier to fix this. This is a Load-Load reordering issue, which only occurs on some weak mem-ordering architectures (e.g. ARM64, ALPHA), but not on strong mem-ordering architectures (e.g. X86). And theoretically the problem doesn't only happen on ext4, filesystems that call filemap_read() but don't hold inode lock (e.g. btrfs, f2fs, ubifs ...) will have this problem, while filesystems with inode lock (e.g. xfs, nfs) won't have this problem. Link: https://lkml.kernel.org/r/20231213062324.739009-1-libaokun1@huawei.com Signed-off-by: Baokun Li <libaokun1(a)huawei.com> Reviewed-by: Jan Kara <jack(a)suse.cz> Cc: Andreas Dilger <adilger.kernel(a)dilger.ca> Cc: Christoph Hellwig <hch(a)infradead.org> Cc: Dave Chinner <david(a)fromorbit.com> Cc: Matthew Wilcox (Oracle) <willy(a)infradead.org> Cc: Ritesh Harjani (IBM) <ritesh.list(a)gmail.com> Cc: Theodore Ts'o <tytso(a)mit.edu> Cc: yangerkun <yangerkun(a)huawei.com> Cc: Yu Kuai <yukuai3(a)huawei.com> Cc: Zhang Yi <yi.zhang(a)huawei.com> Cc: <stable(a)vger.kernel.org> Signed-off-by: Andrew Morton <akpm(a)linux-foundation.org> Conflicts: mm/filemap.c Signed-off-by: Baokun Li <libaokun1(a)huawei.com> --- mm/filemap.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 5bf4645256cb..ecf8bed4c76f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2273,6 +2273,15 @@ static ssize_t generic_file_buffered_read(struct kiocb *iocb, } nr = nr - offset; + /* + * Pairs with a barrier in + * block_write_end()->mark_buffer_dirty() or other page + * dirtying routines like iomap_write_end() to ensure + * changes to page contents are visible before we see + * increased inode size. + */ + smp_rmb(); + /* If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. -- 2.31.1

1 0

[PATCH OLK-6.6] net/kabi: reserve space for net related structures
by Wang Yufen 31 Jan '24

31 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I903T2 -------------------------------- Signed-off-by: Wang Yufen <wangyufen(a)huawei.com> --- include/linux/inetdevice.h | 2 ++ include/net/if_inet6.h | 8 ++++++++ include/net/netns/sctp.h | 2 ++ include/net/sock_reuseport.h | 2 ++ include/rdma/ib_verbs.h | 12 ++++++++++++ 5 files changed, 26 insertions(+) diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h index 5181993..aa816f5 100644 --- a/include/linux/inetdevice.h +++ b/include/linux/inetdevice.h @@ -163,6 +163,8 @@ struct in_ifaddr { __u32 ifa_preferred_lft; unsigned long ifa_cstamp; /* created timestamp */ unsigned long ifa_tstamp; /* updated timestamp */ + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct in_validator_info { diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h index c849072..1199310 100644 --- a/include/net/if_inet6.h +++ b/include/net/if_inet6.h @@ -111,6 +111,8 @@ struct ip6_sf_list { unsigned char sf_oldin; /* change state */ unsigned char sf_crcount; /* retrans. left to send */ struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define MAF_TIMER_RUNNING 0x01 @@ -135,6 +137,10 @@ struct ifmcaddr6 { unsigned long mca_cstamp; unsigned long mca_tstamp; struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) + KABI_RESERVE(3) + KABI_RESERVE(4) }; /* Anycast stuff */ @@ -223,6 +229,8 @@ struct inet6_dev { struct rcu_head rcu; unsigned int ra_mtu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void ipv6_eth_mc_map(const struct in6_addr *addr, char *buf) diff --git a/include/net/netns/sctp.h b/include/net/netns/sctp.h index 7eff3d9..b812c1b 100644 --- a/include/net/netns/sctp.h +++ b/include/net/netns/sctp.h @@ -179,6 +179,8 @@ struct netns_sctp { #ifdef CONFIG_NET_L3_MASTER_DEV int l3mdev_accept; #endif + KABI_RESERVE(1) + KABI_RESERVE(2) }; #endif /* __NETNS_SCTP_H__ */ diff --git a/include/net/sock_reuseport.h b/include/net/sock_reuseport.h index 6ec140b..9458cc5 100644 --- a/include/net/sock_reuseport.h +++ b/include/net/sock_reuseport.h @@ -26,6 +26,8 @@ struct sock_reuseport { unsigned int bind_inany:1; unsigned int has_conns:1; struct bpf_prog __rcu *prog; /* optional BPF sock selector */ + KABI_RESERVE(1) + KABI_RESERVE(2) struct sock *socks[]; /* array of sock pointers */ }; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 62f9d12..3e006a7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -450,6 +450,8 @@ struct ib_device_attr { u64 max_dm_size; /* Max entries for sgl for optimized performance per READ */ u32 max_sgl_rd; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_mtu { @@ -608,6 +610,8 @@ struct rdma_hw_stats { unsigned long *is_disabled; int num_counters; u64 value[]; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define RDMA_HW_STATS_DEFAULT_LIFESPAN 10 @@ -1642,6 +1646,8 @@ struct ib_srq { * Implementation details of the RDMA core, don't use in drivers: */ struct rdma_restrack_entry res; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_raw_packet_caps { @@ -2217,6 +2223,8 @@ struct ib_port_data { struct hlist_node ndev_hash_link; struct rdma_port_counter port_counter; struct ib_port *sysfs; + KABI_RESERVE(1) + KABI_RESERVE(2) }; /* rdma netdev type - specifies protocol type */ @@ -2684,6 +2692,8 @@ struct ib_device_ops { DECLARE_RDMA_OBJ_SIZE(ib_srq); DECLARE_RDMA_OBJ_SIZE(ib_ucontext); DECLARE_RDMA_OBJ_SIZE(ib_xrcd); + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct ib_core_device { @@ -2784,6 +2794,8 @@ struct ib_device { char iw_ifname[IFNAMSIZ]; u32 iw_driver_flags; u32 lag_flags; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void *rdma_zalloc_obj(struct ib_device *dev, size_t size, -- 1.8.3.1

1 0

[PATCH OLK-6.6] net/kabi: reserve space for net related structures
by Wang Yufen 31 Jan '24

31 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I903T2 -------------------------------- Signed-off-by: Wang Yufen <wangyufen(a)huawei.com> --- include/linux/inetdevice.h | 4 ++++ include/net/if_inet6.h | 8 ++++++++ include/net/netns/sctp.h | 2 ++ include/net/sock_reuseport.h | 2 ++ include/rdma/ib_verbs.h | 12 ++++++++++++ 5 files changed, 28 insertions(+) diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h index ddb27fc..19b3677 100644 --- a/include/linux/inetdevice.h +++ b/include/linux/inetdevice.h @@ -50,6 +50,8 @@ struct in_device { struct neigh_parms *arp_parms; struct ipv4_devconf cnf; struct rcu_head rcu_head; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define IPV4_DEVCONF(cnf, attr) ((cnf).data[IPV4_DEVCONF_ ## attr - 1]) @@ -159,6 +161,8 @@ struct in_ifaddr { __u32 ifa_preferred_lft; unsigned long ifa_cstamp; /* created timestamp */ unsigned long ifa_tstamp; /* updated timestamp */ + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct in_validator_info { diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h index c849072..1199310 100644 --- a/include/net/if_inet6.h +++ b/include/net/if_inet6.h @@ -111,6 +111,8 @@ struct ip6_sf_list { unsigned char sf_oldin; /* change state */ unsigned char sf_crcount; /* retrans. left to send */ struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define MAF_TIMER_RUNNING 0x01 @@ -135,6 +137,10 @@ struct ifmcaddr6 { unsigned long mca_cstamp; unsigned long mca_tstamp; struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) + KABI_RESERVE(3) + KABI_RESERVE(4) }; /* Anycast stuff */ @@ -223,6 +229,8 @@ struct inet6_dev { struct rcu_head rcu; unsigned int ra_mtu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void ipv6_eth_mc_map(const struct in6_addr *addr, char *buf) diff --git a/include/net/netns/sctp.h b/include/net/netns/sctp.h index 7eff3d9..b812c1b 100644 --- a/include/net/netns/sctp.h +++ b/include/net/netns/sctp.h @@ -179,6 +179,8 @@ struct netns_sctp { #ifdef CONFIG_NET_L3_MASTER_DEV int l3mdev_accept; #endif + KABI_RESERVE(1) + KABI_RESERVE(2) }; #endif /* __NETNS_SCTP_H__ */ diff --git a/include/net/sock_reuseport.h b/include/net/sock_reuseport.h index 6ec140b..9458cc5 100644 --- a/include/net/sock_reuseport.h +++ b/include/net/sock_reuseport.h @@ -26,6 +26,8 @@ struct sock_reuseport { unsigned int bind_inany:1; unsigned int has_conns:1; struct bpf_prog __rcu *prog; /* optional BPF sock selector */ + KABI_RESERVE(1) + KABI_RESERVE(2) struct sock *socks[]; /* array of sock pointers */ }; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 62f9d12..3e006a7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -450,6 +450,8 @@ struct ib_device_attr { u64 max_dm_size; /* Max entries for sgl for optimized performance per READ */ u32 max_sgl_rd; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_mtu { @@ -608,6 +610,8 @@ struct rdma_hw_stats { unsigned long *is_disabled; int num_counters; u64 value[]; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define RDMA_HW_STATS_DEFAULT_LIFESPAN 10 @@ -1642,6 +1646,8 @@ struct ib_srq { * Implementation details of the RDMA core, don't use in drivers: */ struct rdma_restrack_entry res; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_raw_packet_caps { @@ -2217,6 +2223,8 @@ struct ib_port_data { struct hlist_node ndev_hash_link; struct rdma_port_counter port_counter; struct ib_port *sysfs; + KABI_RESERVE(1) + KABI_RESERVE(2) }; /* rdma netdev type - specifies protocol type */ @@ -2684,6 +2692,8 @@ struct ib_device_ops { DECLARE_RDMA_OBJ_SIZE(ib_srq); DECLARE_RDMA_OBJ_SIZE(ib_ucontext); DECLARE_RDMA_OBJ_SIZE(ib_xrcd); + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct ib_core_device { @@ -2784,6 +2794,8 @@ struct ib_device { char iw_ifname[IFNAMSIZ]; u32 iw_driver_flags; u32 lag_flags; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void *rdma_zalloc_obj(struct ib_device *dev, size_t size, -- 1.8.3.1

1 0

[PATCH OLK-6.6] vfs: reserve kabi space for vfs related structures
by Zhihao Cheng 31 Jan '24

31 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I900QB CVE: NA -------------------------------- struture size reserves address_space_operations 160 4 address_space 200 7 inode 624 2 file_lock_operations 16 2 lock_manager_operations 24 1 super_block 1392 2 Signed-off-by: Zhihao Cheng <chengzhihao1(a)huawei.com> --- include/linux/filelock.h | 5 +++++ include/linux/fs.h | 19 +++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/include/linux/filelock.h b/include/linux/filelock.h index 95e868e09e29..92795ed425d9 100644 --- a/include/linux/filelock.h +++ b/include/linux/filelock.h @@ -31,6 +31,9 @@ struct file_lock; struct file_lock_operations { void (*fl_copy_lock)(struct file_lock *, struct file_lock *); void (*fl_release_private)(struct file_lock *); + + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct lock_manager_operations { @@ -45,6 +48,8 @@ struct lock_manager_operations { bool (*lm_breaker_owns_lease)(struct file_lock *); bool (*lm_lock_expirable)(struct file_lock *cfl); void (*lm_expire_lock)(void); + + KABI_RESERVE(1) }; struct lock_manager { diff --git a/include/linux/fs.h b/include/linux/fs.h index cc34619424cf..daf7221fbdc3 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -451,6 +451,11 @@ struct address_space_operations { sector_t *span); void (*swap_deactivate)(struct file *file); int (*swap_rw)(struct kiocb *iocb, struct iov_iter *iter); + + KABI_RESERVE(1) + KABI_RESERVE(2) + KABI_RESERVE(3) + KABI_RESERVE(4) }; extern const struct address_space_operations empty_aops; @@ -497,6 +502,14 @@ struct address_space { spinlock_t private_lock; struct list_head private_list; void *private_data; + + KABI_RESERVE(1) + KABI_RESERVE(2) + KABI_RESERVE(3) + KABI_RESERVE(4) + KABI_RESERVE(5) + KABI_RESERVE(6) + KABI_RESERVE(7) } __attribute__((aligned(sizeof(long)))) __randomize_layout; /* * On most architectures that alignment is already the case; but @@ -756,6 +769,9 @@ struct inode { #endif void *i_private; /* fs or device private pointer */ + + KABI_RESERVE(1) + KABI_RESERVE(2) } __randomize_layout; struct timespec64 timestamp_truncate(struct timespec64 t, struct inode *inode); @@ -1328,6 +1344,9 @@ struct super_block { spinlock_t s_inode_wblist_lock; struct list_head s_inodes_wb; /* writeback inodes */ + + KABI_RESERVE(1) + KABI_RESERVE(2) } __randomize_layout; static inline struct user_namespace *i_user_ns(const struct inode *inode) -- 2.31.1

1 0

[PATCH OLK-6.6 0/1] misc:fix compiling warning
by Chen Ridong 31 Jan '24

31 Jan '24

*** BLURB HERE *** Chen Ridong (1): misc: fix compiling waring kernel/cgroup/misc.c | 5 +++++ 1 file changed, 5 insertions(+) -- 2.34.1

1 0

[openeuler:OLK-5.10 5032/30000] arch/x86/include/asm/paravirt.h:204:9: sparse: sparse: cast truncates bits from constant value (10000000f becomes f)
by kernel test robot 31 Jan '24

31 Jan '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: 7b3b2702f031a1a8368468aef621f12b23241185 commit: 824e3bf903140e215a2d4c8165ba0e3ced22548e [5032/30000] x86/perf: Add uncore performance monitor support for Zhaoxin CPUs :::::: branch date: 11 hours ago :::::: commit date: 2 years, 6 months ago config: x86_64-randconfig-122-20240130 (attached as .config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (attached as reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202401310228.wuZdscKx-lkp@intel.com/ sparse warnings: (new ones prefixed by >>) arch/x86/events/zhaoxin/uncore.c: note: in included file (through arch/x86/include/asm/msr.h, arch/x86/include/asm/processor.h, arch/x86/include/asm/cpu_device_id.h): >> arch/x86/include/asm/paravirt.h:204:9: sparse: sparse: cast truncates bits from constant value (10000000f becomes f) vim +204 arch/x86/include/asm/paravirt.h d3561b7fa0fb0fc include/asm-i386/paravirt.h Rusty Russell 2006-12-07 201 47edb65178cb705 arch/x86/include/asm/paravirt.h Andy Lutomirski 2015-07-23 202 static inline void wrmsrl(unsigned msr, u64 val) 47edb65178cb705 arch/x86/include/asm/paravirt.h Andy Lutomirski 2015-07-23 203 { 47edb65178cb705 arch/x86/include/asm/paravirt.h Andy Lutomirski 2015-07-23 @204 wrmsr(msr, (u32)val, (u32)(val>>32)); 47edb65178cb705 arch/x86/include/asm/paravirt.h Andy Lutomirski 2015-07-23 205 } 47edb65178cb705 arch/x86/include/asm/paravirt.h Andy Lutomirski 2015-07-23 206 :::::: The code at line 204 was first introduced by commit :::::: 47edb65178cb7056c2eea0b6c41a7d8c84547192 x86/asm/msr: Make wrmsrl() a function :::::: TO: Andy Lutomirski <luto(a)kernel.org> :::::: CC: Ingo Molnar <mingo(a)kernel.org> -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[openeuler:OLK-5.10 1515/30000] htmldocs: Warning: crypto/asymmetric_keys/pgp_preload.c references a file that doesn't exist: Documentation/security/keys-crypto.txt
by kernel test robot 31 Jan '24

31 Jan '24

tree: https://gitee.com/openeuler/kernel.git OLK-5.10 head: 7b3b2702f031a1a8368468aef621f12b23241185 commit: c283657144c1475e571f0c1f34477af1f682a12f [1515/30000] KEYS: Provide a function to load keys from a PGP keyring blob :::::: branch date: 16 hours ago :::::: commit date: 2 years, 10 months ago reproduce: (attached as reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp(a)intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202401310757.wqsgZIwq-lkp@intel.com/ All warnings (new ones prefixed by >>): >> Warning: crypto/asymmetric_keys/pgp_preload.c references a file that doesn't exist: Documentation/security/keys-crypto.txt -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 0

[PATCH OLK-6.6] net/kabi: reserve space for net related structures
by Wang Yufen 31 Jan '24

31 Jan '24

hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I903T2 -------------------------------- Signed-off-by: Wang Yufen <wangyufen(a)huawei.com> --- include/linux/inetdevice.h | 4 ++++ include/net/if_inet6.h | 8 ++++++++ include/net/netns/sctp.h | 2 ++ include/net/sock_reuseport.h | 2 ++ include/rdma/ib_verbs.h | 12 ++++++++++++ 5 files changed, 28 insertions(+) diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h index ddb27fc..19b3677 100644 --- a/include/linux/inetdevice.h +++ b/include/linux/inetdevice.h @@ -50,6 +50,8 @@ struct in_device { struct neigh_parms *arp_parms; struct ipv4_devconf cnf; struct rcu_head rcu_head; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define IPV4_DEVCONF(cnf, attr) ((cnf).data[IPV4_DEVCONF_ ## attr - 1]) @@ -159,6 +161,8 @@ struct in_ifaddr { __u32 ifa_preferred_lft; unsigned long ifa_cstamp; /* created timestamp */ unsigned long ifa_tstamp; /* updated timestamp */ + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct in_validator_info { diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h index c849072..1199310 100644 --- a/include/net/if_inet6.h +++ b/include/net/if_inet6.h @@ -111,6 +111,8 @@ struct ip6_sf_list { unsigned char sf_oldin; /* change state */ unsigned char sf_crcount; /* retrans. left to send */ struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define MAF_TIMER_RUNNING 0x01 @@ -135,6 +137,10 @@ struct ifmcaddr6 { unsigned long mca_cstamp; unsigned long mca_tstamp; struct rcu_head rcu; + KABI_RESERVE(1) + KABI_RESERVE(2) + KABI_RESERVE(3) + KABI_RESERVE(4) }; /* Anycast stuff */ @@ -223,6 +229,8 @@ struct inet6_dev { struct rcu_head rcu; unsigned int ra_mtu; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void ipv6_eth_mc_map(const struct in6_addr *addr, char *buf) diff --git a/include/net/netns/sctp.h b/include/net/netns/sctp.h index 7eff3d9..b812c1b 100644 --- a/include/net/netns/sctp.h +++ b/include/net/netns/sctp.h @@ -179,6 +179,8 @@ struct netns_sctp { #ifdef CONFIG_NET_L3_MASTER_DEV int l3mdev_accept; #endif + KABI_RESERVE(1) + KABI_RESERVE(2) }; #endif /* __NETNS_SCTP_H__ */ diff --git a/include/net/sock_reuseport.h b/include/net/sock_reuseport.h index 6ec140b..9458cc5 100644 --- a/include/net/sock_reuseport.h +++ b/include/net/sock_reuseport.h @@ -26,6 +26,8 @@ struct sock_reuseport { unsigned int bind_inany:1; unsigned int has_conns:1; struct bpf_prog __rcu *prog; /* optional BPF sock selector */ + KABI_RESERVE(1) + KABI_RESERVE(2) struct sock *socks[]; /* array of sock pointers */ }; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 62f9d12..3e006a7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -450,6 +450,8 @@ struct ib_device_attr { u64 max_dm_size; /* Max entries for sgl for optimized performance per READ */ u32 max_sgl_rd; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_mtu { @@ -608,6 +610,8 @@ struct rdma_hw_stats { unsigned long *is_disabled; int num_counters; u64 value[]; + KABI_RESERVE(1) + KABI_RESERVE(2) }; #define RDMA_HW_STATS_DEFAULT_LIFESPAN 10 @@ -1642,6 +1646,8 @@ struct ib_srq { * Implementation details of the RDMA core, don't use in drivers: */ struct rdma_restrack_entry res; + KABI_RESERVE(1) + KABI_RESERVE(2) }; enum ib_raw_packet_caps { @@ -2217,6 +2223,8 @@ struct ib_port_data { struct hlist_node ndev_hash_link; struct rdma_port_counter port_counter; struct ib_port *sysfs; + KABI_RESERVE(1) + KABI_RESERVE(2) }; /* rdma netdev type - specifies protocol type */ @@ -2684,6 +2692,8 @@ struct ib_device_ops { DECLARE_RDMA_OBJ_SIZE(ib_srq); DECLARE_RDMA_OBJ_SIZE(ib_ucontext); DECLARE_RDMA_OBJ_SIZE(ib_xrcd); + KABI_RESERVE(1) + KABI_RESERVE(2) }; struct ib_core_device { @@ -2784,6 +2794,8 @@ struct ib_device { char iw_ifname[IFNAMSIZ]; u32 iw_driver_flags; u32 lag_flags; + KABI_RESERVE(1) + KABI_RESERVE(2) }; static inline void *rdma_zalloc_obj(struct ib_device *dev, size_t size, -- 1.8.3.1

1 0

[PATCH OLK-6.6] ubifs: Queue up space reservation tasks if retrying many times
by Zhihao Cheng 31 Jan '24

31 Jan '24

hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I903EX CVE: NA -------------------------------- Recently we catched ENOSPC returned by make_reservation() while doing fsstress on UBIFS, we got following information when it occurred (See details in Link): UBIFS error (ubi0:0 pid 3640152): make_reservation [ubifs]: cannot reserve 112 bytes in jhead 2, error -28 CPU: 2 PID: 3640152 Comm: kworker/u16:2 Tainted: G B W Hardware name: Hisilicon PhosphorHi1230 EMU (DT) Workqueue: writeback wb_workfn (flush-ubifs_0_0) Call trace: dump_stack+0x114/0x198 make_reservation+0x564/0x610 [ubifs] ubifs_jnl_write_data+0x328/0x48c [ubifs] do_writepage+0x2a8/0x3e4 [ubifs] ubifs_writepage+0x16c/0x374 [ubifs] generic_writepages+0xb4/0x114 do_writepages+0xcc/0x11c writeback_sb_inodes+0x2d0/0x564 wb_writeback+0x20c/0x2b4 wb_workfn+0x404/0x510 process_one_work+0x304/0x4ac worker_thread+0x31c/0x4e4 kthread+0x23c/0x290 Budgeting info: data budget sum 17576, total budget sum 17768 budg_data_growth 4144, budg_dd_growth 13432, budg_idx_growth 192 min_idx_lebs 13, old_idx_sz 988640, uncommitted_idx 0 page_budget 4144, inode_budget 160, dent_budget 312 nospace 0, nospace_rp 0 dark_wm 8192, dead_wm 4096, max_idx_node_sz 192 freeable_cnt 0, calc_idx_sz 988640, idx_gc_cnt 0 dirty_pg_cnt 4, dirty_zn_cnt 0, clean_zn_cnt 4811 gc_lnum 21, ihead_lnum 14 jhead 0 (GC) LEB 16 jhead 1 (base) LEB 34 jhead 2 (data) LEB 23 bud LEB 16 bud LEB 23 bud LEB 34 old bud LEB 33 old bud LEB 31 old bud LEB 15 commit state 4 Budgeting predictions: available: 33832, outstanding 17576, free 15356 (pid 3640152) start dumping LEB properties (pid 3640152) Lprops statistics: empty_lebs 3, idx_lebs 11 taken_empty_lebs 1, total_free 1253376, total_dirty 2445736 total_used 3438712, total_dark 65536, total_dead 17248 LEB 15 free 0 dirty 248000 used 5952 (taken) LEB 16 free 110592 dirty 896 used 142464 (taken, jhead 0 (GC)) LEB 21 free 253952 dirty 0 used 0 (taken, GC LEB) LEB 23 free 0 dirty 248104 used 5848 (taken, jhead 2 (data)) LEB 29 free 253952 dirty 0 used 0 (empty) LEB 33 free 0 dirty 253952 used 0 (taken) LEB 34 free 217088 dirty 36544 used 320 (taken, jhead 1 (base)) LEB 37 free 253952 dirty 0 used 0 (empty) OTHERS: index lebs, zero-available non-index lebs According to the budget algorithm, there are 5 LEBs reserved for budget: three journal heads(16,23,34), 1 GC LEB(21) and 1 deletion LEB(can be used in make_reservation()). There are 2 empty LEBs used for index nodes, which is calculated as min_idx_lebs - idx_lebs = 2. In theory, LEB 15 and 33 should be reclaimed as free state after committing, but it is now in taken state. After looking the realization of reserve_space(), there's a possible situation: LEB 15: free 2000 dirty 248000 used 3952 (jhead 2) LEB 23: free 2000 dirty 248104 used 3848 (bud, taken) LEB 33: free 2000 dirty 251952 used 0 (bud, taken) wb_workfn wb_workfn_2 do_writepage // write 3000 bytes ubifs_jnl_write_data make_reservation reserve_space ubifs_garbage_collect ubifs_find_dirty_leb // ret ENOSPC, dirty LEBs are taken nospc_retries++ // 1 ubifs_run_commit do_commit LEB 15: free 2000 dirty 248000 used 3952 (jhead 2) LEB 23: free 2000 dirty 248104 used 3848 (dirty) LEB 33: free 2000 dirty 251952 used 0 (dirty) do_writepage // write 2000 bytes for 3 times ubifs_jnl_write_data // grabs 15\23\33 LEB 15: free 0 dirty 248000 used 5952 (bud, taken) LEB 23: free 0 dirty 248104 used 5848 (jhead 2) LEB 33: free 0 dirty 253952 used 0 (bud, taken) reserve_space ubifs_garbage_collect ubifs_find_dirty_leb // ret ENOSPC, dirty LEBs are taken if (nospc_retries++ < 2) // false ubifs_ro_mode ! Fetch a reproducer in Link. The dirty LEBs could be grabbed by other threads, which fails finding dirty LEBs of GC in current thread, so make_reservation() could try many times to invoke GC&&committing, but current realization limits the times of retrying as 'nospc_retries'(twice). Fix it by adding a wait queue, start queuing up space reservation tasks when someone task has retried gc + commit for many times. Then there is only one task making space reservation at any time, and it can always make success under the premise of correct budgeting. Link: https://bugzilla.kernel.org/show_bug.cgi?id=218164 Fixes: 1e51764a3c2a ("UBIFS: add new flash file system") Signed-off-by: Zhihao Cheng <chengzhihao1(a)huawei.com> Signed-off-by: Zhang Yi <yi.zhang(a)huawei.com> --- fs/ubifs/journal.c | 171 +++++++++++++++++++++++++++++++++++++++------ fs/ubifs/super.c | 2 + fs/ubifs/ubifs.h | 4 ++ 3 files changed, 155 insertions(+), 22 deletions(-) diff --git a/fs/ubifs/journal.c b/fs/ubifs/journal.c index ffc9beee7be6..877c238f7ccf 100644 --- a/fs/ubifs/journal.c +++ b/fs/ubifs/journal.c @@ -292,6 +292,96 @@ static int write_head(struct ubifs_info *c, int jhead, void *buf, int len, return err; } +/** + * __queue_and_wait - queue a task and wait until the task is waked up. + * @c: UBIFS file-system description object + * + * This function adds current task in queue and waits until the task is waked + * up. This function should be called with @c->reserve_space_wq locked. + */ +static void __queue_and_wait(struct ubifs_info *c) +{ + DEFINE_WAIT(wait); + + __add_wait_queue_entry_tail_exclusive(&c->reserve_space_wq, &wait); + set_current_state(TASK_UNINTERRUPTIBLE); + spin_unlock(&c->reserve_space_wq.lock); + + schedule(); + finish_wait(&c->reserve_space_wq, &wait); +} + +/** + * wait_for_reservation - try queuing current task to wait until waked up. + * @c: UBIFS file-system description object + * + * This function queues current task to wait until waked up, if queuing is + * started(@c->need_wait_space is not %0). Returns %true if current task is + * added in queue, otherwise %false is returned. + */ +static bool wait_for_reservation(struct ubifs_info *c) +{ + if (likely(atomic_read(&c->need_wait_space) == 0)) + /* Quick path to check whether queuing is started. */ + return false; + + spin_lock(&c->reserve_space_wq.lock); + if (atomic_read(&c->need_wait_space) == 0) { + /* Queuing is not started, don't queue current task. */ + spin_unlock(&c->reserve_space_wq.lock); + return false; + } + + __queue_and_wait(c); + return true; +} + +/** + * wake_up_reservation - wake up first task in queue or stop queuing. + * @c: UBIFS file-system description object + * + * This function wakes up the first task in queue if it exists, or stops + * queuing if no tasks in queue. + */ +static void wake_up_reservation(struct ubifs_info *c) +{ + spin_lock(&c->reserve_space_wq.lock); + if (waitqueue_active(&c->reserve_space_wq)) + wake_up_locked(&c->reserve_space_wq); + else + /* + * Compared with wait_for_reservation(), set @c->need_wait_space + * under the protection of wait queue lock, which can avoid that + * @c->need_wait_space is set to 0 after new task queued. + */ + atomic_set(&c->need_wait_space, 0); + spin_unlock(&c->reserve_space_wq.lock); +} + +/** + * add_or_start_queue - add current task in queue or start queuing. + * @c: UBIFS file-system description object + * + * This function starts queuing if queuing is not started, otherwise adds + * current task in queue. + */ +static void add_or_start_queue(struct ubifs_info *c) +{ + spin_lock(&c->reserve_space_wq.lock); + if (atomic_cmpxchg(&c->need_wait_space, 0, 1) == 0) { + /* Starts queuing, task can go on directly. */ + spin_unlock(&c->reserve_space_wq.lock); + return; + } + + /* + * There are at least two tasks have retried more than 32 times + * at certain point, first task has started queuing, just queue + * the left tasks. + */ + __queue_and_wait(c); +} + /** * make_reservation - reserve journal space. * @c: UBIFS file-system description object @@ -311,33 +401,27 @@ static int write_head(struct ubifs_info *c, int jhead, void *buf, int len, static int make_reservation(struct ubifs_info *c, int jhead, int len) { int err, cmt_retries = 0, nospc_retries = 0; + bool blocked = wait_for_reservation(c); again: down_read(&c->commit_sem); err = reserve_space(c, jhead, len); - if (!err) + if (!err) { /* c->commit_sem will get released via finish_reservation(). */ - return 0; + goto out_wake_up; + } up_read(&c->commit_sem); if (err == -ENOSPC) { /* * GC could not make any progress. We should try to commit - * once because it could make some dirty space and GC would - * make progress, so make the error -EAGAIN so that the below + * because it could make some dirty space and GC would make + * progress, so make the error -EAGAIN so that the below * will commit and re-try. */ - if (nospc_retries++ < 2) { - dbg_jnl("no space, retry"); - err = -EAGAIN; - } - - /* - * This means that the budgeting is incorrect. We always have - * to be able to write to the media, because all operations are - * budgeted. Deletions are not budgeted, though, but we reserve - * an extra LEB for them. - */ + nospc_retries++; + dbg_jnl("no space, retry"); + err = -EAGAIN; } if (err != -EAGAIN) @@ -349,15 +433,37 @@ static int make_reservation(struct ubifs_info *c, int jhead, int len) */ if (cmt_retries > 128) { /* - * This should not happen unless the journal size limitations - * are too tough. + * This should not happen unless: + * 1. The journal size limitations are too tough. + * 2. The budgeting is incorrect. We always have to be able to + * write to the media, because all operations are budgeted. + * Deletions are not budgeted, though, but we reserve an + * extra LEB for them. */ - ubifs_err(c, "stuck in space allocation"); + ubifs_err(c, "stuck in space allocation, nospc_retries %d", + nospc_retries); err = -ENOSPC; goto out; - } else if (cmt_retries > 32) - ubifs_warn(c, "too many space allocation re-tries (%d)", - cmt_retries); + } else if (cmt_retries > 32) { + /* + * It's almost impossible to happen, unless there are many tasks + * making reservation concurrently and someone task has retried + * gc + commit for many times, generated available space during + * this period are grabbed by other tasks. + * But if it happens, start queuing up all tasks that will make + * space reservation, then there is only one task making space + * reservation at any time, and it can always make success under + * the premise of correct budgeting. + */ + ubifs_warn(c, "too many space allocation cmt_retries (%d) " + "nospc_retries (%d), start queuing tasks", + cmt_retries, nospc_retries); + + if (!blocked) { + blocked = true; + add_or_start_queue(c); + } + } dbg_jnl("-EAGAIN, commit and retry (retried %d times)", cmt_retries); @@ -365,7 +471,7 @@ static int make_reservation(struct ubifs_info *c, int jhead, int len) err = ubifs_run_commit(c); if (err) - return err; + goto out_wake_up; goto again; out: @@ -380,6 +486,27 @@ static int make_reservation(struct ubifs_info *c, int jhead, int len) cmt_retries = dbg_check_lprops(c); up_write(&c->commit_sem); } +out_wake_up: + if (blocked) { + /* + * Only tasks that have ever started queuing or ever been queued + * can wake up other queued tasks, which can make sure that + * there is only one task waked up to make space reservation. + * For example: + * task A task B task C + * make_reservation make_reservation + * reserve_space // 0 + * wake_up_reservation + * atomic_cmpxchg // 0, start queuing + * reserve_space + * wait_for_reservation + * __queue_and_wait + * add_wait_queue + * if (blocked) // false + * // So that task C won't be waked up to race with task B + */ + wake_up_reservation(c); + } return err; } diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c index b08fb28d16b5..c28779557c43 100644 --- a/fs/ubifs/super.c +++ b/fs/ubifs/super.c @@ -2152,6 +2152,8 @@ static struct ubifs_info *alloc_ubifs_info(struct ubi_volume_desc *ubi) mutex_init(&c->bu_mutex); mutex_init(&c->write_reserve_mutex); init_waitqueue_head(&c->cmt_wq); + init_waitqueue_head(&c->reserve_space_wq); + atomic_set(&c->need_wait_space, 0); c->buds = RB_ROOT; c->old_idx = RB_ROOT; c->size_tree = RB_ROOT; diff --git a/fs/ubifs/ubifs.h b/fs/ubifs/ubifs.h index ebb3ad6b5e7e..e763efe9c7fd 100644 --- a/fs/ubifs/ubifs.h +++ b/fs/ubifs/ubifs.h @@ -1047,6 +1047,8 @@ struct ubifs_debug_info; * @bg_bud_bytes: number of bud bytes when background commit is initiated * @old_buds: buds to be released after commit ends * @max_bud_cnt: maximum number of buds + * @need_wait_space: Non %0 means space reservation tasks need to wait in queue + * @reserve_space_wq: wait queue to sleep on if @need_wait_space is not %0 * * @commit_sem: synchronizes committer with other processes * @cmt_state: commit state @@ -1305,6 +1307,8 @@ struct ubifs_info { long long bg_bud_bytes; struct list_head old_buds; int max_bud_cnt; + atomic_t need_wait_space; + wait_queue_head_t reserve_space_wq; struct rw_semaphore commit_sem; int cmt_state; -- 2.31.1

2 1

2024

2023

2022

2021

2020

2019

Kernel January 2024