[PATCH OLK-6.6 0/4] CVE-2026-98058
Kumar Kartikeya Dwivedi (2): bpf: Fix sleepable context for async callbacks bpf: Mark syscall helpers as sleepable Pu Lehui (2): bpf: Track sleepable state in verifier state for async callbacks bpf: Fix kabi breakage in struct bpf_verifier_state include/linux/bpf_verifier.h | 1 + kernel/bpf/syscall.c | 2 ++ kernel/bpf/verifier.c | 36 ++++++++++++++++++++++++++++++------ 3 files changed, 33 insertions(+), 6 deletions(-) -- 2.34.1
hulk inclusion category: cleanup bugzilla: https://atomgit.com/src-openeuler/kernel/issues/19783 -------------------------------- Async callbacks (e.g. bpf_timer) require dedicated sleepable tracking in verifier state rather than solely relying on env->prog->sleepable. Extract the verifier state tracking logic from upstream commit 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl") without pulling in the full bpf_wq infrastructure: - Add `in_sleepable` to `struct bpf_verifier_state`. - Propagate and compare `in_sleepable` in copy_verifier_state() and states_equal(). - Pass `is_sleepable` parameter to push_async_cb() (false for bpf_timer). - Update in_sleepable() to check env->cur_state->in_sleepable. This serves as a prerequisite for commit 469d638d1520 ("bpf: Fix sleepable context for async callbacks"). Signed-off-by: Pu Lehui <pulehui@huawei.com> --- include/linux/bpf_verifier.h | 1 + kernel/bpf/verifier.c | 12 +++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 52e1bdfe65a0..d5aa1c4cd87b 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -442,6 +442,7 @@ struct bpf_verifier_state { * while they are still in use. */ bool used_as_loop_entry; + bool in_sleepable; /* first and last insn idx of this verifier state */ u32 first_insn_idx; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7f3bb5bf8a57..805f67dc4c81 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1830,6 +1830,7 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state, } dst_state->speculative = src->speculative; dst_state->active_rcu_lock = src->active_rcu_lock; + dst_state->in_sleepable = src->in_sleepable; dst_state->curframe = src->curframe; dst_state->active_lock.ptr = src->active_lock.ptr; dst_state->active_lock.id = src->active_lock.id; @@ -2659,7 +2660,7 @@ static void init_func_state(struct bpf_verifier_env *env, /* Similar to push_stack(), but for async callbacks */ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env, int insn_idx, int prev_insn_idx, - int subprog) + int subprog, bool is_sleepable) { struct bpf_verifier_stack_elem *elem; struct bpf_func_state *frame; @@ -2686,6 +2687,7 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env, * Initialize it similar to do_check_common(). */ elem->st.branches = 1; + elem->st.in_sleepable = is_sleepable; frame = kzalloc(sizeof(*frame), GFP_KERNEL); if (!frame) goto err; @@ -5482,7 +5484,8 @@ static int map_kptr_match_type(struct bpf_verifier_env *env, static bool in_sleepable(struct bpf_verifier_env *env) { - return env->prog->sleepable; + return env->prog->sleepable || + (env->cur_state && env->cur_state->in_sleepable); } /* The non-sleepable programs and sleepable programs with explicit bpf_rcu_read_lock() @@ -9550,7 +9553,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins /* there is no real recursion here. timer callbacks are async */ env->subprog_info[subprog].is_async_cb = true; async_cb = push_async_cb(env, env->subprog_info[subprog].start, - insn_idx, subprog); + insn_idx, subprog, false); if (!async_cb) return -EFAULT; callee = async_cb->frame[0]; @@ -16897,6 +16900,9 @@ static bool states_equal(struct bpf_verifier_env *env, if (old->active_rcu_lock != cur->active_rcu_lock) return false; + if (old->in_sleepable != cur->in_sleepable) + return false; + /* for states to be equal callsites have to be the same * and all frame states need to be equivalent */ -- 2.34.1
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://atomgit.com/openeuler/kernel/merge_requests/30255 邮件列表地址：https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/2SG... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://atomgit.com/openeuler/kernel/merge_requests/30255 Mailing list address: https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/2SG...
From: Kumar Kartikeya Dwivedi <memxor@gmail.com> mainline inclusion from mainline-v6.19-rc1 commit 469d638d1520a9332cd0d034690e75e845610a51 category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/19783 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Fix the BPF verifier to correctly determine the sleepable context of async callbacks based on the async primitive type rather than the arming program's context. The bug is in in_sleepable() which uses OR logic to check if the current execution context is sleepable. When a sleepable program arms a timer callback, the callback's state correctly has in_sleepable=false, but in_sleepable() would still return true due to env->prog->sleepable being true. This incorrectly allows sleepable helpers like bpf_copy_from_user() inside timer callbacks when armed from sleepable programs, even though timer callbacks always execute in non-sleepable context. Fix in_sleepable() to rely solely on env->cur_state->in_sleepable, and initialize state->in_sleepable to env->prog->sleepable in do_check_common() for the main program entry. This ensures the sleepable context is properly tracked per verification state rather than being overridden by the program's sleepability. The env->cur_state NULL check in in_sleepable() was only needed for do_misc_fixups() which runs after verification when env->cur_state is set to NULL. Update do_misc_fixups() to use env->prog->sleepable directly for the storage_get_function check, and remove the redundant NULL check from in_sleepable(). Introduce is_async_cb_sleepable() helper to explicitly determine async callback sleepability based on the primitive type: - bpf_timer callbacks are never sleepable - bpf_wq and bpf_task_work callbacks are always sleepable Add verifier_bug() check to catch unhandled async callback types, ensuring future additions cannot be silently mishandled. Move the is_task_work_add_kfunc() forward declaration to the top alongside other callback-related helpers. We update push_async_cb() to adjust to the new changes. At the same time, while simplifying in_sleepable(), we notice a problem in do_misc_fixups. Fix storage_get helpers to use GFP_ATOMIC when called from non-sleepable contexts within sleepable programs, such as bpf_timer callbacks. Currently, the check in do_misc_fixups assumes that env->prog->sleepable, previously in_sleepable(env) which only resolved to this check before last commit, holds across the program's execution, but that is not true. Instead, the func_atomic bit must be set whenever we see the function being called in an atomic context. Previously, this is being done when the helper is invoked in atomic contexts in sleepable programs, we can simply just set the value to true without doing an in_sleepable() check. We must also do a standalone in_sleepable() check to handle cases where the async callback itself is armed from a sleepable program, but is itself non-sleepable (e.g., timer callback) and invokes such a helper, thus needing the func_atomic bit to be true for the said call. Adjust do_misc_fixups() to drop any checks regarding sleepable nature of the program, and just depend on the func_atomic bit to decide which GFP flag to pass. Fixes: 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl") Fixes: b00fa38a9c1c ("bpf: Enable non-atomic allocations in local storage") Acked-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20251007220349.3852807-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Conflicts: kernel/bpf/verifier.c [OLK-6.6 only have bpf_timer case, and not have bpf_wq and task_work.] Signed-off-by: Pu Lehui <pulehui@huawei.com> --- kernel/bpf/verifier.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 805f67dc4c81..5040796b6a14 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -609,6 +609,17 @@ static bool is_sync_callback_calling_insn(struct bpf_insn *insn) (bpf_pseudo_kfunc_call(insn) && is_sync_callback_calling_kfunc(insn->imm)); } +static bool is_async_cb_sleepable(struct bpf_verifier_env *env, + struct bpf_insn *insn) +{ + /* bpf_timer callbacks are never sleepable. */ + if (bpf_helper_call(insn) && insn->imm == BPF_FUNC_timer_set_callback) + return false; + + verbose(env, "verifier bug: unhandled async callback type\n"); + return false; +} + static bool is_storage_get_function(enum bpf_func_id func_id) { return func_id == BPF_FUNC_sk_storage_get || @@ -5484,8 +5495,7 @@ static int map_kptr_match_type(struct bpf_verifier_env *env, static bool in_sleepable(struct bpf_verifier_env *env) { - return env->prog->sleepable || - (env->cur_state && env->cur_state->in_sleepable); + return env->cur_state->in_sleepable; } /* The non-sleepable programs and sleepable programs with explicit bpf_rcu_read_lock() @@ -9553,7 +9563,8 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins /* there is no real recursion here. timer callbacks are async */ env->subprog_info[subprog].is_async_cb = true; async_cb = push_async_cb(env, env->subprog_info[subprog].start, - insn_idx, subprog, false); + insn_idx, subprog, + is_async_cb_sleepable(env, insn)); if (!async_cb) return -EFAULT; callee = async_cb->frame[0]; @@ -10337,10 +10348,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn return -EINVAL; } - if (in_sleepable(env) && is_storage_get_function(func_id)) + if (is_storage_get_function(func_id)) env->insn_aux_data[insn_idx].storage_get_func_atomic = true; } + /* + * Non-sleepable contexts in sleepable programs (e.g., timer callbacks) + * are atomic and must use GFP_ATOMIC for storage_get helpers. + */ + if (!in_sleepable(env) && is_storage_get_function(func_id)) + env->insn_aux_data[insn_idx].storage_get_func_atomic = true; + meta.func_id = func_id; /* check args */ for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) { @@ -19866,8 +19884,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) } if (is_storage_get_function(insn->imm)) { - if (!in_sleepable(env) || - env->insn_aux_data[i + delta].storage_get_func_atomic) + if (env->insn_aux_data[i + delta].storage_get_func_atomic) insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC); else insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL); @@ -20309,6 +20326,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) state->curframe = 0; state->speculative = false; state->branches = 1; + state->in_sleepable = env->prog->sleepable; state->frame[0] = kzalloc(sizeof(struct bpf_func_state), GFP_KERNEL); if (!state->frame[0]) { kfree(state); -- 2.34.1
From: Kumar Kartikeya Dwivedi <memxor@gmail.com> mainline inclusion from mainline-v7.3-rc2 commit d05524794240b52fdc3b6c1220dd05505715824d category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/19783 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- bpf_sys_bpf() executes the bpf(2) syscall body, which can take mutexes, allocate with GFP_KERNEL, and wait for an RCU grace period. bpf_sys_close() reaches close_fd() and filp_close(), which can sleep as well. Both helpers are limited to BPF_PROG_TYPE_SYSCALL, whose main program is sleepable. That does not make every callback sleepable: a syscall program can register a bpf_timer callback, and the verifier checks that callback in a non-sleepable context while retaining the syscall helper set. Without .might_sleep on the prototypes, such a callback can invoke bpf_sys_bpf() from hrtimer softirq context and trigger a scheduling-while-atomic failure. bpf_sys_close() is exposed through the same missing context check. Set .might_sleep on both prototypes so the existing helper-context check rejects them from timer callbacks and other atomic regions. Calls from the sleepable main body remain valid. Fixes: 79a7f8bdb159 ("bpf: Introduce bpf_sys_bpf() helper and program type.") Fixes: 3abea089246f ("bpf: Add bpf_sys_close() helper.") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260903144433.1716731-10-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Pu Lehui <pulehui@huawei.com> --- kernel/bpf/syscall.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index b462be488724..117f1628e9b7 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -5814,6 +5814,7 @@ EXPORT_SYMBOL(kern_sys_bpf); static const struct bpf_func_proto bpf_sys_bpf_proto = { .func = bpf_sys_bpf, .gpl_only = false, + .might_sleep = true, .ret_type = RET_INTEGER, .arg1_type = ARG_ANYTHING, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, @@ -5839,6 +5840,7 @@ BPF_CALL_1(bpf_sys_close, u32, fd) static const struct bpf_func_proto bpf_sys_close_proto = { .func = bpf_sys_close, .gpl_only = false, + .might_sleep = true, .ret_type = RET_INTEGER, .arg1_type = ARG_ANYTHING, }; -- 2.34.1
hulk inclusion category: bugfix bugzilla: https://atomgit.com/src-openeuler/kernel/issues/19783 -------------------------------- Fix kabi breakage in struct bpf_verifier_state. Add KABI_FILL_HOLE for in_sleepable, as there has 1 byte hole for it. struct bpf_verifier_state { struct bpf_func_state * frame[8]; /* 0 64 */ /* --- cacheline 1 boundary (64 bytes) --- */ struct bpf_verifier_state * parent; /* 64 8 */ u32 branches; /* 72 4 */ u32 insn_idx; /* 76 4 */ u32 curframe; /* 80 4 */ /* XXX 4 bytes hole, try to pack */ struct bpf_active_lock active_lock; /* 88 16 */ /* XXX last struct has 4 bytes of padding */ bool speculative; /* 104 1 */ bool active_rcu_lock; /* 105 1 */ bool used_as_loop_entry; /* 106 1 */ /* XXX 1 byte hole, try to pack */ u32 first_insn_idx; /* 108 4 */ u32 last_insn_idx; /* 112 4 */ Fixes: 0379a79e1a42 ("bpf: Track sleepable state in verifier state for async callbacks") Signed-off-by: Pu Lehui <pulehui@huawei.com> --- include/linux/bpf_verifier.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index d5aa1c4cd87b..ceebb0b9bca5 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -442,7 +442,7 @@ struct bpf_verifier_state { * while they are still in use. */ bool used_as_loop_entry; - bool in_sleepable; + KABI_FILL_HOLE(bool in_sleepable) /* first and last insn idx of this verifier state */ u32 first_insn_idx; -- 2.34.1
participants (2)
-
patchwork bot -
Pu Lehui