From: Krister Johansen kjlx@templeofstupid.com
mainline inclusion from mainline-v6.4 commit 0108a4e9f3584a7a2c026d1601b0682ff7335d95 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAHA5K
Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
--------------------------------
When subprograms are in use, the main program is not jit'd after the subprograms because jit_subprogs sets a value for prog->bpf_func upon success. Subsequent calls to the JIT are bypassed when this value is non-NULL. This leads to a situation where the main program and its func[0] counterpart are both in the bpf kallsyms tree, but only func[0] has an extable. Extables are only created during JIT. Now there are two nearly identical program ksym entries in the tree, but only one has an extable. Depending upon how the entries are placed, there's a chance that a fault will call search_extable on the aux with the NULL entry.
Since jit_subprogs already copies state from func[0] to the main program, include the extable pointer in this state duplication. Additionally, ensure that the copy of the main program in func[0] is not added to the bpf_prog_kallsyms table. Instead, let the main program get added later in bpf_prog_load(). This ensures there is only a single copy of the main program in the kallsyms table, and that its tag matches the tag observed by tooling like bpftool.
Cc: stable@vger.kernel.org Fixes: 1c2a088a6626 ("bpf: x64: add JIT support for multi-function programs") Signed-off-by: Krister Johansen kjlx@templeofstupid.com Acked-by: Yonghong Song yhs@fb.com Acked-by: Ilya Leoshkevich iii@linux.ibm.com Tested-by: Ilya Leoshkevich iii@linux.ibm.com Link: https://lore.kernel.org/r/6de9b2f4b4724ef56efbb0339daaa66c8b68b1e7.168661666... Signed-off-by: Alexei Starovoitov ast@kernel.org
Conflicts: kernel/bpf/verifier.c [This is because we did not backport d00c6473b1ee ("bpf: Use prog->jited_len in bpf_prog_ksym_set_addr()") and ed2d9e1a26cc ("bpf: Use size instead of pages in bpf_binary_header")] Signed-off-by: Tengda Wu wutengda2@huawei.com --- kernel/bpf/verifier.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index b999b801da8c..c9dce39f7644 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -11298,9 +11298,10 @@ static int jit_subprogs(struct bpf_verifier_env *env) }
/* finally lock prog and jit images for all functions and - * populate kallsysm + * populate kallsysm. Begin at the first subprogram, since + * bpf_prog_load will add the kallsyms for the main program. */ - for (i = 0; i < env->subprog_cnt; i++) { + for (i = 1; i < env->subprog_cnt; i++) { bpf_prog_lock_ro(func[i]); bpf_prog_kallsyms_add(func[i]); } @@ -11320,6 +11321,8 @@ static int jit_subprogs(struct bpf_verifier_env *env)
prog->jited = 1; prog->bpf_func = func[0]->bpf_func; + prog->aux->extable = func[0]->aux->extable; + prog->aux->num_exentries = func[0]->aux->num_exentries; prog->aux->func = func; prog->aux->func_cnt = env->subprog_cnt; bpf_prog_free_unused_jited_linfo(prog);