- Kernel - mailweb.openeuler.org

[PATCH openEuler-1.0-LTS v3 0/2] fix CVE-2023-52587
by Yu Liao 21 Mar '24

21 Mar '24

fix CVE-2023-52587 Daniel Vacek (1): IB/ipoib: Fix mcast list locking Jack Wang (1): RDMA/IPoIB: Fix error code return in ipoib_mcast_join drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) -- 2.25.1

2 3

[PATCH OLK-6.6 0/1] arm64: Delete macro in the scsnp feature
by lishusen 21 Mar '24

21 Mar '24

The macro in the scsnp feature is deleted to rectify the compilation problem. lishusen (1): arm64: Delete macro in the scsnp feature arch/arm64/kernel/image-vars.h | 2 -- 1 file changed, 2 deletions(-) -- 2.33.0

2 2

[PATCH openEuler-1.0-LTS v2 0/5] CVE-2021-47131
by Ziyang Xuan 21 Mar '24

21 Mar '24

Patchset for CVE-2021-47131. v2: - Add bugfix patches for CVE patch. Maxim Mikityanskiy (5): net/tls: Replace TLS_RX_SYNC_RUNNING with RCU net/tls: Fix use-after-free after the TLS device goes down and up tls: Fix context leak on tls_device_down net/tls: Remove the context from the list in tls_device_down net/tls: Use RCU API to access tls_ctx->netdev include/net/tls.h | 13 ++++- net/tls/tls_device.c | 89 ++++++++++++++++++++++++++++------- net/tls/tls_device_fallback.c | 9 +++- net/tls/tls_main.c | 1 + 4 files changed, 93 insertions(+), 19 deletions(-) -- 2.25.1

2 6

[PATCH openEuler-1.0-LTS v2 0/2] fix CVE-2023-52587
by Yu Liao 21 Mar '24

21 Mar '24

fix CVE-2023-52587 Daniel Vacek (1): IB/ipoib: Fix mcast list locking Jack Wang (1): RDMA/IPoIB: Fix error code return in ipoib_mcast_join drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) -- 2.25.1

2 3

[PATCH OLK-5.10] ima: fix deadlock when traversing "ima_default_rules".
by GUO Zihua 21 Mar '24

21 Mar '24

From: liqiong <liqiong(a)nfschina.com> mainline inclusion from mainline-v5.16-rc1 commit eb0782bbdfd0d7c4786216659277c3fd585afc0e category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9AAM6 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The current IMA ruleset is identified by the variable "ima_rules" that default to "&ima_default_rules". When loading a custom policy for the first time, the variable is updated to "&ima_policy_rules" instead. That update isn't RCU-safe, and deadlocks are possible. Indeed, some functions like ima_match_policy() may loop indefinitely when traversing "ima_default_rules" with list_for_each_entry_rcu(). When iterating over the default ruleset back to head, if the list head is "ima_default_rules", and "ima_rules" have been updated to "&ima_policy_rules", the loop condition (&entry->list != ima_rules) stays always true, traversing won't terminate, causing a soft lockup and RCU stalls. Introduce a temporary value for "ima_rules" when iterating over the ruleset to avoid the deadlocks. Signed-off-by: liqiong <liqiong(a)nfschina.com> Reviewed-by: THOBY Simon <Simon.THOBY(a)viveris.fr> Fixes: 38d859f991f3 ("IMA: policy can now be updated multiple times") Reported-by: kernel test robot <lkp(a)intel.com> (Fix sparse: incompatible types in comparison expression.) Signed-off-by: Mimi Zohar <zohar(a)linux.ibm.com> Conflicts: security/integrity/ima/ima_policy.c Signed-off-by: GUO Zihua <guozihua(a)huawei.com> --- security/integrity/ima/ima_policy.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/security/integrity/ima/ima_policy.c b/security/integrity/ima/ima_policy.c index f0bbd65e3e9d..a0b0008efc91 100644 --- a/security/integrity/ima/ima_policy.c +++ b/security/integrity/ima/ima_policy.c @@ -245,7 +245,7 @@ static struct ima_rule_entry *arch_policy_entry __ro_after_init; static LIST_HEAD(ima_default_rules); static LIST_HEAD(ima_policy_rules); static LIST_HEAD(ima_temp_rules); -static struct list_head *ima_rules = &ima_default_rules; +static struct list_head __rcu *ima_rules = (struct list_head __rcu *)(&ima_default_rules); static int ima_policy __initdata; @@ -702,12 +702,14 @@ int ima_match_policy(struct inode *inode, const struct cred *cred, u32 secid, { struct ima_rule_entry *entry; int action = 0, actmask = flags | (flags << 1); + struct list_head *ima_rules_tmp; if (template_desc) *template_desc = ima_template_desc_current(); rcu_read_lock(); - list_for_each_entry_rcu(entry, ima_rules, list) { + ima_rules_tmp = rcu_dereference(ima_rules); + list_for_each_entry_rcu(entry, ima_rules_tmp, list) { if (!(entry->action & actmask)) continue; @@ -755,8 +757,10 @@ int ima_match_policy(struct inode *inode, const struct cred *cred, u32 secid, void ima_update_policy_flag(void) { struct ima_rule_entry *entry; + struct list_head *ima_rules_tmp; - list_for_each_entry(entry, ima_rules, list) { + ima_rules_tmp = rcu_dereference(ima_rules); + list_for_each_entry_rcu(entry, ima_rules_tmp, list) { if (entry->action & IMA_DO_MASK) ima_policy_flag |= entry->action; } @@ -1019,10 +1023,10 @@ void ima_update_policy(void) list_splice_tail_init_rcu(&ima_temp_rules, policy, synchronize_rcu); - if (ima_rules != policy) { + if (ima_rules != (struct list_head __rcu *)policy) { ima_policy_flag = 0; - ima_rules = policy; + rcu_assign_pointer(ima_rules, policy); /* * IMA architecture specific policy rules are specified * as strings and converted to an array of ima_entry_rules @@ -1117,7 +1121,7 @@ static int ima_lsm_rule_init(struct ima_rule_entry *entry, pr_warn("rule for LSM \'%s\' is undefined\n", entry->lsm[lsm_rule].args_p); - if (ima_rules == &ima_default_rules) { + if (ima_rules == (struct list_head __rcu *)(&ima_default_rules)) { kfree(entry->lsm[lsm_rule].args_p); entry->lsm[lsm_rule].args_p = NULL; result = -EINVAL; @@ -1764,9 +1768,11 @@ void *ima_policy_start(struct seq_file *m, loff_t *pos) { loff_t l = *pos; struct ima_rule_entry *entry; + struct list_head *ima_rules_tmp; rcu_read_lock(); - list_for_each_entry_rcu(entry, ima_rules, list) { + ima_rules_tmp = rcu_dereference(ima_rules); + list_for_each_entry_rcu(entry, ima_rules_tmp, list) { if (!l--) { rcu_read_unlock(); return entry; @@ -1785,7 +1791,8 @@ void *ima_policy_next(struct seq_file *m, void *v, loff_t *pos) rcu_read_unlock(); (*pos)++; - return (&entry->list == ima_rules) ? NULL : entry; + return (&entry->list == &ima_default_rules || + &entry->list == &ima_policy_rules) ? NULL : entry; } void ima_policy_stop(struct seq_file *m, void *v) @@ -1998,6 +2005,7 @@ bool ima_appraise_signature(enum kernel_read_file_id id) struct ima_rule_entry *entry; bool found = false; enum ima_hooks func; + struct list_head *ima_rules_tmp; if (id >= READING_MAX_ID) return false; @@ -2009,7 +2017,8 @@ bool ima_appraise_signature(enum kernel_read_file_id id) func = read_idmap[id] ?: FILE_CHECK; rcu_read_lock(); - list_for_each_entry_rcu(entry, ima_rules, list) { + ima_rules_tmp = rcu_dereference(ima_rules); + list_for_each_entry_rcu(entry, ima_rules_tmp, list) { if (entry->action != APPRAISE) continue; -- 2.34.1

2 1

[PATCH openEuler-1.0-LTS 0/3] fix CVE-2023-52612
by felix 21 Mar '24

21 Mar '24

From: Felix Fu <fuzhen5(a)huawei.com> *** BLURB HERE *** Chengming Zhou (1): crypto: scomp - fix req->dst buffer overflow Sebastian Andrzej Siewior (2): crypto: scompress - return proper error code for allocation failure crypto: scompress - Use per-CPU struct instead multiple variables crypto/scompress.c | 135 +++++++++++++++++++++------------------------ 1 file changed, 63 insertions(+), 72 deletions(-) -- 2.34.1

2 4

[PATCH openEuler-1.0-LTS 1/3] crypto: scompress - return proper error code for allocation failure
by felix 21 Mar '24

21 Mar '24

From: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> stable inclusion from stable-v4.19.306 commit 1915874d67287ba1ab71825ae6a4efbb2a0e2b11 bugzilla: https://gitee.com/src-openeuler/kernel/issues/I99K14 CVE: CVE-2023-52612 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit 6a4d1b18ef00a7b182740b7b4d8a0fcd317368f8 ] If scomp_acomp_comp_decomp() fails to allocate memory for the destination then we never copy back the data we compressed. It is probably best to return an error code instead 0 in case of failure. I haven't found any user that is using acomp_request_set_params() without the `dst' buffer so there is probably no harm. Signed-off-by: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> Signed-off-by: Herbert Xu <herbert(a)gondor.apana.org.au> Stable-dep-of: 744e1885922a ("crypto: scomp - fix req->dst buffer overflow") Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Felix Fu <fuzhen5(a)huawei.com> --- crypto/scompress.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/crypto/scompress.c b/crypto/scompress.c index 968bbcf65c94..15641c96ff99 100644 --- a/crypto/scompress.c +++ b/crypto/scompress.c @@ -174,8 +174,10 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir) if (!ret) { if (!req->dst) { req->dst = sgl_alloc(req->dlen, GFP_ATOMIC, NULL); - if (!req->dst) + if (!req->dst) { + ret = -ENOMEM; goto out; + } } scatterwalk_map_and_copy(scratch_dst, req->dst, 0, req->dlen, 1); -- 2.34.1

2 4

[PATCH openEuler-22.03-LTS 0/2] Fix CVE-2021-47094
by Liu Shixin 21 Mar '24

21 Mar '24

The first patch is a cleanup patch and no functional change. The second patch is bugfix for CVE-2021-47094. Ben Gardon (1): KVM: x86/mmu: Factor out tdp_iter_return_to_root Sean Christopherson (1): KVM: x86/mmu: Don't advance iterator after restart due to yielding arch/x86/kvm/mmu/tdp_iter.c | 30 +++++++++++++++++++++++------- arch/x86/kvm/mmu/tdp_iter.h | 7 +++++++ arch/x86/kvm/mmu/tdp_mmu.c | 28 ++++++++++++++-------------- 3 files changed, 44 insertions(+), 21 deletions(-) -- 2.25.1

2 3

[PATCH openEuler-1.0-LTS] keys: safe concurrent user->{session,uid}_keyring access
by GUO Zihua 21 Mar '24

21 Mar '24

From: Jann Horn <jannh(a)google.com> mainline inclusion from mainline-v5.2-rc1 commit 0b9dc6c9f01c4a726558b82a3b6082a89d264eb5 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I9355O CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- The current code can perform concurrent updates and reads on user->session_keyring and user->uid_keyring. Add a comment to struct user_struct to document the nontrivial locking semantics, and use READ_ONCE() for unlocked readers and smp_store_release() for writers to prevent memory ordering issues. Fixes: 69664cf16af4 ("keys: don't generate user and user session keyrings unless they're accessed") Signed-off-by: Jann Horn <jannh(a)google.com> Signed-off-by: James Morris <james.morris(a)microsoft.com> Conflicts: security/keys/process_keys.c security/keys/request_key.c Signed-off-by: GUO Zihua <guozihua(a)huawei.com> --- include/linux/sched/user.h | 7 +++++++ security/keys/process_keys.c | 31 +++++++++++++++++-------------- security/keys/request_key.c | 5 +++-- 3 files changed, 27 insertions(+), 16 deletions(-) diff --git a/include/linux/sched/user.h b/include/linux/sched/user.h index 02c3b646bd07..38d3606ff09b 100644 --- a/include/linux/sched/user.h +++ b/include/linux/sched/user.h @@ -32,6 +32,13 @@ struct user_struct { atomic_long_t pipe_bufs; /* how many pages are allocated in pipe buffers */ #ifdef CONFIG_KEYS + /* + * These pointers can only change from NULL to a non-NULL value once. + * Writes are protected by key_user_keyring_mutex. + * Unlocked readers should use READ_ONCE() unless they know that + * install_user_keyrings() has been called successfully (which sets + * these members to non-NULL values, preventing further modifications). + */ struct key *uid_keyring; /* UID specific keyring */ struct key *session_keyring; /* UID's default session keyring */ #endif diff --git a/security/keys/process_keys.c b/security/keys/process_keys.c index d776d2114651..b5a8202af128 100644 --- a/security/keys/process_keys.c +++ b/security/keys/process_keys.c @@ -59,7 +59,7 @@ int install_user_keyrings(void) kenter("%p{%u}", user, uid); - if (user->uid_keyring && user->session_keyring) { + if (READ_ONCE(user->uid_keyring) && READ_ONCE(user->session_keyring)) { kleave(" = 0 [exist]"); return 0; } @@ -112,8 +112,10 @@ int install_user_keyrings(void) } /* install the keyrings */ - user->uid_keyring = uid_keyring; - user->session_keyring = session_keyring; + /* paired with READ_ONCE() */ + smp_store_release(&user->uid_keyring, uid_keyring); + /* paired with READ_ONCE() */ + smp_store_release(&user->session_keyring, session_keyring); } mutex_unlock(&key_user_keyring_mutex); @@ -340,6 +342,7 @@ void key_fsgid_changed(struct task_struct *tsk) key_ref_t search_my_process_keyrings(struct keyring_search_context *ctx) { key_ref_t key_ref, ret, err; + const struct cred *cred = ctx->cred; /* we want to return -EAGAIN or -ENOKEY if any of the keyrings were * searchable, but we failed to find a key or we found a negative key; @@ -353,9 +356,9 @@ key_ref_t search_my_process_keyrings(struct keyring_search_context *ctx) err = ERR_PTR(-EAGAIN); /* search the thread keyring first */ - if (ctx->cred->thread_keyring) { + if (cred->thread_keyring) { key_ref = keyring_search_aux( - make_key_ref(ctx->cred->thread_keyring, 1), ctx); + make_key_ref(cred->thread_keyring, 1), ctx); if (!IS_ERR(key_ref)) goto found; @@ -371,9 +374,9 @@ key_ref_t search_my_process_keyrings(struct keyring_search_context *ctx) } /* search the process keyring second */ - if (ctx->cred->process_keyring) { + if (cred->process_keyring) { key_ref = keyring_search_aux( - make_key_ref(ctx->cred->process_keyring, 1), ctx); + make_key_ref(cred->process_keyring, 1), ctx); if (!IS_ERR(key_ref)) goto found; @@ -391,10 +394,10 @@ key_ref_t search_my_process_keyrings(struct keyring_search_context *ctx) } /* search the session keyring */ - if (ctx->cred->session_keyring) { + if (cred->session_keyring) { rcu_read_lock(); key_ref = keyring_search_aux( - make_key_ref(rcu_dereference(ctx->cred->session_keyring), 1), + make_key_ref(rcu_dereference(cred->session_keyring), 1), ctx); rcu_read_unlock(); @@ -414,9 +417,9 @@ key_ref_t search_my_process_keyrings(struct keyring_search_context *ctx) } } /* or search the user-session keyring */ - else if (ctx->cred->user->session_keyring) { + else if (READ_ONCE(cred->user->session_keyring)) { key_ref = keyring_search_aux( - make_key_ref(ctx->cred->user->session_keyring, 1), + make_key_ref(READ_ONCE(cred->user->session_keyring), 1), ctx); if (!IS_ERR(key_ref)) goto found; @@ -602,7 +605,7 @@ key_ref_t lookup_user_key(key_serial_t id, unsigned long lflags, goto error; goto reget_creds; } else if (ctx.cred->session_keyring == - ctx.cred->user->session_keyring && + READ_ONCE(ctx.cred->user->session_keyring) && lflags & KEY_LOOKUP_CREATE) { ret = join_session_keyring(NULL); if (ret < 0) @@ -618,7 +621,7 @@ key_ref_t lookup_user_key(key_serial_t id, unsigned long lflags, break; case KEY_SPEC_USER_KEYRING: - if (!ctx.cred->user->uid_keyring) { + if (!READ_ONCE(ctx.cred->user->uid_keyring)) { ret = install_user_keyrings(); if (ret < 0) goto error; @@ -630,7 +633,7 @@ key_ref_t lookup_user_key(key_serial_t id, unsigned long lflags, break; case KEY_SPEC_USER_SESSION_KEYRING: - if (!ctx.cred->user->session_keyring) { + if (!READ_ONCE(ctx.cred->user->session_keyring)) { ret = install_user_keyrings(); if (ret < 0) goto error; diff --git a/security/keys/request_key.c b/security/keys/request_key.c index 1a547cec8385..f62bb12bca41 100644 --- a/security/keys/request_key.c +++ b/security/keys/request_key.c @@ -294,11 +294,12 @@ static int construct_get_dest_keyring(struct key **_dest_keyring) case KEY_REQKEY_DEFL_USER_SESSION_KEYRING: dest_keyring = - key_get(cred->user->session_keyring); + key_get(READ_ONCE(cred->user->session_keyring)); break; case KEY_REQKEY_DEFL_USER_KEYRING: - dest_keyring = key_get(cred->user->uid_keyring); + dest_keyring = + key_get(READ_ONCE(cred->user->uid_keyring)); break; case KEY_REQKEY_DEFL_GROUP_KEYRING: -- 2.34.1

2 1

[PATCH OLK-5.10] KVM: x86/mmu: Don't advance iterator after restart due to yielding
by Liu Shixin 21 Mar '24

21 Mar '24

From: Sean Christopherson <seanjc(a)google.com> mainline inclusion from mainline-v5.16-rc7 commit 3a0f64de479cae75effb630a2e0a237ca0d0623c category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/I95REM CVE: CVE-2021-47094 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… -------------------------------- After dropping mmu_lock in the TDP MMU, restart the iterator during tdp_iter_next() and do not advance the iterator. Advancing the iterator results in skipping the top-level SPTE and all its children, which is fatal if any of the skipped SPTEs were not visited before yielding. When zapping all SPTEs, i.e. when min_level == root_level, restarting the iter and then invoking tdp_iter_next() is always fatal if the current gfn has as a valid SPTE, as advancing the iterator results in try_step_side() skipping the current gfn, which wasn't visited before yielding. Sprinkle WARNs on iter->yielded being true in various helpers that are often used in conjunction with yielding, and tag the helper with __must_check to reduce the probabily of improper usage. Failing to zap a top-level SPTE manifests in one of two ways. If a valid SPTE is skipped by both kvm_tdp_mmu_zap_all() and kvm_tdp_mmu_put_root(), the shadow page will be leaked and KVM will WARN accordingly. WARNING: CPU: 1 PID: 3509 at arch/x86/kvm/mmu/tdp_mmu.c:46 [kvm] RIP: 0010:kvm_mmu_uninit_tdp_mmu+0x3e/0x50 [kvm] Call Trace: <TASK> kvm_arch_destroy_vm+0x130/0x1b0 [kvm] kvm_destroy_vm+0x162/0x2a0 [kvm] kvm_vcpu_release+0x34/0x60 [kvm] __fput+0x82/0x240 task_work_run+0x5c/0x90 do_exit+0x364/0xa10 ? futex_unqueue+0x38/0x60 do_group_exit+0x33/0xa0 get_signal+0x155/0x850 arch_do_signal_or_restart+0xed/0x750 exit_to_user_mode_prepare+0xc5/0x120 syscall_exit_to_user_mode+0x1d/0x40 do_syscall_64+0x48/0xc0 entry_SYSCALL_64_after_hwframe+0x44/0xae If kvm_tdp_mmu_zap_all() skips a gfn/SPTE but that SPTE is then zapped by kvm_tdp_mmu_put_root(), KVM triggers a use-after-free in the form of marking a struct page as dirty/accessed after it has been put back on the free list. This directly triggers a WARN due to encountering a page with page_count() == 0, but it can also lead to data corruption and additional errors in the kernel. WARNING: CPU: 7 PID: 1995658 at arch/x86/kvm/../../../virt/kvm/kvm_main.c:171 RIP: 0010:kvm_is_zone_device_pfn.part.0+0x9e/0xd0 [kvm] Call Trace: <TASK> kvm_set_pfn_dirty+0x120/0x1d0 [kvm] __handle_changed_spte+0x92e/0xca0 [kvm] __handle_changed_spte+0x63c/0xca0 [kvm] __handle_changed_spte+0x63c/0xca0 [kvm] __handle_changed_spte+0x63c/0xca0 [kvm] zap_gfn_range+0x549/0x620 [kvm] kvm_tdp_mmu_put_root+0x1b6/0x270 [kvm] mmu_free_root_page+0x219/0x2c0 [kvm] kvm_mmu_free_roots+0x1b4/0x4e0 [kvm] kvm_mmu_unload+0x1c/0xa0 [kvm] kvm_arch_destroy_vm+0x1f2/0x5c0 [kvm] kvm_put_kvm+0x3b1/0x8b0 [kvm] kvm_vcpu_release+0x4e/0x70 [kvm] __fput+0x1f7/0x8c0 task_work_run+0xf8/0x1a0 do_exit+0x97b/0x2230 do_group_exit+0xda/0x2a0 get_signal+0x3be/0x1e50 arch_do_signal_or_restart+0x244/0x17f0 exit_to_user_mode_prepare+0xcb/0x120 syscall_exit_to_user_mode+0x1d/0x40 do_syscall_64+0x4d/0x90 entry_SYSCALL_64_after_hwframe+0x44/0xae Note, the underlying bug existed even before commit 1af4a96025b3 ("KVM: x86/mmu: Yield in TDU MMU iter even if no SPTES changed") moved calls to tdp_mmu_iter_cond_resched() to the beginning of loops, as KVM could still incorrectly advance past a top-level entry when yielding on a lower-level entry. But with respect to leaking shadow pages, the bug was introduced by yielding before processing the current gfn. Alternatively, tdp_mmu_iter_cond_resched() could simply fall through, or callers could jump to their "retry" label. The downside of that approach is that tdp_mmu_iter_cond_resched() _must_ be called before anything else in the loop, and there's no easy way to enfornce that requirement. Ideally, KVM would handling the cond_resched() fully within the iterator macro (the code is actually quite clean) and avoid this entire class of bugs, but that is extremely difficult do while also supporting yielding after tdp_mmu_set_spte_atomic() fails. Yielding after failing to set a SPTE is very desirable as the "owner" of the REMOVED_SPTE isn't strictly bounded, e.g. if it's zapping a high-level shadow page, the REMOVED_SPTE may block operations on the SPTE for a significant amount of time. Fixes: faaf05b00aec ("kvm: x86/mmu: Support zapping SPTEs in the TDP MMU") Fixes: 1af4a96025b3 ("KVM: x86/mmu: Yield in TDU MMU iter even if no SPTES changed") Reported-by: Ignat Korchagin <ignat(a)cloudflare.com> Cc: stable(a)vger.kernel.org Signed-off-by: Sean Christopherson <seanjc(a)google.com> Message-Id: <20211214033528.123268-1-seanjc(a)google.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> Signed-off-by: Liu Shixin <liushixin2(a)huawei.com> --- arch/x86/kvm/mmu/tdp_iter.c | 6 ++++++ arch/x86/kvm/mmu/tdp_iter.h | 6 ++++++ arch/x86/kvm/mmu/tdp_mmu.c | 29 ++++++++++++++++------------- 3 files changed, 28 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.c b/arch/x86/kvm/mmu/tdp_iter.c index b3ed302c1a35..caa96c270b95 100644 --- a/arch/x86/kvm/mmu/tdp_iter.c +++ b/arch/x86/kvm/mmu/tdp_iter.c @@ -26,6 +26,7 @@ static gfn_t round_gfn_for_level(gfn_t gfn, int level) */ void tdp_iter_restart(struct tdp_iter *iter) { + iter->yielded = false; iter->yielded_gfn = iter->next_last_level_gfn; iter->level = iter->root_level; @@ -160,6 +161,11 @@ static bool try_step_up(struct tdp_iter *iter) */ void tdp_iter_next(struct tdp_iter *iter) { + if (iter->yielded) { + tdp_iter_restart(iter); + return; + } + if (try_step_down(iter)) return; diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index b1748b988d3a..e19cabbcb65c 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -45,6 +45,12 @@ struct tdp_iter { * iterator walks off the end of the paging structure. */ bool valid; + /* + * True if KVM dropped mmu_lock and yielded in the middle of a walk, in + * which case tdp_iter_next() needs to restart the walk at the root + * level instead of advancing to the next entry. + */ + bool yielded; }; /* diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 27e18b82f8d5..1af891ff7a2b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -516,6 +516,8 @@ static inline bool tdp_mmu_set_spte_atomic_no_dirty_log(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte) { + WARN_ON_ONCE(iter->yielded); + lockdep_assert_held_read(&kvm->mmu_lock); /* @@ -601,6 +603,8 @@ static inline void __tdp_mmu_set_spte(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte, bool record_acc_track, bool record_dirty_log) { + WARN_ON_ONCE(iter->yielded); + lockdep_assert_held_write(&kvm->mmu_lock); /* @@ -666,18 +670,19 @@ static inline void tdp_mmu_set_spte_no_dirty_log(struct kvm *kvm, * If this function should yield and flush is set, it will perform a remote * TLB flush before yielding. * - * If this function yields, it will also reset the tdp_iter's walk over the - * paging structure and the calling function should skip to the next - * iteration to allow the iterator to continue its traversal from the - * paging structure root. + * If this function yields, iter->yielded is set and the caller must skip to + * the next iteration, where tdp_iter_next() will reset the tdp_iter's walk + * over the paging structures to allow the iterator to continue its traversal + * from the paging structure root. * - * Return true if this function yielded and the iterator's traversal was reset. - * Return false if a yield was not needed. + * Returns true if this function yielded. */ -static inline bool tdp_mmu_iter_cond_resched(struct kvm *kvm, - struct tdp_iter *iter, bool flush, - bool shared) +static inline bool __must_check tdp_mmu_iter_cond_resched(struct kvm *kvm, + struct tdp_iter *iter, + bool flush, bool shared) { + WARN_ON(iter->yielded); + /* Ensure forward progress has been made before yielding. */ if (iter->next_last_level_gfn == iter->yielded_gfn) return false; @@ -697,12 +702,10 @@ static inline bool tdp_mmu_iter_cond_resched(struct kvm *kvm, WARN_ON(iter->gfn > iter->next_last_level_gfn); - tdp_iter_restart(iter); - - return true; + iter->yielded = true; } - return false; + return iter->yielded; } /* -- 2.25.1

2 1