From: Mark Rutland mark.rutland@arm.com
mainline inclusion from mainline-v5.17-rc1 commit 6ce895128b3bff738fe8d9dd74747a03e319e466 category: performance bugzilla: https://gitee.com/openeuler/kernel/issues/IA5WFA
Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
----------------------------------------------------------------------
Some thread flags can be set remotely, and so even when IRQs are disabled, the flags can change under our feet. Generally this is unlikely to cause a problem in practice, but it is somewhat unsound, and KCSAN will legitimately warn that there is a data race.
To avoid such issues, a snapshot of the flags has to be taken prior to using them. Some places already use READ_ONCE() for that, others do not.
Convert them all to the new flag accessor helpers.
Signed-off-by: Mark Rutland mark.rutland@arm.com Signed-off-by: Thomas Gleixner tglx@linutronix.de Acked-by: Paul E. McKenney paulmck@kernel.org Link: https://lore.kernel.org/r/20211129130653.2037928-3-mark.rutland@arm.com
Conflict: kernel/entry/common.c
Signed-off-by: Zheng Zengkai zhengzengkai@huawei.com --- include/linux/entry-kvm.h | 2 +- kernel/entry/common.c | 4 ++-- kernel/entry/kvm.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/linux/entry-kvm.h b/include/linux/entry-kvm.h index 859fdfd7d46c..4fe38c2d2706 100644 --- a/include/linux/entry-kvm.h +++ b/include/linux/entry-kvm.h @@ -60,7 +60,7 @@ int xfer_to_guest_mode_handle_work(struct kvm_vcpu *vcpu); */ static inline bool __xfer_to_guest_mode_work_pending(void) { - unsigned long ti_work = READ_ONCE(current_thread_info()->flags); + unsigned long ti_work = read_thread_flags();
return !!(ti_work & XFER_TO_GUEST_MODE_WORK); } diff --git a/kernel/entry/common.c b/kernel/entry/common.c index a028b28daed5..7e4fc453da7d 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -186,7 +186,7 @@ static unsigned long exit_to_user_mode_loop(struct pt_regs *regs, /* Check if any of the above work has queued a deferred wakeup */ rcu_nocb_flush_deferred_wakeup();
- ti_work = READ_ONCE(current_thread_info()->flags); + ti_work = read_thread_flags(); }
/* Return the latest work state for arch_exit_to_user_mode() */ @@ -195,7 +195,7 @@ static unsigned long exit_to_user_mode_loop(struct pt_regs *regs,
static void exit_to_user_mode_prepare(struct pt_regs *regs) { - unsigned long ti_work = READ_ONCE(current_thread_info()->flags); + unsigned long ti_work = read_thread_flags();
lockdep_assert_irqs_disabled();
diff --git a/kernel/entry/kvm.c b/kernel/entry/kvm.c index 7b946847be78..6d8b245aad4f 100644 --- a/kernel/entry/kvm.c +++ b/kernel/entry/kvm.c @@ -25,7 +25,7 @@ static int xfer_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work) if (ret) return ret;
- ti_work = READ_ONCE(current_thread_info()->flags); + ti_work = read_thread_flags(); } while (ti_work & XFER_TO_GUEST_MODE_WORK || need_resched()); return 0; } @@ -42,7 +42,7 @@ int xfer_to_guest_mode_handle_work(struct kvm_vcpu *vcpu) * disabled in the inner loop before going into guest mode. No need * to disable interrupts here. */ - ti_work = READ_ONCE(current_thread_info()->flags); + ti_work = read_thread_flags(); if (!(ti_work & XFER_TO_GUEST_MODE_WORK)) return 0;