From: Zhihao Cheng chengzhihao1@huawei.com
hulk inclusion category: bugfix bugzilla: NA CVE: NA
---------------------------
The __this_cpu*() accessors are (in general) IRQ-unsafe which, given that percpu-rwsem is a blocking primitive, should be just fine.
However, file_end_write() is used from IRQ context and will cause load-store issues.
Fixing it by using the IRQ-safe this_cpu_*() for operations on read_count. This will generate more expensive code on a number of platforms, which might cause a performance regression for some of the other percpu-rwsem users.
If any such is reported, we can consider alternative solutions.
Fixes: 70fe2f48152e ("aio: fix freeze protection of aio writes") Signed-off-by: Hou Tao houtao1@huawei.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Link: https://lkml.kernel.org/r/20200915140750.137881-1-houtao1@huawei.com
Signed-off-by: Zhihao Cheng chengzhihao1@huawei.com Reviewed-by: Hou Tao houtao1@huawei.com Signed-off-by: Yang Yingliang yangyingliang@huawei.com --- include/linux/percpu-rwsem.h | 6 +++--- kernel/locking/percpu-rwsem.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index 79b99d653e03..78abe15f7e66 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -44,7 +44,7 @@ static inline void percpu_down_read_preempt_disable(struct percpu_rw_semaphore * * and that one the synchronize_sched() is done, the writer will see * anything we did within this RCU-sched read-size critical section. */ - __this_cpu_inc(*sem->read_count); + this_cpu_inc(*sem->read_count); if (unlikely(!rcu_sync_is_idle(&sem->rss))) __percpu_down_read(sem, false); /* Unconditional memory barrier */ barrier(); @@ -68,7 +68,7 @@ static inline int percpu_down_read_trylock(struct percpu_rw_semaphore *sem) /* * Same as in percpu_down_read(). */ - __this_cpu_inc(*sem->read_count); + this_cpu_inc(*sem->read_count); if (unlikely(!rcu_sync_is_idle(&sem->rss))) ret = __percpu_down_read(sem, true); /* Unconditional memory barrier */ preempt_enable(); @@ -94,7 +94,7 @@ static inline void percpu_up_read_preempt_enable(struct percpu_rw_semaphore *sem * Same as in percpu_down_read(). */ if (likely(rcu_sync_is_idle(&sem->rss))) - __this_cpu_dec(*sem->read_count); + this_cpu_dec(*sem->read_count); else __percpu_up_read(sem); /* Unconditional memory barrier */ preempt_enable(); diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c index 883cf1b92d90..6a2824fe7139 100644 --- a/kernel/locking/percpu-rwsem.c +++ b/kernel/locking/percpu-rwsem.c @@ -99,7 +99,7 @@ void __percpu_up_read(struct percpu_rw_semaphore *sem) * zero, as that is the only time it matters) they will also see our * critical section. */ - __this_cpu_dec(*sem->read_count); + this_cpu_dec(*sem->read_count);
/* Prod writer to recheck readers_active */ rcuwait_wake_up(&sem->writer);