On 2021/3/25 3:20, Cong Wang wrote:
On Tue, Mar 23, 2021 at 7:24 PM Yunsheng Lin linyunsheng@huawei.com wrote:
@@ -176,8 +207,23 @@ static inline bool qdisc_run_begin(struct Qdisc *qdisc) static inline void qdisc_run_end(struct Qdisc *qdisc) { write_seqcount_end(&qdisc->running);
if (qdisc->flags & TCQ_F_NOLOCK)
if (qdisc->flags & TCQ_F_NOLOCK) { spin_unlock(&qdisc->seqlock);/* qdisc_run_end() is protected by RCU lock, and* qdisc reset will do a synchronize_net() after* setting __QDISC_STATE_DEACTIVATED, so testing* the below two bits separately should be fine.Hmm, why synchronize_net() after setting this bit is fine? It could still be flipped right after you test RESCHEDULE bit.
That depends on when it will be fliped again.
As I see: 1. __QDISC_STATE_DEACTIVATED is set during dev_deactivate() process, which should also wait for all process related to "test_bit( __QDISC_STATE_NEED_RESCHEDULE, &q->state)" to finish by calling synchronize_net() and checking some_qdisc_is_busy().
2. it is cleared during dev_activate() process.
And dev_deactivate() and dev_activate() is protected by RTNL lock, or serialized by linkwatch.
* For qdisc_run() in net_tx_action() case, we* really should provide rcu protection explicitly* for document purposes or PREEMPT_RCU.*/if (unlikely(test_bit(__QDISC_STATE_NEED_RESCHEDULE,&qdisc->state) &&!test_bit(__QDISC_STATE_DEACTIVATED,&qdisc->state)))Why do you want to test __QDISC_STATE_DEACTIVATED bit at all? dev_deactivate_many() will wait for those scheduled but being deactivated, so what's the problem of scheduling it even with this bit?
The problem I tried to fix is:
CPU0(calling dev_deactivate) CPU1(calling qdisc_run_end) CPU2(calling tx_atcion) . __netif_schedule() . . set __QDISC_STATE_SCHED . . . . clear __QDISC_STATE_DEACTIVATED . . synchronize_net() . . . . . . . clear __QDISC_STATE_SCHED . . . some_qdisc_is_busy() return false . . . . . . . qdisc_run()
some_qdisc_is_busy() checks if the qdisc is busy by checking __QDISC_STATE_SCHED and spin_is_locked(&qdisc->seqlock) for lockless qdisc, and some_qdisc_is_busy() return false for CPU0 because CPU2 has cleared the __QDISC_STATE_SCHED and has not taken the qdisc->seqlock yet, qdisc is clearly still busy when qdisc_run() is run by CPU2 later.
So you are right, testing __QDISC_STATE_DEACTIVATED does not completely solve the above data race, and there are __netif_schedule() called by dev_requeue_skb() and __qdisc_run() too, which need the same fixing.
So will remove the __QDISC_STATE_DEACTIVATED testing for this patch first, and deal with it later.
Thanks.
.