[PATCH 0/9] P2PDMA PATCHSET PART 1
To support iostat statistics of passthrough and support dm/md/nvme-multipath p2pdma. Bart Van Assche (1): block: Fix queue_iostats_passthrough_show() Chaitanya Kulkarni (1): block: clear BLK_FEAT_PCI_P2PDMA in blk_stack_limits() for non-supporting devices Keith Busch (3): block: enable passthrough command statistics block: export passthrough stats enabled nvme: add support multipath passthrough iostats Kiran Kumar Modukuri (2): md: propagate BLK_FEAT_PCI_P2PDMA from member devices to RAID device nvme-multipath: enable PCI P2PDMA for multipath devices Zizhi Wo (1): block: withhold P2PDMA from integrity-bearing stacks hotforest (1): dm: pass PCI P2PDMA through linear and striped targets Documentation/ABI/stable/sysfs-block | 8 ++++++ block/blk-mq-debugfs.c | 1 + block/blk-settings.c | 5 ++++ block/blk-sysfs.c | 26 ++++++++++++++++++ block/blk.h | 4 ++- drivers/md/dm-linear.c | 3 +- drivers/md/dm-stripe.c | 3 +- drivers/md/dm-table.c | 27 ++++++++++++++++++ drivers/md/raid0.c | 2 ++ drivers/md/raid1.c | 1 + drivers/md/raid10.c | 1 + drivers/nvme/host/core.c | 6 ++++ drivers/nvme/host/ioctl.c | 9 ++++++ drivers/nvme/host/multipath.c | 6 +++- include/linux/blk-mq.h | 41 ++++++++++++++++++++++++++++ include/linux/blkdev.h | 3 ++ include/linux/device-mapper.h | 5 ++++ 17 files changed, 147 insertions(+), 4 deletions(-) -- 2.52.0
From: Keith Busch <kbusch@kernel.org> mainline inclusion from mainline-v6.13-rc1 commit 110234da18ab482f6f583d28eff26b9569bf5622 category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Applications using the passthrough interfaces for IO want to continue seeing the disk stats. These requests had been fenced off from this block layer feature. While the block layer doesn't necessarily know what a passthrough command does, we do know the data size and direction, which is enough to account for the command's stats. Since tracking these has the potential to produce unexpected results, the passthrough stats are locked behind a new queue flag that needs to be enabled with the /sys/block/<dev>/queue/iostats_passthrough attribute. Signed-off-by: Keith Busch <kbusch@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/blk.h block/blk-sysfs.c block/blk-mq-debugfs.c include/linux/blkdev.h Documentation/ABI/stable/sysfs-block [6.6 uses queue flags instead of queue_limits/BLK_FEAT_*. Mainline puts blk_rq_passthrough_stats() in block/blk-mq.c because it changes blk_account_io_start() there. In 6.6 the passthrough gate is blk_do_io_stat() in block/blk.h, which is also used from block/blk-merge.c, so blk_rq_passthrough_stats() is defined in block/blk.h and called from blk_do_io_stat(). The new QUEUE_FLAG_IOSTATS_PASSTHROUGH uses free bit 23 and is toggled with blk_queue_flag_set/clear; its name is added to blk_queue_flag_name[] in block/blk-mq-debugfs.c. The sysfs callbacks take struct request_queue *q instead of struct gendisk *disk.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- Documentation/ABI/stable/sysfs-block | 8 +++++++ block/blk-mq-debugfs.c | 1 + block/blk-sysfs.c | 26 ++++++++++++++++++++++ block/blk.h | 33 +++++++++++++++++++++++++++- include/linux/blkdev.h | 3 +++ 5 files changed, 70 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 97fc8034a271..8967a4927047 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -427,10 +427,18 @@ Description: accounted by sampling and the result is not accurate; If this value is 2, iostats accounting is enabled and io_ticks is accounted precisely, but there will be slightly more overhead. +What: /sys/block/<disk>/queue/iostats_passthrough +Date: October 2024 +Contact: linux-block@vger.kernel.org +Description: + [RW] This file is used to control (on/off) the iostats + accounting of the disk for passthrough commands. + + What: /sys/block/<disk>/queue/logical_block_size Date: May 2009 Contact: Martin K. Petersen <martin.petersen@oracle.com> Description: [RO] This is the smallest unit the storage device can address. diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index 033f0171c8ba..8ac15fb05ee7 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -98,10 +98,11 @@ static const char *const blk_queue_flag_name[] = { QUEUE_FLAG_NAME(WC), QUEUE_FLAG_NAME(FUA), QUEUE_FLAG_NAME(DAX), QUEUE_FLAG_NAME(STATS), QUEUE_FLAG_NAME(REGISTERED), + QUEUE_FLAG_NAME(IOSTATS_PASSTHROUGH), QUEUE_FLAG_NAME(QUIESCED), QUEUE_FLAG_NAME(PCI_P2PDMA), QUEUE_FLAG_NAME(ZONE_RESETALL), QUEUE_FLAG_NAME(RQ_ALLOC_TIME), QUEUE_FLAG_NAME(HCTX_ACTIVE), diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 89a142955c28..3384e6793c0c 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -356,10 +356,34 @@ static ssize_t queue_max_open_zones_show(struct request_queue *q, char *page) static ssize_t queue_max_active_zones_show(struct request_queue *q, char *page) { return queue_var_show(bdev_max_active_zones(q->disk->part0), page); } +static ssize_t queue_iostats_passthrough_show(struct request_queue *q, + char *page) +{ + return queue_var_show(blk_queue_passthrough_stat(q), page); +} + +static ssize_t queue_iostats_passthrough_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long ios; + ssize_t ret; + + ret = queue_var_store(&ios, page, count); + if (ret < 0) + return ret; + + if (ios) + blk_queue_flag_set(QUEUE_FLAG_IOSTATS_PASSTHROUGH, q); + else + blk_queue_flag_clear(QUEUE_FLAG_IOSTATS_PASSTHROUGH, q); + + return count; +} + static ssize_t queue_nomerges_show(struct request_queue *q, char *page) { return queue_var_show((blk_queue_nomerges(q) << 1) | blk_queue_noxmerges(q), page); } @@ -585,10 +609,11 @@ QUEUE_RO_ENTRY(queue_zoned, "zoned"); QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); +QUEUE_RW_ENTRY(queue_iostats_passthrough, "iostats_passthrough"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); QUEUE_RW_ENTRY(queue_poll_delay, "io_poll_delay"); QUEUE_RW_ENTRY(queue_wc, "write_cache"); QUEUE_RO_ENTRY(queue_fua, "fua"); @@ -713,10 +738,11 @@ static struct attribute *queue_attrs[] = { &queue_zoned_entry.attr, &queue_nr_zones_entry.attr, &queue_max_open_zones_entry.attr, &queue_max_active_zones_entry.attr, &queue_nomerges_entry.attr, + &queue_iostats_passthrough_entry.attr, &queue_iostats_entry.attr, &queue_stable_writes_entry.attr, &queue_random_entry.attr, &queue_poll_entry.attr, &queue_wc_entry.attr, diff --git a/block/blk.h b/block/blk.h index 0edc05898ca0..9ab353171702 100644 --- a/block/blk.h +++ b/block/blk.h @@ -358,19 +358,50 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio); enum elv_merge blk_try_merge(struct request *rq, struct bio *bio); void blk_set_default_limits(struct queue_limits *lim); int blk_dev_init(void); +static inline bool blk_rq_passthrough_stats(struct request *req) +{ + struct bio *bio = req->bio; + + if (!blk_queue_passthrough_stat(req->q)) + return false; + + /* Requests without a bio do not transfer data. */ + if (!bio) + return false; + + /* + * Stats are accumulated in the bdev, so must have one attached to a + * bio to track stats. Most drivers do not set the bdev for passthrough + * requests, but nvme is one that will set it. + */ + if (!bio->bi_bdev) + return false; + + /* + * We don't know what a passthrough command does, but we know the + * payload size and data direction. Ensuring the size is aligned to the + * block size filters out most commands with payloads that don't + * represent sector access. + */ + if (blk_rq_bytes(req) & (bdev_logical_block_size(bio->bi_bdev) - 1)) + return false; + return true; +} + /* * Contribute to IO statistics IFF: * * a) it's attached to a gendisk, and * b) the queue had IO stats enabled when this request was started */ static inline bool blk_do_io_stat(struct request *rq) { - return (rq->rq_flags & RQF_IO_STAT) && !blk_rq_is_passthrough(rq); + return (rq->rq_flags & RQF_IO_STAT) && + (!blk_rq_is_passthrough(rq) || blk_rq_passthrough_stats(rq)); } void update_io_ticks(struct block_device *part, unsigned long now, bool end); unsigned int part_in_flight(struct block_device *part); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 442c5c82b36b..d1abffe59af9 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -612,10 +612,11 @@ struct request_queue { #define QUEUE_FLAG_WC 17 /* Write back caching */ #define QUEUE_FLAG_FUA 18 /* device supports FUA writes */ #define QUEUE_FLAG_DAX 19 /* device supports DAX */ #define QUEUE_FLAG_STATS 20 /* track IO start and completion times */ #define QUEUE_FLAG_REGISTERED 22 /* queue has been registered to a disk */ +#define QUEUE_FLAG_IOSTATS_PASSTHROUGH 23 /* passthrough command IO accounting */ #define QUEUE_FLAG_QUIESCED 24 /* queue has been quiesced */ #define QUEUE_FLAG_PCI_P2PDMA 25 /* device supports PCI p2p requests */ #define QUEUE_FLAG_ZONE_RESETALL 26 /* supports Zone Reset All */ #define QUEUE_FLAG_RQ_ALLOC_TIME 27 /* record rq->alloc_time_ns */ #define QUEUE_FLAG_HCTX_ACTIVE 28 /* at least one blk-mq hctx is active */ @@ -639,10 +640,12 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); test_bit(QUEUE_FLAG_NOXMERGES, &(q)->queue_flags) #define blk_queue_nonrot(q) test_bit(QUEUE_FLAG_NONROT, &(q)->queue_flags) #define blk_queue_stable_writes(q) \ test_bit(QUEUE_FLAG_STABLE_WRITES, &(q)->queue_flags) #define blk_queue_io_stat(q) test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags) +#define blk_queue_passthrough_stat(q) \ + test_bit(QUEUE_FLAG_IOSTATS_PASSTHROUGH, &(q)->queue_flags) #define blk_queue_precise_io_stat(q) \ test_bit(QUEUE_FLAG_PRECISE_IO_STAT, &(q)->queue_flags) #define blk_queue_add_random(q) test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags) #define blk_queue_zone_resetall(q) \ test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags) -- 2.52.0
From: Bart Van Assche <bvanassche@acm.org> mainline inclusion from mainline-v6.13-rc3 commit a6fe7b70513fbf11ffa5e85f7b6ba444497a5a3d category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Make queue_iostats_passthrough_show() report 0/1 in sysfs instead of 0/4. This patch fixes the following sparse warning: block/blk-sysfs.c:266:31: warning: incorrect type in argument 1 (different base types) block/blk-sysfs.c:266:31: expected unsigned long var block/blk-sysfs.c:266:31: got restricted blk_flags_t Cc: Keith Busch <kbusch@kernel.org> Cc: Christoph Hellwig <hch@lst.de> Fixes: 110234da18ab ("block: enable passthrough command statistics") Signed-off-by: Bart Van Assche <bvanassche@acm.org> Link: https://lore.kernel.org/r/20241212212941.1268662-4-bvanassche@acm.org Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/blk-sysfs.c [6.6's blk_queue_passthrough_stat() uses test_bit() and already returns 0/1, so the !! is not required for correctness; it is kept to match the mainline fix.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- block/blk-sysfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 3384e6793c0c..4e4fe61c58ff 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -359,11 +359,11 @@ static ssize_t queue_max_active_zones_show(struct request_queue *q, char *page) } static ssize_t queue_iostats_passthrough_show(struct request_queue *q, char *page) { - return queue_var_show(blk_queue_passthrough_stat(q), page); + return queue_var_show(!!blk_queue_passthrough_stat(q), page); } static ssize_t queue_iostats_passthrough_store(struct request_queue *q, const char *page, size_t count) { -- 2.52.0
From: Keith Busch <kbusch@kernel.org> mainline inclusion from mainline-v7.2-rc1 commit b7f40ab50190e2500c3c297d15e00040dca47feb category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- A user can enable io accounting for passthrough requests, so export the helper that checks if the request should be tracked. This will enable stacking drivers to to report iostats for passthrough workloads. Since the stacking request_queue may not be the one providing the request, the API has to add a parameter for the caller to specify which one to check. Reviewed-by: Nilay Shroff <nilay@linux.ibm.com> Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com> Signed-off-by: Keith Busch <kbusch@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/blk.h include/linux/blk-mq.h [Mainline moves blk_rq_passthrough_stats() from block/blk-mq.c to include/linux/blk-mq.h. In this backport it starts in block/blk.h, so removes it from block/blk.h and adds it to include/linux/blk-mq.h with the new queue argument. The caller is blk_do_io_stat() in block/blk.h, which passes rq->q.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- block/blk.h | 33 ++------------------------------- include/linux/blk-mq.h | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 31 deletions(-) diff --git a/block/blk.h b/block/blk.h index 9ab353171702..949257c0ffdc 100644 --- a/block/blk.h +++ b/block/blk.h @@ -358,50 +358,21 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio); enum elv_merge blk_try_merge(struct request *rq, struct bio *bio); void blk_set_default_limits(struct queue_limits *lim); int blk_dev_init(void); -static inline bool blk_rq_passthrough_stats(struct request *req) -{ - struct bio *bio = req->bio; - - if (!blk_queue_passthrough_stat(req->q)) - return false; - - /* Requests without a bio do not transfer data. */ - if (!bio) - return false; - - /* - * Stats are accumulated in the bdev, so must have one attached to a - * bio to track stats. Most drivers do not set the bdev for passthrough - * requests, but nvme is one that will set it. - */ - if (!bio->bi_bdev) - return false; - - /* - * We don't know what a passthrough command does, but we know the - * payload size and data direction. Ensuring the size is aligned to the - * block size filters out most commands with payloads that don't - * represent sector access. - */ - if (blk_rq_bytes(req) & (bdev_logical_block_size(bio->bi_bdev) - 1)) - return false; - return true; -} - /* * Contribute to IO statistics IFF: * * a) it's attached to a gendisk, and * b) the queue had IO stats enabled when this request was started */ static inline bool blk_do_io_stat(struct request *rq) { return (rq->rq_flags & RQF_IO_STAT) && - (!blk_rq_is_passthrough(rq) || blk_rq_passthrough_stats(rq)); + (!blk_rq_is_passthrough(rq) || + blk_rq_passthrough_stats(rq, rq->q)); } void update_io_ticks(struct block_device *part, unsigned long now, bool end); unsigned int part_in_flight(struct block_device *part); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 8ab506d96604..69abd003ad38 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1324,6 +1324,47 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) { return true; } #endif /* CONFIG_BLK_DEV_ZONED */ +/** + * blk_rq_passthrough_stats - check if this request should account stats + * @rq: request to check + * @q: the queue accumulating the stats + * + * Note, @q does not necessarily need to be the request_queue that provides + * @rq. + * + * Return: true if stats should be accounted. + */ +static inline bool blk_rq_passthrough_stats(struct request *rq, + struct request_queue *q) +{ + struct bio *bio = rq->bio; + + if (!blk_queue_passthrough_stat(q)) + return false; + + /* Requests without a bio do not transfer data. */ + if (!bio) + return false; + + /* + * Stats are accumulated in the bdev, so must have one attached to a + * bio to track stats. Most drivers do not set the bdev for passthrough + * requests, but nvme is one that will set it. + */ + if (!bio->bi_bdev) + return false; + + /* + * We don't know what a passthrough command does, but we know the + * payload size and data direction. Ensuring the size is aligned to the + * block size filters out most commands with payloads that don't + * represent sector access. + */ + if (blk_rq_bytes(rq) & (bdev_logical_block_size(bio->bi_bdev) - 1)) + return false; + return true; +} + #endif /* BLK_MQ_H */ -- 2.52.0
From: Keith Busch <kbusch@kernel.org> mainline inclusion from mainline-v7.2-rc1 commit 7d6eb455ecf0f95c54257ae372ac1272cff834e3 category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Don't skip the io accounting for passthrough commands if the user enabled tracking these. Reviewed-by: Nilay Shroff <nilay@linux.ibm.com> Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com> Signed-off-by: Keith Busch <kbusch@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- drivers/nvme/host/ioctl.c | 9 +++++++++ drivers/nvme/host/multipath.c | 5 ++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 5ae3b70d29b7..1ad2f96c37a3 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -184,12 +184,21 @@ static int nvme_finish_user_metadata(struct request *req, void __user *ubuf, static struct request *nvme_alloc_user_request(struct request_queue *q, struct nvme_command *cmd, blk_opf_t rq_flags, blk_mq_req_flags_t blk_flags) { + struct nvme_ns *ns = q->queuedata; struct request *req; + /* + * The NVME_MPATH flag is set only for IO commands sent to a namespace + * with a multipath enabled head. The request is not eligible for + * failover as passthrough requests also append REQ_FAILFAST_DRIVER. + */ + if (ns && nvme_ns_head_multipath(ns->head)) + rq_flags |= REQ_NVME_MPATH; + req = blk_mq_alloc_request(q, nvme_req_op(cmd) | rq_flags, blk_flags); if (IS_ERR(req)) return req; nvme_init_request(req, cmd); nvme_req(req)->flags |= NVME_REQ_USERCMD; diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index fdd64ce6dedb..09cc0f880e83 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -135,13 +135,16 @@ void nvme_mpath_start_request(struct request *rq) !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE)) { atomic_inc(&ns->ctrl->nr_active); nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE; } - if (!blk_queue_io_stat(disk->queue) || blk_rq_is_passthrough(rq) || + if (!blk_queue_io_stat(disk->queue) || (nvme_req(rq)->flags & NVME_MPATH_IO_STATS)) return; + if (blk_rq_is_passthrough(rq) && + !blk_rq_passthrough_stats(rq, disk->queue)) + return; nvme_req(rq)->flags |= NVME_MPATH_IO_STATS; nvme_req(rq)->start_time = bdev_start_io_acct(disk->part0, req_op(rq), jiffies); } -- 2.52.0
From: Chaitanya Kulkarni <kch@nvidia.com> mainline inclusion from mainline-v7.2-rc1 commit 7882834048f110931275357db60dccff906dc96a category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- BLK_FEAT_NOWAIT and BLK_FEAT_POLL are cleared in blk_stack_limits() when an underlying device does not support them. Apply the same treatment to BLK_FEAT_PCI_P2PDMA: stacking drivers set it unconditionally and rely on the core to clear it whenever a non-supporting member device is stacked. Tested-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com> Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> Tested=by: Pranjal Shrivastava <praan@google.com> Link: https://patch.msgid.link/20260513185153.95552-2-kch@nvidia.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/blk-settings.c [OLK-6.6 has no queue_limits.features. PCI P2PDMA support is tracked as QUEUE_FLAG_PCI_P2PDMA in struct request_queue, which blk_stack_limits() cannot access because it only receives struct queue_limits. Clear the queue flag in disk_stack_limits() instead, which has both the stacking queue and the member queue.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- block/blk-settings.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-settings.c b/block/blk-settings.c index 66c4d8a6399c..23e3b6b0c266 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -799,10 +799,13 @@ void disk_stack_limits(struct gendisk *disk, struct block_device *bdev, if (blk_stack_limits(&t->limits, &bdev_get_queue(bdev)->limits, get_start_sect(bdev) + (offset >> 9)) < 0) pr_notice("%s: Warning: Device %pg is misaligned\n", disk->disk_name, bdev); + if (!blk_queue_pci_p2pdma(bdev_get_queue(bdev))) + blk_queue_flag_clear(QUEUE_FLAG_PCI_P2PDMA, t); + disk_update_readahead(disk); } EXPORT_SYMBOL(disk_stack_limits); /** -- 2.52.0
From: Kiran Kumar Modukuri <kmodukuri@nvidia.com> mainline inclusion from mainline-v7.2-rc1 commit 02666132403aec8fc5de315002894f713ef17dbc category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- MD RAID does not propagate BLK_FEAT_PCI_P2PDMA from member devices to the RAID device, preventing peer-to-peer DMA through the RAID layer even when all underlying devices support it. Enable BLK_FEAT_PCI_P2PDMA unconditionally in raid0, raid1 and raid10 personalities during queue limits setup. blk_stack_limits() clears it automatically if any member device lacks support, consistent with how BLK_FEAT_NOWAIT and BLK_FEAT_POLL are handled in the block core. Parity RAID personalities (raid4/5/6) are excluded because they require CPU access to data pages for parity computation, which is incompatible with P2P mappings. Tested with RAID0/1/10 arrays containing multiple NVMe devices with P2PDMA support, confirming that peer-to-peer transfers work correctly through the RAID layer. Tested-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Reviewed-by: Xiao Ni <xni@redhat.com> Signed-off-by: Kiran Kumar Modukuri <kmodukuri@nvidia.com> Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> Reviewed-by: Yu Kuai <yukuai@fygo.io> Tested=by: Pranjal Shrivastava <praan@google.com> Link: https://patch.msgid.link/20260513185153.95552-3-kch@nvidia.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: drivers/md/raid0.c drivers/md/raid1.c drivers/md/raid10.c [OLK-6.6 has no raid*_set_limits() helpers and no queue_limits.features. The personalities set their limits inline in raid*_run() and stack member limits with disk_stack_limits(). Set QUEUE_FLAG_PCI_P2PDMA in those inline limit blocks before the disk_stack_limits() loop, so the clear added by the preceding block patch still enforces the all-member rule.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- drivers/md/raid0.c | 2 ++ drivers/md/raid1.c | 1 + drivers/md/raid10.c | 1 + 3 files changed, 4 insertions(+) diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c index c97daac92439..dfa93be8415a 100644 --- a/drivers/md/raid0.c +++ b/drivers/md/raid0.c @@ -402,10 +402,12 @@ static int raid0_run(struct mddev *mddev) blk_queue_io_min(mddev->queue, mddev->chunk_sectors << 9); blk_queue_io_opt(mddev->queue, (mddev->chunk_sectors << 9) * mddev->raid_disks); + blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, mddev->queue); + rdev_for_each(rdev, mddev) { disk_stack_limits(mddev->gendisk, rdev->bdev, rdev->data_offset << 9); } } diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index e045073b09f2..ce7092b57238 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -3218,10 +3218,11 @@ static int raid1_run(struct mddev *mddev) if (IS_ERR(conf)) return PTR_ERR(conf); if (!mddev_is_dm(mddev)) { blk_queue_max_write_zeroes_sectors(mddev->queue, 0); + blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, mddev->queue); rdev_for_each(rdev, mddev) disk_stack_limits(mddev->gendisk, rdev->bdev, rdev->data_offset << 9); } diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index e8ce8cdd7fce..d267a02af348 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4006,10 +4006,11 @@ static int raid10_run(struct mddev *mddev) if (!mddev_is_dm(conf->mddev)) { blk_queue_max_write_zeroes_sectors(mddev->queue, 0); blk_queue_io_min(mddev->queue, mddev->chunk_sectors << 9); raid10_set_io_opt(conf); + blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, mddev->queue); } rdev_for_each(rdev, mddev) { long long diff; -- 2.52.0
From: Kiran Kumar Modukuri <kmodukuri@nvidia.com> mainline inclusion from mainline-v7.2-rc1 commit fb0eeeed91f3236133383445fee5cc8f20330e6e category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- NVMe multipath does not expose BLK_FEAT_PCI_P2PDMA on the head disk even when all underlying controllers support it. Set BLK_FEAT_PCI_P2PDMA unconditionally in nvme_mpath_alloc_disk() alongside the other features. nvme_update_ns_info_block() already calls queue_limits_stack_bdev() to stack each path's limits onto the head disk, which routes through blk_stack_limits(). The core now clears BLK_FEAT_PCI_P2PDMA automatically if any path (e.g., FC) does not support it, consistent with how BLK_FEAT_NOWAIT and BLK_FEAT_POLL are handled. Tested-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com> Signed-off-by: Kiran Kumar Modukuri <kmodukuri@nvidia.com> Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> Tested=by: Pranjal Shrivastava <praan@google.com> Link: https://patch.msgid.link/20260513185153.95552-4-kch@nvidia.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: drivers/nvme/host/multipath.c drivers/nvme/host/core.c [OLK-6.6 has no queue_limits.features; the head disk capability is expressed as QUEUE_FLAG_PCI_P2PDMA. The path limits are stacked with blk_stack_limits() directly rather than through disk_stack_limits(), so the generic clear does not cover this path. Set the queue flag in nvme_mpath_alloc_disk() and clear it after the blk_stack_limits() calls in nvme_update_ns_info_generic() and nvme_update_ns_info_block() when the path queue does not advertise P2P.] Signed-off-by: Zizhi Wo <wozizhi@huawei.com> Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- drivers/nvme/host/core.c | 6 ++++++ drivers/nvme/host/multipath.c | 1 + 2 files changed, 7 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 317ce71b7904..d049b55efd03 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2052,10 +2052,13 @@ static int nvme_update_ns_info_generic(struct nvme_ns *ns, blk_mq_freeze_queue(ns->head->disk->queue); set_disk_ro(ns->head->disk, nvme_ns_is_readonly(ns, info)); nvme_mpath_revalidate_paths(ns); blk_stack_limits(&ns->head->disk->queue->limits, &ns->queue->limits, 0); + if (!blk_queue_pci_p2pdma(ns->queue)) + blk_queue_flag_clear(QUEUE_FLAG_PCI_P2PDMA, + ns->head->disk->queue); ns->head->disk->flags |= GENHD_FL_HIDDEN; blk_mq_unfreeze_queue(ns->head->disk->queue); } /* Hide the block-interface for these devices */ @@ -2127,10 +2130,13 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, nvme_update_disk_info(ns->head->disk, ns, id); set_disk_ro(ns->head->disk, nvme_ns_is_readonly(ns, info)); nvme_mpath_revalidate_paths(ns); blk_stack_limits(&ns->head->disk->queue->limits, &ns->queue->limits, 0); + if (!blk_queue_pci_p2pdma(ns->queue)) + blk_queue_flag_clear(QUEUE_FLAG_PCI_P2PDMA, + ns->head->disk->queue); disk_update_readahead(ns->head->disk); blk_mq_unfreeze_queue(ns->head->disk->queue); } ret = 0; diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 09cc0f880e83..5faf599cbea2 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -650,10 +650,11 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head) ctrl->subsys->instance, head->instance); blk_queue_flag_set(QUEUE_FLAG_NONROT, head->disk->queue); blk_queue_flag_set(QUEUE_FLAG_NOWAIT, head->disk->queue); blk_queue_flag_set(QUEUE_FLAG_IO_STAT, head->disk->queue); + blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, head->disk->queue); /* * This assumes all controllers that refer to a namespace either * support poll queues or not. That is not a strict guarantee, * but if the assumption is wrong the effect is only suboptimal * performance but not correctness problem. -- 2.52.0
From: hotforest <hotforest@gmail.com> hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 -------------------------------- Advertise P2PDMA only when every target in a table remaps bio data without CPU access, and every underlying device advertises P2PDMA. Reevaluate the queue flag on table activation so that mixed or reloaded tables cannot accept unsupported P2PDMA pages. - Define the DM_TARGET_PCI_P2PDMA target feature. A target sets this feature when it passes bio data to its underlying devices without CPU access to the data pages. The feature uses an unused target feature bit and changes no public structure. - Mark the linear and striped targets with this feature. Their data paths remap or split bios. They do not read or transform data pages. - dm_table_set_restrictions() sets QUEUE_FLAG_PCI_P2PDMA on the mapped device queue only when all targets in the table have the feature and all their underlying devices advertise the flag. Otherwise it clears the flag. A mixed table stays unsupported. - Other targets, including dm-crypt and dm-raid, do not set the feature. They keep the flag cleared. Signed-off-by: hotforest <hotforest@gmail.com> Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- drivers/md/dm-linear.c | 3 ++- drivers/md/dm-stripe.c | 3 ++- drivers/md/dm-table.c | 26 ++++++++++++++++++++++++++ include/linux/device-mapper.h | 5 +++++ 4 files changed, 35 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index f4448d520ee9..1cf4812a5614 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -198,11 +198,12 @@ static size_t linear_dax_recovery_write(struct dm_target *ti, pgoff_t pgoff, static struct target_type linear_target = { .name = "linear", .version = {1, 4, 0}, .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_NOWAIT | - DM_TARGET_ZONED_HM | DM_TARGET_PASSES_CRYPTO, + DM_TARGET_ZONED_HM | DM_TARGET_PASSES_CRYPTO | + DM_TARGET_PCI_P2PDMA, .report_zones = linear_report_zones, .module = THIS_MODULE, .ctr = linear_ctr, .dtr = linear_dtr, .map = linear_map, diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c index efc289c3a573..b56f5478a1e2 100644 --- a/drivers/md/dm-stripe.c +++ b/drivers/md/dm-stripe.c @@ -466,11 +466,12 @@ static void stripe_io_hints(struct dm_target *ti, } static struct target_type stripe_target = { .name = "striped", .version = {1, 6, 0}, - .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_NOWAIT, + .features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_NOWAIT | + DM_TARGET_PCI_P2PDMA, .module = THIS_MODULE, .ctr = stripe_ctr, .dtr = stripe_dtr, .map = stripe_map, .end_io = stripe_end_io, diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index ed0a5e91968d..6dfe6ec08133 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1564,10 +1564,31 @@ static bool dm_table_supports_poll(struct dm_table *t) } return true; } +static int device_not_pci_p2pdma_capable(struct dm_target *ti, + struct dm_dev *dev, sector_t start, sector_t len, void *data) +{ + return !blk_queue_pci_p2pdma(bdev_get_queue(dev->bdev)); +} + +static bool dm_table_supports_pci_p2pdma(struct dm_table *t) +{ + for (unsigned int i = 0; i < t->num_targets; i++) { + struct dm_target *ti = dm_table_get_target(t, i); + + if (!dm_target_supports_pci_p2pdma(ti->type) || + !ti->type->iterate_devices || + ti->type->iterate_devices(ti, + device_not_pci_p2pdma_capable, NULL)) + return false; + } + + return t->num_targets > 0; +} + /* * Check whether a table has no data devices attached using each * target's iterate_devices method. * Returns false if the result is unknown because a target doesn't * support iterate_devices. @@ -1961,10 +1982,15 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, if (dm_table_supports_nowait(t)) blk_queue_flag_set(QUEUE_FLAG_NOWAIT, q); else blk_queue_flag_clear(QUEUE_FLAG_NOWAIT, q); + if (dm_table_supports_pci_p2pdma(t)) + blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, q); + else + blk_queue_flag_clear(QUEUE_FLAG_PCI_P2PDMA, q); + if (!dm_table_supports_discards(t)) { q->limits.max_discard_sectors = 0; q->limits.max_hw_discard_sectors = 0; q->limits.discard_granularity = 0; q->limits.discard_alignment = 0; diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 772ab4d74d94..4dc7d21e1d8d 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -292,10 +292,15 @@ struct target_type { #else #define DM_TARGET_MIXED_ZONED_MODEL 0x00000000 #define dm_target_supports_mixed_zoned_model(type) (false) #endif +/* Target passes bio data to its underlying devices without CPU access. */ +#define DM_TARGET_PCI_P2PDMA 0x00000400 +#define dm_target_supports_pci_p2pdma(type) \ + ((type)->features & DM_TARGET_PCI_P2PDMA) + struct dm_target { struct dm_table *table; struct target_type *type; /* target limits */ -- 2.52.0
hulk inclusion category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/10302 --------------------------------- Reject PCI P2PDMA on dm and md when any lower device has a block integrity profile. A mixed dm table may omit that profile on its top disk, so the P2PDMA client cannot detect the CPU integrity path there. This is the 6.6 queue-flag adaptation of the integrity withhold: dm_table_supports_pci_p2pdma() checks the underlying device integrity profile, and disk_stack_limits() clears QUEUE_FLAG_PCI_P2PDMA for integrity-bearing members so md personalities do not advertise P2P. Signed-off-by: Zizhi Wo <wozizhi@huawei.com> --- block/blk-settings.c | 4 +++- drivers/md/dm-table.c | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/block/blk-settings.c b/block/blk-settings.c index 23e3b6b0c266..f684135c3145 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -12,10 +12,11 @@ #include <linux/gcd.h> #include <linux/lcm.h> #include <linux/jiffies.h> #include <linux/gfp.h> #include <linux/dma-mapping.h> +#include <linux/blk-integrity.h> #include "blk.h" #include "blk-rq-qos.h" #include "blk-wbt.h" @@ -799,11 +800,12 @@ void disk_stack_limits(struct gendisk *disk, struct block_device *bdev, if (blk_stack_limits(&t->limits, &bdev_get_queue(bdev)->limits, get_start_sect(bdev) + (offset >> 9)) < 0) pr_notice("%s: Warning: Device %pg is misaligned\n", disk->disk_name, bdev); - if (!blk_queue_pci_p2pdma(bdev_get_queue(bdev))) + if (!blk_queue_pci_p2pdma(bdev_get_queue(bdev)) || + bdev_get_integrity(bdev)) blk_queue_flag_clear(QUEUE_FLAG_PCI_P2PDMA, t); disk_update_readahead(disk); } EXPORT_SYMBOL(disk_stack_limits); diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index 6dfe6ec08133..2197e6da2148 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1567,11 +1567,12 @@ static bool dm_table_supports_poll(struct dm_table *t) } static int device_not_pci_p2pdma_capable(struct dm_target *ti, struct dm_dev *dev, sector_t start, sector_t len, void *data) { - return !blk_queue_pci_p2pdma(bdev_get_queue(dev->bdev)); + return !blk_queue_pci_p2pdma(bdev_get_queue(dev->bdev)) || + bdev_get_integrity(dev->bdev); } static bool dm_table_supports_pci_p2pdma(struct dm_table *t) { for (unsigned int i = 0; i < t->num_targets; i++) { -- 2.52.0
participants (1)
-
Zizhi Wo