[PATCH OLK-5.10 00/14] try to backport "block: fix rq-qos breakage from skipping rq_qos_done_bio"
Christoph Hellwig (9): [Backport] block: reuse BIO_INLINE_VECS for integrity bvecs [Backport] block: move struct biovec_slab to bio.c [Backport] block: factor out a bvec_alloc_gfp helper [Backport] block: streamline bvec_alloc [Backport] block: remove the 1 and 4 vec bvec_slabs entries [Backport] block: turn the nr_iovecs argument to bio_alloc* into an unsigned short [Backport] block: mark the bio as cloned in bio_iov_bvec_set [Backport] md/raid10: remove dead code in reshape_request [Backport] block: use bi_max_vecs to find the bvec pool Jens Axboe (2): [Backport] block: add a BUILD_BUG_ON() for adding more bio flags than we have space [Backport] block: only mark bio as tracked if it really is tracked Li Lingfeng (2): block: fix kabi broken block: fix kabi broken due to removing bip_slab Tejun Heo (1): [Backport] block: fix rq-qos breakage from skipping rq_qos_done_bio() block/blk-rq-qos.h | 23 +++-- block/blk.h | 1 + include/linux/bio.h | 13 +-- include/linux/blk_types.h | 32 +----- block/bio-integrity.c | 17 ++-- block/bio.c | 202 ++++++++++++++++---------------------- block/blk-iolatency.c | 2 +- drivers/md/raid10.c | 4 - 8 files changed, 110 insertions(+), 184 deletions(-) -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit dc0b8a57ad7b05036fcb19a5bf0319467597e67a bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- bvec_alloc always uses biovec_slabs, and thus always needs to use the same number of inline vecs. Share a single definition for the data and integrity bvecs. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c block/blk.h [Commit 82b10af05f7a ("[Huawei] blk-io-hierarchy: factor out a new struct bio_hierarchy_data from bio") add inclusion of "blk-io-hierarchy/stats.h"; commit eec716a1c18c ("block: move three bvec helpers declaration into private helper") move declarations of three bvec helpers after blk_freeze_queue.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/blk.h | 1 + block/bio-integrity.c | 6 ++---- block/bio.c | 6 ------ 3 files changed, 3 insertions(+), 10 deletions(-) diff --git a/block/blk.h b/block/blk.h index c86d27d80ba0..ca40bcb0abd5 100644 --- a/block/blk.h +++ b/block/blk.h @@ -52,6 +52,7 @@ void blk_free_flush_queue(struct blk_flush_queue *q); void blk_freeze_queue(struct request_queue *q); +#define BIO_INLINE_VECS 4 static inline bool biovec_phys_mergeable(struct request_queue *q, struct bio_vec *vec1, struct bio_vec *vec2) { diff --git a/block/bio-integrity.c b/block/bio-integrity.c index a5fd04db5ae8..70d4118c173f 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -14,8 +14,6 @@ #include <linux/slab.h> #include "blk.h" -#define BIP_INLINE_VECS 4 - static struct kmem_cache *bip_slab; static struct workqueue_struct *kintegrityd_wq; @@ -63,7 +61,7 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, inline_vecs = nr_vecs; } else { bip = mempool_alloc(&bs->bio_integrity_pool, gfp_mask); - inline_vecs = BIP_INLINE_VECS; + inline_vecs = BIO_INLINE_VECS; } if (unlikely(!bip)) @@ -480,6 +478,6 @@ void __init bio_integrity_init(void) bip_slab = kmem_cache_create("bio_integrity_payload", sizeof(struct bio_integrity_payload) + - sizeof(struct bio_vec) * BIP_INLINE_VECS, + sizeof(struct bio_vec) * BIO_INLINE_VECS, 0, SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); } diff --git a/block/bio.c b/block/bio.c index 43bf0920aa32..26b30cd7be8c 100644 --- a/block/bio.c +++ b/block/bio.c @@ -24,12 +24,6 @@ #include "blk.h" #include "blk-rq-qos.h" -/* - * Test patch to inline a certain number of bi_io_vec's inside the bio - * itself, to shrink a bio data allocation from two mempool calls to one - */ -#define BIO_INLINE_VECS 4 - /* * if you change this list, also change bvec_alloc or things will * break badly! cannot be bigger than what you can fit into an -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit 6ac0b71537e1c14e7532408fe4aae553aa314237 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- struct biovec_slab is only used inside of bio.c, so move it there. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c include/linux/bio.h [Commit 82b10af05f7a ("[Huawei] blk-io-hierarchy: factor out a new struct bio_hierarchy_data from bio") add inclusion of "blk-io-hierarchy/stats.h"; commit dde4fe567523 ("kabi: Add kabi reservation for storage module") add kabi reservation in bio_set.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- include/linux/bio.h | 6 ------ block/bio.c | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index e895c84be0bc..f385ebf4c342 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -706,12 +706,6 @@ struct bio_set { KABI_RESERVE(4) }; -struct biovec_slab { - int nr_vecs; - char *name; - struct kmem_cache *slab; -}; - static inline bool bioset_initialized(struct bio_set *bs) { return bs->bio_slab != NULL; diff --git a/block/bio.c b/block/bio.c index 26b30cd7be8c..83df5e983083 100644 --- a/block/bio.c +++ b/block/bio.c @@ -24,6 +24,12 @@ #include "blk.h" #include "blk-rq-qos.h" +struct biovec_slab { + int nr_vecs; + char *name; + struct kmem_cache *slab; +}; + /* * if you change this list, also change bvec_alloc or things will * break badly! cannot be bigger than what you can fit into an -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit f2c3eb9bb0ef77517976f8be926a77a574da8fe3 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Clean up bvec_alloc a little by factoring out a helper for the gfp_t manipulations. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/bio.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/block/bio.c b/block/bio.c index 83df5e983083..78c567bff06e 100644 --- a/block/bio.c +++ b/block/bio.c @@ -168,6 +168,16 @@ void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned int idx) } } +/* + * Make the first allocation restricted and don't dump info on allocation + * failures, since we'll fall back to the mempool in case of failure. + */ +static inline gfp_t bvec_alloc_gfp(gfp_t gfp) +{ + return (gfp & ~(__GFP_DIRECT_RECLAIM | __GFP_IO)) | + __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; +} + struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, mempool_t *pool) { @@ -208,20 +218,12 @@ struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, bvl = mempool_alloc(pool, gfp_mask); } else { struct biovec_slab *bvs = bvec_slabs + *idx; - gfp_t __gfp_mask = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_IO); - - /* - * Make this allocation restricted and don't dump info on - * allocation failures, since we'll fallback to the mempool - * in case of failure. - */ - __gfp_mask |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; /* * Try a slab allocation. If this fails and __GFP_DIRECT_RECLAIM * is set, retry with the 1-entry mempool */ - bvl = kmem_cache_alloc(bvs->slab, __gfp_mask); + bvl = kmem_cache_alloc(bvs->slab, bvec_alloc_gfp(gfp_mask)); if (unlikely(!bvl && (gfp_mask & __GFP_DIRECT_RECLAIM))) { *idx = BVEC_POOL_MAX; goto fallback; -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit f007a3d66c5480c8dae3fa20a89a06861ef1f5db bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Avoid the pointless goto by trying the slab allocation first and falling through to the mempool. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/bio.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/block/bio.c b/block/bio.c index 78c567bff06e..c20c02f43535 100644 --- a/block/bio.c +++ b/block/bio.c @@ -181,8 +181,6 @@ static inline gfp_t bvec_alloc_gfp(gfp_t gfp) struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, mempool_t *pool) { - struct bio_vec *bvl; - /* * see comment near bvec_array define! */ @@ -210,28 +208,24 @@ struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, } /* - * idx now points to the pool we want to allocate from. only the - * 1-vec entry pool is mempool backed. + * Try a slab allocation first for all smaller allocations. If that + * fails and __GFP_DIRECT_RECLAIM is set retry with the mempool. + * The mempool is sized to handle up to BIO_MAX_PAGES entries. */ - if (*idx == BVEC_POOL_MAX) { -fallback: - bvl = mempool_alloc(pool, gfp_mask); - } else { + if (*idx < BVEC_POOL_MAX) { struct biovec_slab *bvs = bvec_slabs + *idx; + struct bio_vec *bvl; - /* - * Try a slab allocation. If this fails and __GFP_DIRECT_RECLAIM - * is set, retry with the 1-entry mempool - */ bvl = kmem_cache_alloc(bvs->slab, bvec_alloc_gfp(gfp_mask)); - if (unlikely(!bvl && (gfp_mask & __GFP_DIRECT_RECLAIM))) { - *idx = BVEC_POOL_MAX; - goto fallback; + if (likely(bvl) || !(gfp_mask & __GFP_DIRECT_RECLAIM)) { + (*idx)++; + return bvl; } + *idx = BVEC_POOL_MAX; } (*idx)++; - return bvl; + return mempool_alloc(pool, gfp_mask); } void bio_uninit(struct bio *bio) -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit de76fd893074ab2cea132c28ac9efd9d0434215e bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- All bios with up to 4 bvecs use the inline bvecs in the bio itself, so don't bother to define bvec_slabs entries for them. Also decruftify the bvec_slabs definition and initialization while we're at it. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c [Commit 82b10af05f7a ("[Huawei] blk-io-hierarchy: factor out a new struct bio_hierarchy_data from bio") add inclusion of "blk-io-hierarchy/stats.h"; commit 49d1ec8573f7 ("block: manage bio slab cache by xarray") remove bio_slabs/bio_slab_nr/bio_slab_max.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/bio.c | 54 ++++++++++++++++------------------------------------- 1 file changed, 16 insertions(+), 38 deletions(-) diff --git a/block/bio.c b/block/bio.c index c20c02f43535..a574c7d2fbfe 100644 --- a/block/bio.c +++ b/block/bio.c @@ -24,23 +24,17 @@ #include "blk.h" #include "blk-rq-qos.h" -struct biovec_slab { +static struct biovec_slab { int nr_vecs; char *name; struct kmem_cache *slab; +} bvec_slabs[] __read_mostly = { + { .nr_vecs = 16, .name = "biovec-16" }, + { .nr_vecs = 64, .name = "biovec-64" }, + { .nr_vecs = 128, .name = "biovec-128" }, + { .nr_vecs = BIO_MAX_PAGES, .name = "biovec-max" }, }; -/* - * if you change this list, also change bvec_alloc or things will - * break badly! cannot be bigger than what you can fit into an - * unsigned short - */ -#define BV(x, n) { .nr_vecs = x, .name = "biovec-"#n } -static struct biovec_slab bvec_slabs[BVEC_POOL_NR] __read_mostly = { - BV(1, 1), BV(4, 4), BV(16, 16), BV(64, 64), BV(128, 128), BV(BIO_MAX_PAGES, max), -}; -#undef BV - /* * fs_bio_set is the bio_set containing bio and iovec memory pools used by * IO code that does not need private memory pools. @@ -185,12 +179,7 @@ struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, * see comment near bvec_array define! */ switch (nr) { - case 1: - *idx = 0; - break; - case 2 ... 4: - *idx = 1; - break; + /* smaller bios use inline vecs */ case 5 ... 16: *idx = 2; break; @@ -1640,27 +1629,9 @@ int bioset_init_from_src(struct bio_set *bs, struct bio_set *src) } EXPORT_SYMBOL(bioset_init_from_src); -static void __init biovec_init_slabs(void) -{ - int i; - - for (i = 0; i < BVEC_POOL_NR; i++) { - int size; - struct biovec_slab *bvs = bvec_slabs + i; - - if (bvs->nr_vecs <= BIO_INLINE_VECS) { - bvs->slab = NULL; - continue; - } - - size = bvs->nr_vecs * sizeof(struct bio_vec); - bvs->slab = kmem_cache_create(bvs->name, size, 0, - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); - } -} - static int __init init_bio(void) { + int i; bio_slab_max = 2; bio_slab_nr = 0; bio_slabs = kcalloc(bio_slab_max, sizeof(struct bio_slab), @@ -1672,7 +1643,14 @@ static int __init init_bio(void) panic("bio: can't allocate bios\n"); bio_integrity_init(); - biovec_init_slabs(); + + for (i = 0; i < ARRAY_SIZE(bvec_slabs); i++) { + struct biovec_slab *bvs = bvec_slabs + i; + + bvs->slab = kmem_cache_create(bvs->name, + bvs->nr_vecs * sizeof(struct bio_vec), 0, + SLAB_HWCACHE_ALIGN | SLAB_PANIC, NULL); + } if (bioset_init(&fs_bio_set, BIO_POOL_SIZE, 0, BIOSET_NEED_BVECS)) panic("bio: can't allocate bios\n"); -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit 0f2e6ab851ae146c468bc5151c302c6e2473f70a bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- The bi_max_vecs and bi_vcnt fields are defined as unsigned short, so don't allow passing larger values in. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c include/linux/bio.h [Commit 3175199ab0ac ("block: split bio_kmalloc from bio_alloc_bioset") change the comment of bio_alloc_bioset and move bio_kmalloc from include/linux/bio.h to block/bio.c] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- include/linux/bio.h | 6 +++--- block/bio.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index f385ebf4c342..7b0bdedea7fa 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -394,7 +394,7 @@ extern void bioset_exit(struct bio_set *); extern int biovec_init_pool(mempool_t *pool, int pool_entries); extern int bioset_init_from_src(struct bio_set *bs, struct bio_set *src); -extern struct bio *bio_alloc_bioset(gfp_t, unsigned int, struct bio_set *); +extern struct bio *bio_alloc_bioset(gfp_t, unsigned short, struct bio_set *); extern void bio_put(struct bio *); extern void __bio_clone_fast(struct bio *, struct bio *); @@ -402,12 +402,12 @@ extern struct bio *bio_clone_fast(struct bio *, gfp_t, struct bio_set *); extern struct bio_set fs_bio_set; -static inline struct bio *bio_alloc(gfp_t gfp_mask, unsigned int nr_iovecs) +static inline struct bio *bio_alloc(gfp_t gfp_mask, unsigned short nr_iovecs) { return bio_alloc_bioset(gfp_mask, nr_iovecs, &fs_bio_set); } -static inline struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned int nr_iovecs) +static inline struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned short nr_iovecs) { return bio_alloc_bioset(gfp_mask, nr_iovecs, NULL); } diff --git a/block/bio.c b/block/bio.c index a574c7d2fbfe..814c384ad4c3 100644 --- a/block/bio.c +++ b/block/bio.c @@ -419,7 +419,7 @@ static void punt_bios_to_rescuer(struct bio_set *bs) * RETURNS: * Pointer to new bio on success, NULL on failure. */ -struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned int nr_iovecs, +struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned short nr_iovecs, struct bio_set *bs) { gfp_t saved_gfp = gfp_mask; -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit 977be01273844626ddeef4a464b42b99418d76e6 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- bio_iov_bvec_set clones the bio_vecs from the iter, and thus should be treated like a cloned bio in every respect. That also includes not touching bi_max_vecs as that is a property of the bio allocation and not its current payload. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c [Commit c42bca92be92 ("bio: don't copy bvec for direct IO") add bio_iov_bvec_set to initialize bio.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/bio.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index 814c384ad4c3..daa15dcd5f65 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1118,8 +1118,10 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) } } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); - if (is_bvec) + if (is_bvec) { bio_set_flag(bio, BIO_NO_PAGE_REF); + bio_set_flag(bio, BIO_CLONED); + } /* don't account direct I/O as memory stall */ bio_clear_flag(bio, BIO_WORKINGSET); -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit 72b043654ba8b8ce2e0cf3da49247b2db3acb2c1 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- A bio allocated by bio_alloc_bioset comes pre-zeroed, no need to clear random fields. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Song Liu <song@kernel.org> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- drivers/md/raid10.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 0162dbe17cf3..bd99296c7b6a 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4554,10 +4554,6 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, read_bio->bi_private = r10_bio; read_bio->bi_end_io = end_reshape_read; bio_set_op_attrs(read_bio, REQ_OP_READ, 0); - read_bio->bi_flags &= (~0UL << BIO_RESET_BITS); - read_bio->bi_status = 0; - read_bio->bi_vcnt = 0; - read_bio->bi_iter.bi_size = 0; r10_bio->master_bio = read_bio; r10_bio->read_slot = r10_bio->devs[r10_bio->read_slot].devnum; -- 2.39.2
From: Christoph Hellwig <hch@lst.de> mainline inclusion from mainline-v5.12-rc1 commit 7a800a20ae6329e803c5c646b20811a6ae9ca136 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- Instead of encoding of the bvec pool using magic bio flags, just use a helper to find the pool based on the max_vecs value. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: include/linux/bio.h block/bio.c block/blk.h [Commit 3175199ab0ac ("block: split bio_kmalloc from bio_alloc_bioset") modifies the method of determining whether bio_vec needs to be allocated; commit 309dca309fc3 ("block: store a block_device pointer in struct bio") change the comment of __bio_clone_fast; commit c42bca92be92 ("bio: don't copy bvec for direct IO") add WARN_ON_ONCE when "BVEC_POOL_IDX(bio) != 0"; commit 49d1ec8573f7 ("block: manage bio slab cache by xarray") remove bio_slabs/bio_slab_nr/bio_slab_max; commit eec716a1c18c ("block: move three bvec helpers declaration into private helper") move declarations of three bvec helpers after blk_freeze_queue.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- include/linux/bio.h | 6 +-- include/linux/blk_types.h | 29 +----------- block/bio-integrity.c | 11 ++--- block/bio.c | 98 ++++++++++++++++----------------------- 4 files changed, 49 insertions(+), 95 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 7b0bdedea7fa..005b62c67a2c 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -312,7 +312,6 @@ struct bio_integrity_payload { struct bvec_iter bip_iter; - unsigned short bip_slab; /* slab the bip came from */ unsigned short bip_vcnt; /* # of integrity bio_vecs */ unsigned short bip_max_vcnt; /* integrity bio_vec slots */ unsigned short bip_flags; /* control flags */ @@ -466,8 +465,9 @@ static inline void zero_fill_bio(struct bio *bio) zero_fill_bio_iter(bio, bio->bi_iter); } -extern struct bio_vec *bvec_alloc(gfp_t, int, unsigned long *, mempool_t *); -extern void bvec_free(mempool_t *, struct bio_vec *, unsigned int); +extern struct bio_vec *bvec_alloc(mempool_t *pool, unsigned short *nr_vecs, + gfp_t gfp_mask); +extern void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned short nr_vecs); extern unsigned int bvec_nr_vecs(unsigned short idx); extern const char *bio_devname(struct bio *bio, char *buffer); diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index b49d97547009..07ca91863716 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -223,7 +223,7 @@ struct bio { * top bits REQ_OP. Use * accessors. */ - unsigned short bi_flags; /* status, etc and bvec pool number */ + unsigned short bi_flags; /* BIO_* below */ unsigned short bi_ioprio; unsigned short bi_write_hint; blk_status_t bi_status; @@ -310,33 +310,6 @@ enum { BIO_FLAG_LAST }; -/* See BVEC_POOL_OFFSET below before adding new flags */ - -/* - * We support 6 different bvec pools, the last one is magic in that it - * is backed by a mempool. - */ -#define BVEC_POOL_NR 6 -#define BVEC_POOL_MAX (BVEC_POOL_NR - 1) - -/* - * Top 3 bits of bio flags indicate the pool the bvecs came from. We add - * 1 to the actual index so that 0 indicates that there are no bvecs to be - * freed. - */ -#define BVEC_POOL_BITS (3) -#define BVEC_POOL_OFFSET (16 - BVEC_POOL_BITS) -#define BVEC_POOL_IDX(bio) ((bio)->bi_flags >> BVEC_POOL_OFFSET) -#if (1<< BVEC_POOL_BITS) < (BVEC_POOL_NR+1) -# error "BVEC_POOL_BITS is too small" -#endif - -/* - * Flags starting here get preserved by bio_reset() - this includes - * only BVEC_POOL_IDX() - */ -#define BIO_RESET_BITS BVEC_POOL_OFFSET - typedef __u32 __bitwise blk_mq_req_flags_t; /* diff --git a/block/bio-integrity.c b/block/bio-integrity.c index 70d4118c173f..7ce634cffeb5 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -28,7 +28,7 @@ static void __bio_integrity_free(struct bio_set *bs, if (bs && mempool_initialized(&bs->bio_integrity_pool)) { if (bip->bip_vec) bvec_free(&bs->bvec_integrity_pool, bip->bip_vec, - bip->bip_slab); + bip->bip_max_vcnt); mempool_free(bip, &bs->bio_integrity_pool); } else { kfree(bip); @@ -70,14 +70,11 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, memset(bip, 0, sizeof(*bip)); if (nr_vecs > inline_vecs) { - unsigned long idx = 0; - - bip->bip_vec = bvec_alloc(gfp_mask, nr_vecs, &idx, - &bs->bvec_integrity_pool); + bip->bip_max_vcnt = nr_vecs; + bip->bip_vec = bvec_alloc(&bs->bvec_integrity_pool, + &bip->bip_max_vcnt, gfp_mask); if (!bip->bip_vec) goto err; - bip->bip_max_vcnt = bvec_nr_vecs(idx); - bip->bip_slab = idx; } else { bip->bip_vec = bip->bip_inline_vecs; bip->bip_max_vcnt = inline_vecs; diff --git a/block/bio.c b/block/bio.c index daa15dcd5f65..6e1d3b259f29 100644 --- a/block/bio.c +++ b/block/bio.c @@ -35,6 +35,24 @@ static struct biovec_slab { { .nr_vecs = BIO_MAX_PAGES, .name = "biovec-max" }, }; +static struct biovec_slab *biovec_slab(unsigned short nr_vecs) +{ + switch (nr_vecs) { + /* smaller bios use inline vecs */ + case 5 ... 16: + return &bvec_slabs[0]; + case 17 ... 64: + return &bvec_slabs[1]; + case 65 ... 128: + return &bvec_slabs[2]; + case 129 ... BIO_MAX_PAGES: + return &bvec_slabs[3]; + default: + BUG(); + return NULL; + } +} + /* * fs_bio_set is the bio_set containing bio and iovec memory pools used by * IO code that does not need private memory pools. @@ -140,26 +158,14 @@ static void bio_put_slab(struct bio_set *bs) mutex_unlock(&bio_slab_lock); } -unsigned int bvec_nr_vecs(unsigned short idx) -{ - return bvec_slabs[--idx].nr_vecs; -} - -void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned int idx) +void bvec_free(mempool_t *pool, struct bio_vec *bv, unsigned short nr_vecs) { - if (!idx) - return; - idx--; + BIO_BUG_ON(nr_vecs > BIO_MAX_PAGES); - BIO_BUG_ON(idx >= BVEC_POOL_NR); - - if (idx == BVEC_POOL_MAX) { + if (nr_vecs == BIO_MAX_PAGES) mempool_free(bv, pool); - } else { - struct biovec_slab *bvs = bvec_slabs + idx; - - kmem_cache_free(bvs->slab, bv); - } + else if (nr_vecs > BIO_INLINE_VECS) + kmem_cache_free(biovec_slab(nr_vecs)->slab, bv); } /* @@ -172,48 +178,34 @@ static inline gfp_t bvec_alloc_gfp(gfp_t gfp) __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; } -struct bio_vec *bvec_alloc(gfp_t gfp_mask, int nr, unsigned long *idx, - mempool_t *pool) +struct bio_vec *bvec_alloc(mempool_t *pool, unsigned short *nr_vecs, + gfp_t gfp_mask) { + struct biovec_slab *bvs = biovec_slab(*nr_vecs); + + if (WARN_ON_ONCE(!bvs)) + return NULL; + /* - * see comment near bvec_array define! + * Upgrade the nr_vecs request to take full advantage of the allocation. + * We also rely on this in the bvec_free path. */ - switch (nr) { - /* smaller bios use inline vecs */ - case 5 ... 16: - *idx = 2; - break; - case 17 ... 64: - *idx = 3; - break; - case 65 ... 128: - *idx = 4; - break; - case 129 ... BIO_MAX_PAGES: - *idx = 5; - break; - default: - return NULL; - } + *nr_vecs = bvs->nr_vecs; /* * Try a slab allocation first for all smaller allocations. If that * fails and __GFP_DIRECT_RECLAIM is set retry with the mempool. * The mempool is sized to handle up to BIO_MAX_PAGES entries. */ - if (*idx < BVEC_POOL_MAX) { - struct biovec_slab *bvs = bvec_slabs + *idx; + if (*nr_vecs < BIO_MAX_PAGES) { struct bio_vec *bvl; bvl = kmem_cache_alloc(bvs->slab, bvec_alloc_gfp(gfp_mask)); - if (likely(bvl) || !(gfp_mask & __GFP_DIRECT_RECLAIM)) { - (*idx)++; + if (likely(bvl) || !(gfp_mask & __GFP_DIRECT_RECLAIM)) return bvl; - } - *idx = BVEC_POOL_MAX; + *nr_vecs = BIO_MAX_PAGES; } - (*idx)++; return mempool_alloc(pool, gfp_mask); } @@ -240,7 +232,7 @@ static void bio_free(struct bio *bio) bio_uninit(bio); if (bs) { - bvec_free(&bs->bvec_pool, bio->bi_io_vec, BVEC_POOL_IDX(bio)); + bvec_free(&bs->bvec_pool, bio->bi_io_vec, bio->bi_max_vecs); /* * If we have front padding, adjust the bio pointer before freeing @@ -284,12 +276,8 @@ EXPORT_SYMBOL(bio_init); */ void bio_reset(struct bio *bio) { - unsigned long flags = bio->bi_flags & (~0UL << BIO_RESET_BITS); - bio_uninit(bio); - memset(bio, 0, BIO_RESET_BYTES); - bio->bi_flags = flags; atomic_set(&bio->__bi_remaining, 1); } EXPORT_SYMBOL(bio_reset); @@ -486,19 +474,17 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned short nr_iovecs, bio_init(bio, NULL, 0); if (nr_iovecs > inline_vecs) { - unsigned long idx = 0; - bvl = bvec_alloc(gfp_mask, nr_iovecs, &idx, &bs->bvec_pool); + bvl = bvec_alloc(&bs->bvec_pool, &nr_iovecs, gfp_mask); if (!bvl && gfp_mask != saved_gfp) { punt_bios_to_rescuer(bs); gfp_mask = saved_gfp; - bvl = bvec_alloc(gfp_mask, nr_iovecs, &idx, &bs->bvec_pool); + bvl = bvec_alloc(&bs->bvec_pool, &nr_iovecs, gfp_mask); } if (unlikely(!bvl)) goto err_free; - bio->bi_flags |= idx << BVEC_POOL_OFFSET; } else if (nr_iovecs) { bvl = bio->bi_inline_vecs; } @@ -659,7 +645,7 @@ EXPORT_SYMBOL(bio_put); */ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) { - BUG_ON(bio->bi_pool && BVEC_POOL_IDX(bio)); + WARN_ON_ONCE(bio->bi_pool && bio->bi_max_vecs); /* * most users will be overriding ->bi_disk with a new target, @@ -1527,7 +1513,7 @@ EXPORT_SYMBOL_GPL(bio_trim); */ int biovec_init_pool(mempool_t *pool, int pool_entries) { - struct biovec_slab *bp = bvec_slabs + BVEC_POOL_MAX; + struct biovec_slab *bp = bvec_slabs + ARRAY_SIZE(bvec_slabs) - 1; return mempool_init_slab_pool(pool, pool_entries, bp->slab); } @@ -1639,8 +1625,6 @@ static int __init init_bio(void) bio_slabs = kcalloc(bio_slab_max, sizeof(struct bio_slab), GFP_KERNEL); - BUILD_BUG_ON(BIO_FLAG_LAST > BVEC_POOL_OFFSET); - if (!bio_slabs) panic("bio: can't allocate bios\n"); -- 2.39.2
From: Jens Axboe <axboe@kernel.dk> mainline inclusion from mainline-v6.3-rc1 commit a3df2e456c0f9db4cc7f2013eb0beb853945b937 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- We have BIO_FLAG_LAST in the enum for bio specific flags, but it's not used to check that we're not exceeding the size of them. Add such a check. Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c [Commit 49d1ec8573f7 ("block: manage bio slab cache by xarray") remove bio_slabs/bio_slab_nr/bio_slab_max.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/bio.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/bio.c b/block/bio.c index 6e1d3b259f29..efd432594446 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1628,6 +1628,8 @@ static int __init init_bio(void) if (!bio_slabs) panic("bio: can't allocate bios\n"); + BUILD_BUG_ON(BIO_FLAG_LAST > 8 * sizeof_field(struct bio, bi_flags)); + bio_integrity_init(); for (i = 0; i < ARRAY_SIZE(bvec_slabs); i++) { -- 2.39.2
From: Jens Axboe <axboe@kernel.dk> mainline inclusion from mainline-v5.16-rc1 commit 90b8faa0e8de1b02b619fb33f6c6e1e13e7d1d70 bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- We set BIO_TRACKED unconditionally when rq_qos_throttle() is called, even though we may not even have an rq_qos handler. Only mark it as TRACKED if it really is potentially tracked. This saves considerable time for the case where the bio isn't tracked: 2.64% -1.65% [kernel.vmlinux] [k] bio_endio Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/blk-rq-qos.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/block/blk-rq-qos.h b/block/blk-rq-qos.h index 37c59d7d6ba7..8d6e3bbdd8f2 100644 --- a/block/blk-rq-qos.h +++ b/block/blk-rq-qos.h @@ -210,9 +210,10 @@ static inline void rq_qos_throttle(struct request_queue *q, struct bio *bio) * BIO_TRACKED lets controllers know that a bio went through the * normal rq_qos path. */ - bio_set_flag(bio, BIO_TRACKED); - if (q->rq_qos) + if (q->rq_qos) { + bio_set_flag(bio, BIO_TRACKED); __rq_qos_throttle(q->rq_qos, bio); + } } static inline void rq_qos_track(struct request_queue *q, struct request *rq, -- 2.39.2
From: Tejun Heo <tj@kernel.org> mainline inclusion from mainline-v5.18-rc1 commit aa1b46dcdc7baaf5fec0be25782ef24b26aa209e bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- a647a524a467 ("block: don't call rq_qos_ops->done_bio if the bio isn't tracked") made bio_endio() skip rq_qos_done_bio() if BIO_TRACKED is not set. While this fixed a potential oops, it also broke blk-iocost by skipping the done_bio callback for merged bios. Before, whether a bio goes through rq_qos_throttle() or rq_qos_merge(), rq_qos_done_bio() would be called on the bio on completion with BIO_TRACKED distinguishing the former from the latter. rq_qos_done_bio() is not called for bios which wenth through rq_qos_merge(). This royally confuses blk-iocost as the merged bios never finish and are considered perpetually in-flight. One reliably reproducible failure mode is an intermediate cgroup geting stuck active preventing its children from being activated due to the leaf-only rule, leading to loss of control. The following is from resctl-bench protection scenario which emulates isolating a web server like workload from a memory bomb run on an iocost configuration which should yield a reasonable level of protection. # cat /sys/block/nvme2n1/device/model Samsung SSD 970 PRO 512GB # cat /sys/fs/cgroup/io.cost.model 259:0 ctrl=user model=linear rbps=834913556 rseqiops=93622 rrandiops=102913 wbps=618985353 wseqiops=72325 wrandiops=71025 # cat /sys/fs/cgroup/io.cost.qos 259:0 enable=1 ctrl=user rpct=95.00 rlat=18776 wpct=95.00 wlat=8897 min=60.00 max=100.00 # resctl-bench -m 29.6G -r out.json run protection::scenario=mem-hog,loops=1 ... Memory Hog Summary ================== IO Latency: R p50=242u:336u/2.5m p90=794u:1.4m/7.5m p99=2.7m:8.0m/62.5m max=8.0m:36.4m/350m W p50=221u:323u/1.5m p90=709u:1.2m/5.5m p99=1.5m:2.5m/9.5m max=6.9m:35.9m/350m Isolation and Request Latency Impact Distributions: min p01 p05 p10 p25 p50 p75 p90 p95 p99 max mean stdev isol% 15.90 15.90 15.90 40.05 57.24 59.07 60.01 74.63 74.63 90.35 90.35 58.12 15.82 lat-imp% 0 0 0 0 0 4.55 14.68 15.54 233.5 548.1 548.1 53.88 143.6 Result: isol=58.12:15.82% lat_imp=53.88%:143.6 work_csv=100.0% missing=3.96% The isolation result of 58.12% is close to what this device would show without any IO control. Fix it by introducing a new flag BIO_QOS_MERGED to mark merged bios and calling rq_qos_done_bio() on them too. For consistency and clarity, rename BIO_TRACKED to BIO_QOS_THROTTLED. The flag checks are moved into rq_qos_done_bio() so that it's next to the code paths that set the flags. With the patch applied, the above same benchmark shows: # resctl-bench -m 29.6G -r out.json run protection::scenario=mem-hog,loops=1 ... Memory Hog Summary ================== IO Latency: R p50=123u:84.4u/985u p90=322u:256u/2.5m p99=1.6m:1.4m/9.5m max=11.1m:36.0m/350m W p50=429u:274u/995u p90=1.7m:1.3m/4.5m p99=3.4m:2.7m/11.5m max=7.9m:5.9m/26.5m Isolation and Request Latency Impact Distributions: min p01 p05 p10 p25 p50 p75 p90 p95 p99 max mean stdev isol% 84.91 84.91 89.51 90.73 92.31 94.49 96.36 98.04 98.71 100.0 100.0 94.42 2.81 lat-imp% 0 0 0 0 0 2.81 5.73 11.11 13.92 17.53 22.61 4.10 4.68 Result: isol=94.42:2.81% lat_imp=4.10%:4.68 work_csv=58.34% missing=0% Signed-off-by: Tejun Heo <tj@kernel.org> Fixes: a647a524a467 ("block: don't call rq_qos_ops->done_bio if the bio isn't tracked") Cc: stable@vger.kernel.org # v5.15+ Cc: Ming Lei <ming.lei@redhat.com> Cc: Yu Kuai <yukuai3@huawei.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/Yi7rdrzQEHjJLGKB@slm.duckdns.org Signed-off-by: Jens Axboe <axboe@kernel.dk> Conflicts: block/bio.c block/blk-rq-qos.h include/linux/blk_types.h [Commit 309dca309fc3 ("block: store a block_device pointer in struct bio") replace bi_disk with bi_bdev in struct bio; commit 7eeb22198dcc ("[Huawei] blk-io-hierarchy: support hierarchy stats for bio lifetime") add BIO_HAS_DATA/BIO_HIERARCHY_ACCT.] Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- block/blk-rq-qos.h | 20 +++++++++++--------- include/linux/blk_types.h | 3 ++- block/bio.c | 3 +-- block/blk-iolatency.c | 2 +- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/block/blk-rq-qos.h b/block/blk-rq-qos.h index 8d6e3bbdd8f2..31e54f84ac89 100644 --- a/block/blk-rq-qos.h +++ b/block/blk-rq-qos.h @@ -198,20 +198,20 @@ static inline void rq_qos_requeue(struct request_queue *q, struct request *rq) __rq_qos_requeue(q->rq_qos, rq); } -static inline void rq_qos_done_bio(struct request_queue *q, struct bio *bio) +static inline void rq_qos_done_bio(struct bio *bio) { - if (q->rq_qos) - __rq_qos_done_bio(q->rq_qos, bio); + if (bio->bi_disk && (bio_flagged(bio, BIO_QOS_THROTTLED) || + bio_flagged(bio, BIO_QOS_MERGED))) { + struct request_queue *q = bio->bi_disk->queue; + if (q->rq_qos) + __rq_qos_done_bio(q->rq_qos, bio); + } } static inline void rq_qos_throttle(struct request_queue *q, struct bio *bio) { - /* - * BIO_TRACKED lets controllers know that a bio went through the - * normal rq_qos path. - */ if (q->rq_qos) { - bio_set_flag(bio, BIO_TRACKED); + bio_set_flag(bio, BIO_QOS_THROTTLED); __rq_qos_throttle(q->rq_qos, bio); } } @@ -226,8 +226,10 @@ static inline void rq_qos_track(struct request_queue *q, struct request *rq, static inline void rq_qos_merge(struct request_queue *q, struct request *rq, struct bio *bio) { - if (q->rq_qos) + if (q->rq_qos) { + bio_set_flag(bio, BIO_QOS_MERGED); __rq_qos_merge(q->rq_qos, rq, bio); + } } static inline void rq_qos_queue_depth_changed(struct request_queue *q) diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 07ca91863716..adc411d427ff 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -305,7 +305,8 @@ enum { BIO_TRACE_COMPLETION, /* bio_endio() should trace the final completion * of this bio. */ BIO_CGROUP_ACCT, /* has been accounted to a cgroup */ - BIO_TRACKED, /* set if bio goes through the rq_qos path */ + BIO_QOS_THROTTLED, /* bio went through rq_qos throttle path */ + BIO_QOS_MERGED, /* but went through rq_qos merge path */ BIO_ASYNC, /* has been dispatched asynchronously */ BIO_FLAG_LAST }; diff --git a/block/bio.c b/block/bio.c index efd432594446..e5e180436f47 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1409,8 +1409,7 @@ void bio_endio(struct bio *bio) if (!bio_integrity_endio(bio)) return; - if (bio->bi_disk && bio_flagged(bio, BIO_TRACKED)) - rq_qos_done_bio(bio->bi_disk->queue, bio); + rq_qos_done_bio(bio); /* * Need to have a real endio function for chained bios, otherwise diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c index 9811ee74b69f..3e42f7d84588 100644 --- a/block/blk-iolatency.c +++ b/block/blk-iolatency.c @@ -601,7 +601,7 @@ static void blkcg_iolatency_done_bio(struct rq_qos *rqos, struct bio *bio) int inflight = 0; blkg = bio->bi_blkg; - if (!blkg || !bio_flagged(bio, BIO_TRACKED)) + if (!blkg || !bio_flagged(bio, BIO_QOS_THROTTLED)) return; iolat = blkg_to_lat(bio->bi_blkg); -- 2.39.2
From: Li Lingfeng <lilingfeng3@huawei.com> hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IB7FJU -------------------------------- Commit 0f2e6ab851ae ("block: turn the nr_iovecs argument to bio_alloc* into an unsigned short") change the type of parameter of bio_alloc_bioset. Change it back to fix kabi broken since it does't matter. Fixes: 0f2e6ab851ae ("block: turn the nr_iovecs argument to bio_alloc* into an unsigned short") Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- include/linux/bio.h | 6 +++--- block/bio.c | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index 005b62c67a2c..c571ed1d7168 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -393,7 +393,7 @@ extern void bioset_exit(struct bio_set *); extern int biovec_init_pool(mempool_t *pool, int pool_entries); extern int bioset_init_from_src(struct bio_set *bs, struct bio_set *src); -extern struct bio *bio_alloc_bioset(gfp_t, unsigned short, struct bio_set *); +extern struct bio *bio_alloc_bioset(gfp_t, unsigned int, struct bio_set *); extern void bio_put(struct bio *); extern void __bio_clone_fast(struct bio *, struct bio *); @@ -401,12 +401,12 @@ extern struct bio *bio_clone_fast(struct bio *, gfp_t, struct bio_set *); extern struct bio_set fs_bio_set; -static inline struct bio *bio_alloc(gfp_t gfp_mask, unsigned short nr_iovecs) +static inline struct bio *bio_alloc(gfp_t gfp_mask, unsigned int nr_iovecs) { return bio_alloc_bioset(gfp_mask, nr_iovecs, &fs_bio_set); } -static inline struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned short nr_iovecs) +static inline struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned int nr_iovecs) { return bio_alloc_bioset(gfp_mask, nr_iovecs, NULL); } diff --git a/block/bio.c b/block/bio.c index e5e180436f47..8168636d7ad3 100644 --- a/block/bio.c +++ b/block/bio.c @@ -407,7 +407,7 @@ static void punt_bios_to_rescuer(struct bio_set *bs) * RETURNS: * Pointer to new bio on success, NULL on failure. */ -struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned short nr_iovecs, +struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned int nr_iovecs_int, struct bio_set *bs) { gfp_t saved_gfp = gfp_mask; @@ -416,6 +416,7 @@ struct bio *bio_alloc_bioset(gfp_t gfp_mask, unsigned short nr_iovecs, struct bio_vec *bvl = NULL; struct bio *bio; void *p; + unsigned short nr_iovecs = (unsigned short)nr_iovecs_int; if (!bs) { if (nr_iovecs > UIO_MAXIOV) -- 2.39.2
From: Li Lingfeng <lilingfeng3@huawei.com> hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IB6YDK -------------------------------- Commit 7a800a20ae63 ("block: use bi_max_vecs to find the bvec pool") remove bip_slab from struct bio_integrity_payload and result in kabi broken. Fix it by add bip_slab back. Fixes: 7a800a20ae63 ("block: use bi_max_vecs to find the bvec pool") Signed-off-by: Li Lingfeng <lilingfeng3@huawei.com> Signed-off-by: Li Nan <linan122@huawei.com> --- include/linux/bio.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/bio.h b/include/linux/bio.h index c571ed1d7168..85bb3e5b313d 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -312,6 +312,7 @@ struct bio_integrity_payload { struct bvec_iter bip_iter; + KABI_DEPRECATE(unsigned short, bip_slab) unsigned short bip_vcnt; /* # of integrity bio_vecs */ unsigned short bip_max_vcnt; /* integrity bio_vec slots */ unsigned short bip_flags; /* control flags */ -- 2.39.2
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://gitee.com/openeuler/kernel/pulls/15053 邮件列表地址：https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/J... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://gitee.com/openeuler/kernel/pulls/15053 Mailing list address: https://mailweb.openeuler.org/hyperkitty/list/kernel@openeuler.org/message/J...
participants (2)
-
Li Nan -
patchwork bot