Changes since v1:
* Corrected patch header format.
This patch set enables erofs/fscache/cachefiles to support ondemand mode,
including readahead, domain sharing, failover, etc.
In addition to patches from the open source community, we have made some
compatibility modifications and quality hardening for this feature:
* Added the dynamic switches of erofs_enabled and
cachefiles_ondemand_enabled.
* The structure of the cache file directory is modified to be the same as
that of the mainline to prevent the problem that the existing cache
cannot be found due to kernel upgrade and downgrade.
* Use the same xattr as that in the mainline to prevent cache failure
caused by kernel upgrade or downgrade.
* Various bugfixes...
Al Viro (1):
erofs: fix handling kern_mount() failure
Baokun Li (29):
openeuler_defconfig: enable erofs ondemand for x86 and arm64
fscache: fix reference count leakage during abort init
fscache: fix assertion failure in fscache_put_object()
erofs: fix lockdep false positives on initializing erofs_pseudo_mnt
erofs: remove erofs_fscache_netfs
fscache: rename new_location to new_version
cachefiles: use mainline xattr in ondemand mode
cachefiles: remove err_put_fd tag in cachefiles_ondemand_daemon_read()
cachefiles: fix slab-use-after-free in cachefiles_ondemand_get_fd()
cachefiles: fix slab-use-after-free in
cachefiles_ondemand_daemon_read()
cachefiles: add output string to cachefiles_obj_[get|put]_ondemand_fd
cachefiles: add consistency check for copen/cread
cachefiles: stop sending new request when dropping object
cachefiles: flush all requests for the object that is being dropped
fscache: limit fscache_object_max_active to avoid blocking
cachefiles: add spin_lock for cachefiles_ondemand_info
cachefiles: never get a new anon fd if ondemand_id is valid
cachefiles: make on-demand read killable
fscache: set the default value of object_max_active to 256
cachefiles: flush all requests after setting CACHEFILES_DEAD
cachefiles: call cachefiles_ondemand_init_object() out of dir inode
lock
cachefiles: defer exposing anon_fd until after copy_to_user() succeeds
cachefiles: disallow to complete open requests with uninitialised
ondemand_id
cachefiles: clear FSCACHE_COOKIE_NO_DATA_YET for the new ondemand
object
erofs: correct the blknr/blkoff calculation in erofs_read_raw_page()
fscache: fix op leak due to abort init after parent ready
cachefiles: cyclic allocation of msg_id to avoid reuse
fscache: fix assertion failure in cachefiles_put_object()
cachefiles: prefault in user pages to aovid ABBA deadlock
David Howells (1):
cachefiles, erofs: Fix NULL deref in when cachefiles is not doing
ondemand-mode
Dawei Li (1):
erofs: protect s_inodes with s_inode_list_lock for fscache
Gao Xiang (26):
erofs: clean up file headers & footers
erofs: introduce chunk-based file on-disk format
erofs: support reading chunk-based uncompressed files
erofs: fix double free of 'copied'
erofs: introduce erofs_sb_has_xxx() helpers
erofs: decouple basic mount options from fs_context
erofs: add multiple device support
erofs: clean up erofs_map_blocks tracepoints
erofs: introduce meta buffer operations
erofs: use meta buffers for inode operations
erofs: use meta buffers for super operations
erofs: use meta buffers for xattr operations
erofs: use meta buffers for zmap operations
erofs: fix misbehavior of unsupported chunk format check
erofs: add fscache mode check helper
erofs: register fscache volume
erofs: add fscache context helper functions
erofs: add anonymous inode caching metadata for data blobs
erofs: register fscache context for primary data blob
erofs: register fscache context for extra data blobs
erofs: implement fscache-based metadata read
erofs: implement fscache-based data read for non-inline layout
erofs: implement fscache-based data read for inline layout
erofs: add 'fsid' mount option
erofs: scan devices from device table
erofs: fix order >= MAX_ORDER warning due to crafted negative i_size
Hou Tao (2):
erofs: check the uniqueness of fsid in shared domain in advance
cachefiles: flush ondemand_object_worker during clean object
Jeffle Xu (9):
erofs: use meta buffers for erofs_read_superblock()
cachefiles: notify the user daemon when looking up cookie
cachefiles: unbind cachefiles gracefully in on-demand mode
cachefiles: notify the user daemon when withdrawing cookie
cachefiles: implement on-demand read
cachefiles: enable on-demand read mode
anolis: cachefiles: skip check for n_children in on-demand mode
erofs: make erofs_map_blocks() generally available
erofs: leave compressed inodes unsupported in fscache mode for now
Jia Zhu (12):
cachefiles: narrow the scope of flushed requests when releasing fd
erofs: use kill_anon_super() to kill super in fscache mode
erofs: code clean up for fscache
erofs: introduce fscache-based domain
erofs: introduce a pseudo mnt to manage shared cookies
erofs: Support sharing cookies in the same domain
erofs: introduce 'domain_id' mount option
anolis: cachefiles: introduce object ondemand state
anolis: cachefiles: extract ondemand info field from cachefiles_object
anolis: cachefiles: resend an open request if the read request's
object is closed
anolis: cachefiles: narrow the scope of triggering EPOLLIN events in
ondemand mode
anolis: cachefiles: add restore command to recover inflight ondemand
read requests
Jingbo Xu (18):
anolis: cachefiles: replace BUG_ON() with WARN_ON()
anolis: cachefiles: fix volume key setup for cachefiles_open
anolis: cachefiles: refactor cachefiles_ondemand_daemon_read()
anolis: erofs: fix the name of erofs_fscache_super_index_def
anolis: cachefiles: maintain a file descriptor to the backing file
anolis: fscache,cachefiles: add fscache_prepare_read() helper
anolis: erofs: implement fscache-based data readahead
erofs: fix use-after-free of fsid and domain_id string
erofs: remove unused EROFS_GET_BLOCKS_RAW flag
anolis: cachefiles: optimize on-demand IO path with buffer IO
anolis: fscache: export fscache_object_wq
anolis: cachefiles: reset object->private to NULL when it's freed
anolis: cachefiles: add missing lock protection when polling
anolis: cachefiles: fix potential NULL in error path
erofs: relinquish volume with mutex held
erofs: maintain cookies of share domain in self-contained list
erofs: remove unused device mapping in meta routine
erofs: unify anonymous inodes for blob
Mikulas Patocka (1):
wait_on_bit: add an acquire memory barrier
Sun Ke (1):
cachefiles: fix error return code in cachefiles_ondemand_copen()
Xin Yin (1):
cachefiles: make on-demand request distribution fairer
Yu Kuai (10):
fscache: add new helper to determine cachefile location
fscache: generate new key_hash for new location
cachefiles: factor out helper to generate acc from
cachefiles_cook_key()
cachefiles: factor out helper to generate csum from
cachefiles_cook_key()
cachefiles: use volume key directly in cachefiles_cook_key() for new
location
cachefiles: skip acc in cachefiles_cook_key() for new location
cachefiles: skip volum csum cachefiles_cook_key() for new location
cachefiles: use key_hash as csum for new location
cachefiles: handle the unprintable case for new location in
cachefiles_cook_key()
erofs: switch to use new location
Yue Hu (1):
erofs: don't use erofs_map_blocks() any more
Zizhi Wo (20):
erofs: fix page unlock advance during readahead
erofs: add erofs switch to better control it
erofs: add erofs_ondemand switch
fscache: fix kernel BUG at __fscache_read_or_alloc_page
fscache: add a waiting mechanism when duplicate cookies are detected
fscache: Fix trace UAF in fscache_cookie_put()
cachefiles: Fix NULL pointer dereference in object->file
cachefiles: Restrict monitor calls to read_page
cachefiles: Fix cookie reference count leakage error
fscache: Add the unhash_cookie mechanism to fscache_drop_object()
cachefiles: Add restrictions to cachefiles_daemon_cull()
cachefiles: Change the mark inactive sequence in erofs ondemand mode
cachefiles: Set object to close if ondemand_id < 0 in copen
s390: provide arch_test_bit_acquire() for architecture s390
wait_on_bit: Add wait_on_bit_acquire() to provide memory barrier
fscache: add a memory barrier for FSCACHE_COOKIE_LOOKING_UP
fscache/cachefiles: add a memory barrier for waking and waiting
fscache/cachefiles: add a memory barrier for page_write
fscache: modify fscache_hash_cookie() to enhance security
cachefiles: modify inappropriate error return value in
cachefiles_daemon_secctx
Documentation/atomic_bitops.txt | 10 +-
Documentation/filesystems/erofs.rst | 28 +-
arch/arm64/configs/openeuler_defconfig | 9 +-
arch/s390/include/asm/bitops.h | 7 +
arch/x86/configs/openeuler_defconfig | 9 +-
arch/x86/include/asm/bitops.h | 21 +
fs/Makefile | 2 +
fs/cachefiles/Kconfig | 12 +
fs/cachefiles/Makefile | 1 +
fs/cachefiles/bind.c | 28 +-
fs/cachefiles/daemon.c | 134 +++-
fs/cachefiles/interface.c | 34 +-
fs/cachefiles/internal.h | 132 ++-
fs/cachefiles/key.c | 167 +++-
fs/cachefiles/namei.c | 45 +-
fs/cachefiles/ondemand.c | 752 ++++++++++++++++++
fs/cachefiles/rdwr.c | 201 ++++-
fs/cachefiles/xattr.c | 179 ++++-
fs/erofs/Kconfig | 35 +-
fs/erofs/Makefile | 2 +-
fs/erofs/compress.h | 2 -
fs/erofs/data.c | 259 ++++--
fs/erofs/decompressor.c | 5 +-
fs/erofs/dir.c | 2 -
fs/erofs/erofs_fs.h | 68 +-
fs/erofs/fscache.c | 568 +++++++++++++
fs/erofs/inode.c | 98 ++-
fs/erofs/internal.h | 161 +++-
fs/erofs/namei.c | 2 -
fs/erofs/super.c | 353 ++++++--
fs/erofs/tagptr.h | 3 -
fs/erofs/utils.c | 2 -
fs/erofs/xattr.c | 141 +---
fs/erofs/xattr.h | 2 -
fs/erofs/zdata.c | 35 +-
fs/erofs/zdata.h | 1 -
fs/erofs/zmap.c | 48 +-
fs/erofs/zpvec.h | 2 -
fs/fs_ctl.c | 43 +
fs/fscache/cookie.c | 149 ++--
fs/fscache/internal.h | 1 +
fs/fscache/main.c | 17 +-
fs/fscache/object.c | 39 +-
fs/fscache/page.c | 90 ++-
.../bitops/instrumented-non-atomic.h | 12 +
include/asm-generic/bitops/non-atomic.h | 14 +
include/linux/fs.h | 17 +
include/linux/fscache-cache.h | 8 +
include/linux/fscache.h | 60 +-
include/linux/wait_bit.h | 25 +
include/trace/events/cachefiles.h | 10 +-
include/trace/events/erofs.h | 7 +-
include/uapi/linux/cachefiles.h | 69 ++
kernel/sched/wait_bit.c | 40 +
lib/radix-tree.c | 1 +
55 files changed, 3614 insertions(+), 548 deletions(-)
create mode 100644 fs/cachefiles/ondemand.c
create mode 100644 fs/erofs/fscache.c
create mode 100644 fs/fs_ctl.c
create mode 100644 include/uapi/linux/cachefiles.h
--
2.46.1