Recently, when install package in a docker which almost reached its memory limit, the installer has no respond severely for more than 15 minutes. During this period, I/O stays high(~1G/s) and influence the whole machine. I've constructed a use case as follows:
1. create a docker:
$ cat test.sh #!/bin/bash
docker rm centos7 --force
docker create --name centos7 --memory 4G --memory-swap 6G centos:7 /usr/sbin/init docker start centos7 sleep 1
docker cp ./alloc_page centos7:/ docker cp ./reproduce.sh centos7:/
docker exec -it centos7 /bin/bash
2. try reproduce the problem in docker:
$ cat reproduce.sh #!/bin/bash
while true; do flag=$(ps -ef | grep -v grep | grep alloc_page| wc -l) if [ "$flag" -eq 0 ]; then /alloc_page & fi
sleep 30
start_time=$(date +%s) yum install -y expect > /dev/null 2>&1
end_time=$(date +%s)
elapsed_time=$((end_time - start_time))
echo "$elapsed_time seconds" yum remove -y expect > /dev/null 2>&1 done
$ cat alloc_page.c: #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h>
#define SIZE 1*1024*1024 //1M
int main() { void *addr = NULL; int i;
for (i = 0; i < 1024 * 6 - 50;i++) { addr = (void *)malloc(SIZE); if (!addr) return -1;
memset(addr, 0, SIZE); }
sleep(99999); return 0; }
We found that this problem is caused by a lot ot meaningless read-ahead. Since the docker is almost met memory limit, the page will be reclaimed immediately after read-ahead and will read-ahead again immediately. The program is executed slowly and waste a lot of I/O resource.
These patches aim to break the read-ahead in above scenario.
Liu Shixin (3): mm/readahead: break read-ahead loop if filemap_add_folio return -ENOMEM mm/filemap: don't decrease mmap_miss when folio has workingset flag mm/readahead: add sysctl to enable early break readahead
Documentation/admin-guide/sysctl/vm.rst | 7 +++++++ include/linux/pagemap.h | 2 ++ kernel/sysctl.c | 9 +++++++++ mm/filemap.c | 14 +++++++++++++- mm/readahead.c | 20 +++++++++++++++----- 5 files changed, 46 insertions(+), 6 deletions(-)