- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] feat: a link for one command to build centos initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-$version initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..2d22759 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos/run \ No newline at end of file -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/srv-http allow access file with level 4 path
by Lu Weitao 12 Oct '20

12 Oct '20

problem: can not access file with level 4 path like: http://localhost:11300/result/unixbench/crystal.82998/boot-time.json cause analysis: we had forbidden access level 4 catalog and file solution: keep forbidden access level 4 catalog, allow access file with level 4 path. Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 403b0dc..9c15d99 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -22,7 +22,7 @@ server { } } - location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/?)$ { + location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/)$ { deny all; } } -- 2.23.0

1 0

[PATCH compass-ci 1/2] container/srv-http fix Chinese garbled
by Lu Weitao 12 Oct '20

12 Oct '20

problem: Chinese file name and content display garbled characters Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 2bf55b6..403b0dc 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -6,6 +6,8 @@ server { root /usr/share/nginx/html; index index.html; + charset utf-8,gbk; + location /favicon.ico { log_not_found off; } -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: update_id need set_initrds_uri
by Tong Qunfeng 12 Oct '20

12 Oct '20

Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index bf959c2..504bc19 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -382,9 +382,13 @@ class Job def update_id(id) @hash["id"] = JSON::Any.new(id) - # "result_root" is based on "id" + # "result_root" => "/result/#{suite}/#{tbox_group}/#{date}/#{id}" + # set_initrds_uri -> get_initrds -> common_initrds => ".../#{id}/job.cgz" + # + # "result_root, common_initrds" is associate with "id" # so when update id, we need redo set_ set_result_root() + set_initrds_uri() end def get_uuid_tag -- 2.23.0

1 0

[PATCH v4 compass-ci] feat: build centos-7 initramfs image with one command
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with one command. and we also can customize the pre-installed software by configuring ./bin/install-list. --- container/osimage/centos-7/README.md | 13 +++++++ container/osimage/centos-7/bin/create-image | 38 +++++++++++++++++++++ container/osimage/centos-7/bin/delete-list | 8 +++++ container/osimage/centos-7/bin/install-list | 2 ++ container/osimage/centos-7/bin/lib | 30 ++++++++++++++++ container/osimage/centos-7/build | 29 ++++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 container/osimage/centos-7/README.md create mode 100755 container/osimage/centos-7/bin/create-image create mode 100644 container/osimage/centos-7/bin/delete-list create mode 100644 container/osimage/centos-7/bin/install-list create mode 100755 container/osimage/centos-7/bin/lib create mode 100755 container/osimage/centos-7/build diff --git a/container/osimage/centos-7/README.md b/container/osimage/centos-7/README.md new file mode 100644 index 0000000..2c2f956 --- /dev/null +++ b/container/osimage/centos-7/README.md @@ -0,0 +1,13 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + cd ${CCI_SRC/rootfs/initramfs/centos/aarch64/7} + ./build + +Some configuration items: +./bin/install-list + If you want to pre-install the software, you can write the package names in ./bin/install-list. + +./bin/delete-list + If you want remove some unnecessary files, you can write the names in ./bin/delete-list + diff --git a/container/osimage/centos-7/bin/create-image b/container/osimage/centos-7/bin/create-image new file mode 100755 index 0000000..ad51986 --- /dev/null +++ b/container/osimage/centos-7/bin/create-image @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/install-list) + yum clean all + rm -rf $(</root/bin/delete-list) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/container/osimage/centos-7/bin/delete-list b/container/osimage/centos-7/bin/delete-list new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/container/osimage/centos-7/bin/delete-list @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/container/osimage/centos-7/bin/install-list b/container/osimage/centos-7/bin/install-list new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/container/osimage/centos-7/bin/install-list @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/container/osimage/centos-7/bin/lib b/container/osimage/centos-7/bin/lib new file mode 100755 index 0000000..c3c470f --- /dev/null +++ b/container/osimage/centos-7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(basename $(dirname $(realpath $0)))" +name="$(echo $DIR |awk -F "-" '{print $1}')" +version="$(echo $DIR |awk -F "-" '{print $2}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/container/osimage/centos-7/build b/container/osimage/centos-7/build new file mode 100755 index 0000000..5e46306 --- /dev/null +++ b/container/osimage/centos-7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos-7/bin/lib +. ${CCI_SRC}/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/create-image +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH lkp-tests] /lkp-tests/jobs: add test-tools.yaml
by Li Ping 12 Oct '20

12 Oct '20

--- jobs/test-tools.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 jobs/test-tools.yaml diff --git a/jobs/test-tools.yaml b/jobs/test-tools.yaml new file mode 100644 index 00000000..43ca1343 --- /dev/null +++ b/jobs/test-tools.yaml @@ -0,0 +1,8 @@ +suite: test-tools +category: functional +os_mount: initramfs + +test-tools: + add: 1 + testsuite: smoke-testing + testcase: oe_test_dd_001 -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: output exception backtrace
by Tong Qunfeng 12 Oct '20

12 Oct '20

we catch exception at submit_job and prepare_job, and just output the exception message. it is not easy to find out detail reason, so add backtrace. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/sched.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2f432a9..23332df 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -188,6 +188,7 @@ class Sched return submit_single_job(job) rescue ex + puts ex.inspect_with_backtrace return [{ "job_id" => "0", "message" => ex.to_s, @@ -444,6 +445,7 @@ class Sched job = @es.get_job(job_id.to_s) rescue ex puts "Invalid job (id=#{job_id}) in es. Info: #{ex}" + puts ex.inspect_with_backtrace end end -- 2.23.0

1 0

[PATCH compass-ci] mount busybox into container
by Xiao Shenwei 12 Oct '20

12 Oct '20

some container may missing basic commands, such as curl wget ... use busybox to replace. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/run.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 6798f48..3c3b0ec 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -18,6 +18,7 @@ host=${tbox_group%%--*} create_yaml_variables "$LKP_SRC/hosts/${host}" DIR=$(dirname $(realpath $0)) +busybox_path=$(command -v busybox) cmd=( docker run --rm @@ -28,6 +29,7 @@ cmd=( -v ${DIR}/bin:/root/bin:ro -v $CCI_SRC:/c/compass-ci -v /srv/git:/srv/git:ro + -v ${busybox_path}:/bin/busybox --oom-score-adj="-1000" ${docker_image} /root/bin/entrypoint.sh -- 2.23.0

2 1

[PATCH v3 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initramfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initramfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initramfs/centos/aarch64/7/bin/lib | 30 +++++++++++++++ .../initramfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initramfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 rootfs/initramfs/centos/aarch64/7/README.md create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/README.md b/rootfs/initramfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..1e47e11 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/creat b/rootfs/initramfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..4dacbdf --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initramfs/centos/aarch64/7/bin/dellist b/rootfs/initramfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initramfs/centos/aarch64/7/bin/lib b/rootfs/initramfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..1bf0b5d --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $(NF - 2)}')" +version="$(echo $DIR |awk -F "/" '{print $NF}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/repolist b/rootfs/initramfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH compass-ci] feat: a link for one command to build centos-7 initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-7 initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..e3adb9c --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos-7/build \ No newline at end of file -- 2.23.0

1 0