- Compass-ci - mailweb.openeuler.org

[PATCH v5 compass-ci 5/6] container/mail-robot: start file
by Luan Shengde 12 Oct '20

12 Oct '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..a2e183d --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,29 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative '../defconfig.rb' +docker_rm 'mail-robot' + +JUMPER_IP = '192.168.132.2' +JUMPER_PORT = '29999' +SEND_MAIL_PORT = '11311' + +cmd = %W[ + docker run + --restart=always + --name=mail-robot + -d + -e JUMPER_IP=#{JUMPER_IP} + -e JUMPER_PORT=#{JUMPER_PORT} + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -v /c/upstream-repos:/upstream-repos:ro + -v /srv/cci/Maildir:/Maildir:rw + -v /c/compass-ci:/compass-ci:ro + mail-robot +] + +cmd += ['bash', '-c', 'umask 002 && ruby ./mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci] feat: a link for one command to build centos initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-$version initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..2d22759 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos/run \ No newline at end of file -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/srv-http allow access file with level 4 path
by Lu Weitao 12 Oct '20

12 Oct '20

problem: can not access file with level 4 path like: http://localhost:11300/result/unixbench/crystal.82998/boot-time.json cause analysis: we had forbidden access level 4 catalog and file solution: keep forbidden access level 4 catalog, allow access file with level 4 path. Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 403b0dc..9c15d99 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -22,7 +22,7 @@ server { } } - location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/?)$ { + location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/)$ { deny all; } } -- 2.23.0

1 0

[PATCH compass-ci 1/2] container/srv-http fix Chinese garbled
by Lu Weitao 12 Oct '20

12 Oct '20

problem: Chinese file name and content display garbled characters Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 2bf55b6..403b0dc 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -6,6 +6,8 @@ server { root /usr/share/nginx/html; index index.html; + charset utf-8,gbk; + location /favicon.ico { log_not_found off; } -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: update_id need set_initrds_uri
by Tong Qunfeng 12 Oct '20

12 Oct '20

Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index bf959c2..504bc19 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -382,9 +382,13 @@ class Job def update_id(id) @hash["id"] = JSON::Any.new(id) - # "result_root" is based on "id" + # "result_root" => "/result/#{suite}/#{tbox_group}/#{date}/#{id}" + # set_initrds_uri -> get_initrds -> common_initrds => ".../#{id}/job.cgz" + # + # "result_root, common_initrds" is associate with "id" # so when update id, we need redo set_ set_result_root() + set_initrds_uri() end def get_uuid_tag -- 2.23.0

1 0

[PATCH v4 compass-ci] feat: build centos-7 initramfs image with one command
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with one command. and we also can customize the pre-installed software by configuring ./bin/install-list. --- container/osimage/centos-7/README.md | 13 +++++++ container/osimage/centos-7/bin/create-image | 38 +++++++++++++++++++++ container/osimage/centos-7/bin/delete-list | 8 +++++ container/osimage/centos-7/bin/install-list | 2 ++ container/osimage/centos-7/bin/lib | 30 ++++++++++++++++ container/osimage/centos-7/build | 29 ++++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 container/osimage/centos-7/README.md create mode 100755 container/osimage/centos-7/bin/create-image create mode 100644 container/osimage/centos-7/bin/delete-list create mode 100644 container/osimage/centos-7/bin/install-list create mode 100755 container/osimage/centos-7/bin/lib create mode 100755 container/osimage/centos-7/build diff --git a/container/osimage/centos-7/README.md b/container/osimage/centos-7/README.md new file mode 100644 index 0000000..2c2f956 --- /dev/null +++ b/container/osimage/centos-7/README.md @@ -0,0 +1,13 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + cd ${CCI_SRC/rootfs/initramfs/centos/aarch64/7} + ./build + +Some configuration items: +./bin/install-list + If you want to pre-install the software, you can write the package names in ./bin/install-list. + +./bin/delete-list + If you want remove some unnecessary files, you can write the names in ./bin/delete-list + diff --git a/container/osimage/centos-7/bin/create-image b/container/osimage/centos-7/bin/create-image new file mode 100755 index 0000000..ad51986 --- /dev/null +++ b/container/osimage/centos-7/bin/create-image @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/install-list) + yum clean all + rm -rf $(</root/bin/delete-list) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/container/osimage/centos-7/bin/delete-list b/container/osimage/centos-7/bin/delete-list new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/container/osimage/centos-7/bin/delete-list @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/container/osimage/centos-7/bin/install-list b/container/osimage/centos-7/bin/install-list new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/container/osimage/centos-7/bin/install-list @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/container/osimage/centos-7/bin/lib b/container/osimage/centos-7/bin/lib new file mode 100755 index 0000000..c3c470f --- /dev/null +++ b/container/osimage/centos-7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(basename $(dirname $(realpath $0)))" +name="$(echo $DIR |awk -F "-" '{print $1}')" +version="$(echo $DIR |awk -F "-" '{print $2}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/container/osimage/centos-7/build b/container/osimage/centos-7/build new file mode 100755 index 0000000..5e46306 --- /dev/null +++ b/container/osimage/centos-7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos-7/bin/lib +. ${CCI_SRC}/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/create-image +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH lkp-tests] /lkp-tests/jobs: add test-tools.yaml
by Li Ping 12 Oct '20

12 Oct '20

--- jobs/test-tools.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 jobs/test-tools.yaml diff --git a/jobs/test-tools.yaml b/jobs/test-tools.yaml new file mode 100644 index 00000000..43ca1343 --- /dev/null +++ b/jobs/test-tools.yaml @@ -0,0 +1,8 @@ +suite: test-tools +category: functional +os_mount: initramfs + +test-tools: + add: 1 + testsuite: smoke-testing + testcase: oe_test_dd_001 -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: output exception backtrace
by Tong Qunfeng 12 Oct '20

12 Oct '20

we catch exception at submit_job and prepare_job, and just output the exception message. it is not easy to find out detail reason, so add backtrace. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/sched.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2f432a9..23332df 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -188,6 +188,7 @@ class Sched return submit_single_job(job) rescue ex + puts ex.inspect_with_backtrace return [{ "job_id" => "0", "message" => ex.to_s, @@ -444,6 +445,7 @@ class Sched job = @es.get_job(job_id.to_s) rescue ex puts "Invalid job (id=#{job_id}) in es. Info: #{ex}" + puts ex.inspect_with_backtrace end end -- 2.23.0

1 0

[PATCH compass-ci] mount busybox into container
by Xiao Shenwei 12 Oct '20

12 Oct '20

some container may missing basic commands, such as curl wget ... use busybox to replace. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/run.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 6798f48..3c3b0ec 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -18,6 +18,7 @@ host=${tbox_group%%--*} create_yaml_variables "$LKP_SRC/hosts/${host}" DIR=$(dirname $(realpath $0)) +busybox_path=$(command -v busybox) cmd=( docker run --rm @@ -28,6 +29,7 @@ cmd=( -v ${DIR}/bin:/root/bin:ro -v $CCI_SRC:/c/compass-ci -v /srv/git:/srv/git:ro + -v ${busybox_path}:/bin/busybox --oom-score-adj="-1000" ${docker_image} /root/bin/entrypoint.sh -- 2.23.0

2 1

[PATCH v3 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initramfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initramfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initramfs/centos/aarch64/7/bin/lib | 30 +++++++++++++++ .../initramfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initramfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 rootfs/initramfs/centos/aarch64/7/README.md create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/README.md b/rootfs/initramfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..1e47e11 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/creat b/rootfs/initramfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..4dacbdf --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initramfs/centos/aarch64/7/bin/dellist b/rootfs/initramfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initramfs/centos/aarch64/7/bin/lib b/rootfs/initramfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..1bf0b5d --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $(NF - 2)}')" +version="$(echo $DIR |awk -F "/" '{print $NF}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/repolist b/rootfs/initramfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH compass-ci] feat: a link for one command to build centos-7 initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-7 initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..e3adb9c --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos-7/build \ No newline at end of file -- 2.23.0

1 0

[PATCH compass-ci] fix rubocop warnings on *.cr
by Wu Fengguang 12 Oct '20

12 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- src/extract-stats/constants.cr | 2 +- src/extract-stats/extract_stats.cr | 10 +++++----- src/monitoring/monitoring.cr | 1 + src/scheduler/jobfile_operate.cr | 21 +++++++++++---------- src/spec/scheduler/tools_spec.cr | 8 ++++---- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/src/extract-stats/constants.cr b/src/extract-stats/constants.cr index 332962a..70c66ad 100644 --- a/src/extract-stats/constants.cr +++ b/src/extract-stats/constants.cr @@ -5,4 +5,4 @@ EXTRACT_STATS_QUEUE_PATH = "extract_stats" STATS_WORKER_COUNT = 10 DELIMITER_TASK_QUEUE = "delimiter" -ERROR_ID_FILES = %w(build-pkg.json) +ERROR_ID_FILES = %w[build-pkg.json] diff --git a/src/extract-stats/extract_stats.cr b/src/extract-stats/extract_stats.cr index d3f3203..45310ad 100644 --- a/src/extract-stats/extract_stats.cr +++ b/src/extract-stats/extract_stats.cr @@ -7,15 +7,15 @@ require "./stats_worker" module ExtractStats # Consume scheduler queue def self.in_extract_stats - self.back_fill_task + back_fill_task STATS_WORKER_COUNT.times do - Process.fork { - self.consume_task - } + Process.fork do + consume_task + end end # keep main-process alive - sleep() + sleep end def self.consume_task diff --git a/src/monitoring/monitoring.cr b/src/monitoring/monitoring.cr index 178e4c9..e665d5f 100644 --- a/src/monitoring/monitoring.cr +++ b/src/monitoring/monitoring.cr @@ -24,6 +24,7 @@ module Monitoring socket.on_close do next unless query.as_h? + filter.remove_filter_rule(query, socket) end end diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 0533346..e0e298f 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -15,7 +15,7 @@ end module Jobfile::Operate def self.prepare_dir(file_path : String) file_path_dir = File.dirname(file_path) - if !File.exists?(file_path_dir) + unless File.exists?(file_path_dir) FileUtils.mkdir_p(file_path_dir) end end @@ -59,11 +59,12 @@ module Jobfile::Operate if val.as_h? return false end - if val.as_a? - value = shell_escape(val.as_a) - else - value = shell_escape(val.to_s) - end + + value = if val.as_a? + shell_escape(val.as_a) + else + shell_escape(val.to_s) + end script_lines << "\texport #{key}=" + value if value end end @@ -78,7 +79,7 @@ module Jobfile::Operate script_lines << "\texport LKP_SRC=/lkp/${user:-lkp}/src" script_lines << "}\n\n" - script_lines = "#{script_lines}" + script_lines = script_lines.to_s script_lines = JSON.parse(script_lines) end @@ -121,7 +122,7 @@ module Jobfile::Operate if job_sh_array.empty? lkp_src = prepare_lkp_tests(job_content["lkp_initrd_user"], - job_content["os_arch"]) + job_content["os_arch"]) cmd = "#{lkp_src}/sbin/create-job-cpio.sh #{temp_yaml}" idd = `#{cmd}` @@ -158,8 +159,8 @@ module Jobfile::Operate # update lkp-xxx.cgz if they are different target_path = update_lkp_when_different(expand_dir_base, - lkp_initrd_user, - os_arch) + lkp_initrd_user, + os_arch) # delete oldest lkp, if exists too much del_lkp_if_too_much(expand_dir_base) diff --git a/src/spec/scheduler/tools_spec.cr b/src/spec/scheduler/tools_spec.cr index 0a71af0..4c3a6aa 100644 --- a/src/spec/scheduler/tools_spec.cr +++ b/src/spec/scheduler/tools_spec.cr @@ -8,16 +8,16 @@ require "file_utils" describe Public do describe "hash replace" do it "can replace k:v" do - hash_old = {"name" => "OldValue"} - hash_new = {"name" => "NewValue"} + hash_old = { "name" => "OldValue" } + hash_new = { "name" => "NewValue" } hash_result = Public.hash_replace_with(hash_old, hash_new) (hash_result["name"]).should eq("NewValue") end it "can add k:v" do - hash_old = {"name2" => "OldValue"} - hash_new = {"name" => "NewValue"} + hash_old = { "name2" => "OldValue" } + hash_new = { "name" => "NewValue" } hash_result = Public.hash_replace_with(hash_old, hash_new) (hash_result["name"]).should eq("NewValue") -- 2.23.0

1 0

[PATCH lkp-tests] tests: tests: add get_ipmi_ip function to host-info
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..3cabebbf 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

2 1

[PATCH lkp-tests] tests: add get_ipmi_ip function to host-info
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..11cbed38 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip() } main -- 2.23.0

1 0

[PATCH lkp-tests] tests: fix 'memory' field more precise
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..a57855dc 100755 --- a/tests/host-info +++ b/tests/host-info @@ -5,8 +5,8 @@ get_node() echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') + memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -- 2.23.0

1 0

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initramfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..1e47e11 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 3

[PATCH lkp-tests] tests: fix 'memory' field more precise and add ipmi_ip
by Zhang Yu 10 Oct '20

10 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..6ee8df21 100755 --- a/tests/host-info +++ b/tests/host-info @@ -5,8 +5,8 @@ get_node() echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') + memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

2 1

[PATCH compass-ci] mount busybox into container
by Xiao Shenwei 10 Oct '20

10 Oct '20

some container may missing basic commands, such as curl wget ... use busybox to replace. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/run.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 6798f48..f8aba13 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -28,6 +28,7 @@ cmd=( -v ${DIR}/bin:/root/bin:ro -v $CCI_SRC:/c/compass-ci -v /srv/git:/srv/git:ro + -v /usr/bin/busybox:/usr/bin/busybox --oom-score-adj="-1000" ${docker_image} /root/bin/entrypoint.sh -- 2.23.0

3 8

[PATCH compass-ci] feat: build initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 +++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 46 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 32 +++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 28 +++++++++++ 6 files changed, 128 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..c503925 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +#Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package name in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the name in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..8cd0d2c --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,46 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +#Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idmv >/dev/null + + +} + +#config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + echo $passwd_md5 + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" + } + +} + +#pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" |cpio -o -Hnewc |gzip -9 >/${image_name} + chmod 644 /${image_name} +} + + +make_rootfs +pre_config_rootfs +cat $ROOTFS_DIR/etc/shadow +pack_cgz + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..da0cdaa --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,32 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +image="centos:7" +name="$(echo "$image" | cut -d '/' -f 2 | cut -d ':' -f 1)" +version="$(echo "$image" | cut -d '/' -f 2 | cut -d ':' -f 2)" +image_name="${name}-${version}.cgz" + +#check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + + +#pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh down image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..36a8216 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker rm -f init_docker 2&> /dev/null +. ./bin/lib + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

4 13

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..ff79352 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

1 1

[PATCH compass-ci] fix(dnsmasq): DHCP lease record will be lost after restart container
by Yu Chuan 10 Oct '20

10 Oct '20

Persist the lease file to /etc/compass-ci/dnsmasq/lease_dir so that each restart of the container won't cause the lease file to be lost. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/start | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/container/dnsmasq/start b/container/dnsmasq/start index 7230c9cf6188..ff4247046eb1 100755 --- a/container/dnsmasq/start +++ b/container/dnsmasq/start @@ -9,10 +9,15 @@ docker_rm dnsmasq kill $(ps -ef| grep dnsmasq| grep libvirt| awk '{print $2}') 2> /dev/null WORKSPACE=$(dirname "$(realpath "$0")") +DNSMASQ_CONF_DIR="/etc/compass-ci/dnsmasq" # load custom dnsmasq.conf -[ -d "/etc/compass-ci/dnsmasq.d/" ] && - cp -a /etc/compass-ci/dnsmasq.d/* ${WORKSPACE}/dnsmasq.d/ +[ -d "${DNSMASQ_CONF_DIR}/dnsmasq.d/" ] && + cp -a ${DNSMASQ_CONF_DIR}/dnsmasq.d/* ${WORKSPACE}/dnsmasq.d/ + +# persist the dhcp lease file +[ -d "${DNSMASQ_CONF_DIR}/lease_dir" ] || + mkdir -p ${DNSMASQ_CONF_DIR}/lease_dir cmd=( docker run @@ -22,6 +27,7 @@ cmd=( --publish 67:67/udp --publish 69:69/udp -v ${WORKSPACE}/dnsmasq.d:/etc/dnsmasq.d + -v ${DNSMASQ_CONF_DIR}/lease_dir:/var/lib/misc # persist the dhcp lease file -v /etc/localtime:/etc/localtime:ro -v /tftpboot:/tftpboot:ro --detach -- 2.23.0

1 0

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 28 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..f6af3ca --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +#Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..f47a701 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker rm -f init_docker 2&> /dev/null +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

1 1

[PATCH compass-ci] get busybox file from debian container
by Xiao Shenwei 10 Oct '20

10 Oct '20

add apt-get install -y busybox then copy it to host Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/qemu-efi/Dockerfile | 2 +- container/qemu-efi/install | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/container/qemu-efi/Dockerfile b/container/qemu-efi/Dockerfile index e1087c5..ced7f66 100644 --- a/container/qemu-efi/Dockerfile +++ b/container/qemu-efi/Dockerfile @@ -8,4 +8,4 @@ ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y qemu-efi-aarch64 + apt-get install -y qemu-efi-aarch64 busybox diff --git a/container/qemu-efi/install b/container/qemu-efi/install index 96be9aa..bc61b9a 100755 --- a/container/qemu-efi/install +++ b/container/qemu-efi/install @@ -12,4 +12,7 @@ cmd=( "${cmd[@]}" docker cp qemu-efi:/usr/share/qemu-efi-aarch64 /usr/share/qemu-efi-aarch64 +[ -f "/usr/bin/busybox" ] || { + docker cp qemu-efi:/bin/busybox /usr/bin/busybox +} docker_rm qemu-efi -- 2.23.0

3 3

[PATCH compass-ci] sparrow: install busybox
by Wu Fengguang 10 Oct '20

10 Oct '20

For use by docker provider. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 8700ac5..261592c 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -19,6 +19,7 @@ pkgs=( redis liblockfile lockfile-progs + busybox ) yum install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 68423d7..020c34a 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -17,6 +17,7 @@ pkgs=( dmidecode lockfile-progs bridge-utils + busybox ) apt-get install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 3573006..5c64fe8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -18,6 +18,7 @@ pkgs=( dmidecode redis wget + busybox ) yum install -y "${pkgs[@]}" -- 2.23.0

2 1

[PATCH lkp-tests] add depends for multi-docker
by Wu Fengguang 10 Oct '20

10 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- distro/depends/multi-docker | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 distro/depends/multi-docker diff --git a/distro/depends/multi-docker b/distro/depends/multi-docker new file mode 100644 index 00000000..b7b0bacf --- /dev/null +++ b/distro/depends/multi-docker @@ -0,0 +1,4 @@ +docker.io +busybox +ruby +ruby-json -- 2.23.0

1 0