January 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci] sbin/build-my-info: optimize options
by Luan Shengde 29 Jan '21

29 Jan '21

add output usage add option -h|--help set default option to -h|--help if no options added Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/{build-my-info.rb => build-my-info} | 12 ++++++++++++ 1 file changed, 12 insertions(+) rename sbin/{build-my-info.rb => build-my-info} (78%) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info similarity index 78% rename from sbin/build-my-info.rb rename to sbin/build-my-info index fd75f51..ca6a1de 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info @@ -14,6 +14,11 @@ option = { } options = OptionParser.new do |opts| + opts.banner = "Usage: build-my-info -e email -n name -l lab\n" + + opts.separator '' + opts.separator 'options:' + opts.on('-e email', 'my_email') do |email| option[:my_email] = email end @@ -25,8 +30,15 @@ options = OptionParser.new do |opts| opts.on('-l lab', 'lab') do |lab| option[:lab] = lab end + + opts.on_tail('-h', '--help', 'show this message') do + puts opts + exit + end end +ARGV << '-h' if ARGV.empty? + options.parse! build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -- 2.23.0

1 0

[PATCH v2 compass-ci] sbin/build-my-info: optimize options
by Luan Shengde 29 Jan '21

29 Jan '21

add output usage add option -h|--help set default option to -h|--help if no options added Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index fd75f51..028fbac 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -14,6 +14,11 @@ option = { } options = OptionParser.new do |opts| + opts.banner = "Usage: ./build-my-info.rb -e email -n name -l lab\n" + + opts.separator '' + opts.separator 'options:' + opts.on('-e email', 'my_email') do |email| option[:my_email] = email end @@ -25,8 +30,15 @@ options = OptionParser.new do |opts| opts.on('-l lab', 'lab') do |lab| option[:lab] = lab end + + opts.on_tail('-h', '--help', 'show this message') do + puts opts + exit + end end +ARGV << '-h' if ARGV.empty? + options.parse! build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -- 2.23.0

2 3

[PATCH v2 compass-ci 2/2] sparrow: call download-rootfs script
by Liu Yinsi 29 Jan '21

29 Jan '21

For locall deploy compass-ci, call $CCI_SRC/sbin/download-rootfs to download default os rootfs openeuler aarch64 20.03. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/os-ready | 5 +++++ sparrow/install-tiny | 1 + 2 files changed, 6 insertions(+) create mode 100755 sparrow/5-build/os-ready diff --git a/sparrow/5-build/os-ready b/sparrow/5-build/os-ready new file mode 100755 index 0000000..48a740b --- /dev/null +++ b/sparrow/5-build/os-ready @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/download-rootfs diff --git a/sparrow/install-tiny b/sparrow/install-tiny index a291549..5f026e5 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -19,4 +19,5 @@ cd $CCI_SRC/sparrow || exit . /etc/profile.d/compass.sh 4-docker/buildall 5-build/ipxe +5-build/os-ready 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH compass-ci] providers/qemu: add name parameter for qemu-kvm
by Xiao Shenwei 29 Jan '21

29 Jan '21

[why] when we need reboot a vm, we need get it's pid from hostname, then check job_id is the same as required. [how] -name guest=$hostname,process=$job_id if you need reboot the vm: pkill $job_id Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/qemu/kvm.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 66099ee..dd657cb 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -39,6 +39,9 @@ parse_ipxe_script() ;; initrd) file=$(basename "$b") + [ $file == "job.cgz" ] && { + job_id=$(basename $(dirname "$b")) + } wget --timestamping -a ${log_file} --progress=bar:force $b initrds+="$file " ;; @@ -144,6 +147,7 @@ public_option() { kvm=( $qemu + -name guest=$hostname,process=$job_id -kernel $kernel -initrd $initrd -smp $nr_cpu -- 2.23.0

2 2

[PATCH compass-ci] lib/compare_matrixes.rb: treatment of stddev < 1%
by Lu Weitao 29 Jan '21

29 Jan '21

[Why] if stddev < 1% | == 0 We display nothing for stddev column like bellow, it may confuse user. 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 276589.17 -5.2% 262249.03 redis.get_avg_throughput_rps 61.47 +5.5% 64.83 redis.get_avg_time_sec 58.33 ± 53% -100.0% 0.00 redis.get_latency_0ms% [How] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 276589.17 < 1% -5.2% 262249.03 < 1% redis.get_avg_throughput_rps 61.47 < 1% +5.5% 64.83 < 1% redis.get_avg_time_sec 58.33 ± 53% -100.0% 0.00 redis.get_latency_0ms% Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 7760c93..6c88384 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -72,7 +72,7 @@ def get_values(value_list, success) if length > 1 && average != 0 stddev_percent = ( standard_deviation(value_list, average, length) * 100 / average - ).to_i + ) end { average: average, stddev_percent: stddev_percent, min: sorted[0], max: sorted[-1], sorted: sorted } @@ -204,7 +204,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, suite_list, latest_jobs = nil, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suite_list, latest_jobs=nil, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -615,6 +615,8 @@ def format_stddev_percent(stddev_percent, average_width) percent_width = SUB_LONG_COLUMN_WIDTH - average_width if stddev_percent if stddev_percent != 0 + return " < 1% " if stddev_percent < 1 + percent_str = get_suitable_number_str( stddev_percent.abs, percent_width - 4, -- 2.23.0

1 0

[PATCH compass-ci 1/2] srv-http/start: add os images data download directory
by Cui Yili 29 Jan '21

29 Jan '21

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/srv-http/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start b/container/srv-http/start index b56fe4d..8c8ca0b 100755 --- a/container/srv-http/start +++ b/container/srv-http/start @@ -14,6 +14,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result:/srv/result:ro -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro + -v /srv/os:/srv/os:ro -d srv-http ) -- 2.23.0

3 4

[PATCH v4 compass-ci 3/3] replace log.sh
by Liu Yinsi 29 Jan '21

29 Jan '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 2 +- container/result-cifs/start | 2 +- providers/my-qemu.sh | 4 +--- providers/qemu/kvm.sh | 3 +-- providers/qemu/pxe.sh | 4 +--- sparrow/4-docker/buildall | 6 +++--- 8 files changed, 10 insertions(+), 15 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 0b9601a..6a91b37 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-cifs/start b/container/os-cifs/start index 09ae3d2..85017cc 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-nfs/start b/container/os-nfs/start index 9539561..1f4ed42 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { log_error "kernel module nfs is missing (try: sudo modprobe nfs)" diff --git a/container/result-cifs/start b/container/result-cifs/start index de38ca2..7ce066a 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 113b3ef..3e3b2f4 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -2,9 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh help() { diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 66099ee..d394681 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -6,9 +6,8 @@ : ${nr_cpu:=1} : ${memory:=1G} -: ${LKP_SRC:="/c/lkp-tests"} -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh check_logfile() { diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index 0ba052c..3ab3c2b 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -4,9 +4,7 @@ # - nr_cpu # - memory -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh : ${nr_cpu:=1} : ${memory:=1G} diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 014036b..8d6a85a 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -5,7 +5,7 @@ CONTAINER_PATH="$CCI_SRC/container" export action=$1 -[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" +source ${CCI_SRC}/lib/log.sh build_depends() { @@ -27,7 +27,7 @@ do_one() lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock echo "$started_containers" | grep -w -q "$container_name" && return 0 - [ -n "$LKP_SRC" ] && log_info "start build $container." + log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" @@ -41,7 +41,7 @@ do_one() [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run ) - [ -n "$LKP_SRC" ] && log_info "finish build $container." + log_info "finish build $container." started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

1 0

[PATCH v4 compass-ci 2/3] container: delete redundant log error info
by Liu Yinsi 29 Jan '21

29 Jan '21

[how] before: log_error "[ERROR] kernel module ..." after: log_error "kernel module ..." [why] log_error(): echo "$(date +'%F %T') [ERROR] -- $*" >&2 when call log_error(), already has complete log format, so delete redundant "[ERROR]" in parmas. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 4 ++-- container/result-cifs/start | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 119d1b1..0b9601a 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-cifs/start b/container/os-cifs/start index 47da260..09ae3d2 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-nfs/start b/container/os-nfs/start index 8c08db8..9539561 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -6,11 +6,11 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { - log_error "[ERROR] kernel module nfs is missing (try: sudo modprobe nfs)" + log_error "kernel module nfs is missing (try: sudo modprobe nfs)" exit 1 } lsmod | grep -q "^nfsd\s" || { - log_error "[ERROR] kernel module nfsd is missing (try: sudo modprobe nfsd)" + log_error "kernel module nfsd is missing (try: sudo modprobe nfsd)" exit 1 } diff --git a/container/result-cifs/start b/container/result-cifs/start index 4f8c2b9..de38ca2 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } -- 2.23.0

1 0

[PATCH v4 compass-ci 1/3] lib: add log.sh
by Liu Yinsi 29 Jan '21

29 Jan '21

Records the run logs of the shell program Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/log.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 lib/log.sh diff --git a/lib/log.sh b/lib/log.sh new file mode 100644 index 0000000..626a6e4 --- /dev/null +++ b/lib/log.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +log_info() +{ + date +"[INFO] %F %T ${0##*/}: $*" +} + +log_debug() +{ + date +"[DEBUG] %F %T ${0##*/}: $*" +} + +log_warn() +{ + date +"[WARN] %F %T ${0##*/}: $*" >&2 +} + +log_error() +{ + date +"[ERROR] %F %T ${0##*/}: $*" >&2 +} -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] replace log.sh
by Liu Yinsi 29 Jan '21

29 Jan '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 4 ++-- container/os-cifs/start | 4 ++-- container/os-nfs/start | 4 ++-- container/result-cifs/start | 4 ++-- providers/my-qemu.sh | 4 +--- providers/qemu/kvm.sh | 3 +-- providers/qemu/pxe.sh | 4 +--- sparrow/4-docker/buildall | 8 ++++---- 8 files changed, 15 insertions(+), 20 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 119d1b1..6a91b37 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -3,10 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-cifs/start b/container/os-cifs/start index 47da260..85017cc 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -3,10 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-nfs/start b/container/os-nfs/start index 8c08db8..7577647 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -3,10 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { - log_error "[ERROR] kernel module nfs is missing (try: sudo modprobe nfs)" + log_error "kernel module nfs is missing (try: sudo modprobe nfs)" exit 1 } lsmod | grep -q "^nfsd\s" || { diff --git a/container/result-cifs/start b/container/result-cifs/start index 4f8c2b9..7ce066a 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -3,10 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 113b3ef..3e3b2f4 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -2,9 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh help() { diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 66099ee..d394681 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -6,9 +6,8 @@ : ${nr_cpu:=1} : ${memory:=1G} -: ${LKP_SRC:="/c/lkp-tests"} -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh check_logfile() { diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index 0ba052c..3ab3c2b 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -4,9 +4,7 @@ # - nr_cpu # - memory -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh : ${nr_cpu:=1} : ${memory:=1G} diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index a60067f..9d6c977 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -2,11 +2,11 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +source ${CCI_SRC}/lib/log.sh + CONTAINER_PATH="$CCI_SRC/container" export action=$1 -[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" - build_depends() { local container=$1 @@ -27,7 +27,7 @@ do_one() lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock echo "$started_containers" | grep -w -q "$container_name" && return 0 - [ -n "$LKP_SRC" ] && log_info "start build $container." + log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" @@ -40,7 +40,7 @@ do_one() [ -x first-run ] && ./first-run [ -x start ] && ./start ) - [ -n "$LKP_SRC" ] && log_info "finish build $container." + log_info "finish build $container." started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 4