- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 4/4] distro/depends: add vim in lkp
by Zhang Yale 18 Jan '21

18 Jan '21

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- distro/depends/lkp | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/lkp b/distro/depends/lkp index 82c67184..5a8196ac 100644 --- a/distro/depends/lkp +++ b/distro/depends/lkp @@ -6,3 +6,4 @@ gzip which rsync ntpdate +vim -- 2.23.0

1 0

[PATCH lkp-tests 1/4] pkg/turbostat: changed turbostat's source url
by Zhang Yale 18 Jan '21

18 Jan '21

turbostat can not be found in x86_64 Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- pkg/turbostat/PKGBUILD | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkg/turbostat/PKGBUILD b/pkg/turbostat/PKGBUILD index d7e536c7..24adcffe 100644 --- a/pkg/turbostat/PKGBUILD +++ b/pkg/turbostat/PKGBUILD @@ -4,7 +4,7 @@ pkgrel=1 url='https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git' arch=('i386' 'x86_64') license=('GPL') -source=('https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git') +source=("git://$GIT_SERVER/l/linux/linux.git") md5sums=('SKIP') build() -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] lib/compare_error_messages.rb: add the credibility check of job
by Lin Jiaxin 18 Jan '21

18 Jan '21

defect: may match the wrong result error_id="build-pkg.pixz.c:warning:'subsuf'defined-but-not-used[-Wunused-function]" filename="pixz.c" correct result: gcc -DHAVE_CONFIG_H -I. -I.. -D_FORTIFY_SOURCE=2 -pthread -Wall -Wno-unknown-pragmas -march=native -O2 -pipe -fstack-protector-strong -fno-plt -Wl,-O1, --sort-common,--as-needed,-z,relro,-z,now -MT pixz-pixz.o -MD -MP -MF .deps/pixz-pixz.Tpo -c -o pixz-pixz.o `test -f 'pixz.c' || echo './'`pixz.c ^^^^^^ wrong result: mv -f .deps/pixz.cpu.Tpo .deps/pixz.cpu.Po ^^^^^^ Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 0b51fc3..b3812bb 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -8,6 +8,39 @@ require_relative 'es_query' require_relative 'error_messages' require_relative "#{ENV['LKP_SRC']}/lib/common" +# previous_job_id's error_ids include error_id +# If later_job_id's output include error filename that extract from error_id, +# the later_job_id is credible. +def credible?(previous_job_id, later_job_id, error_id) + es = ESQuery.new + + previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') + later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') + + return false if filenames_check(previous_result_file, later_result_file, error_id).value?(false) + + return true +end + +def filenames_check(previous_result_file, later_result_file, error_id) + filenames = Set.new + filenames_check = Hash.new { |h, k| h[k] = false } + + error_lines = ErrorMessages.new(previous_result_file).obtain_error_messages_by_error_id(error_id, true) + error_lines.each do |error_line| + filenames << $1 if error_line =~ /(.*)(:.*){2}: (error|warning):/ + end + + File.open(later_result_file).each_line do |line| + filenames.each do |filename| + filenames_check[filename] + filenames_check[filename] = true if line.include?(filename) + end + end + + return filenames_check +end + def get_compare_result(previous_job_id, later_job_id) es = ESQuery.new -- 2.23.0

2 1

[PATCH v3 compass-ci] lib/error_messages.rb: obtain_error_messages_by_error_id function add keyword argument
by Lin Jiaxin 18 Jan '21

18 Jan '21

Determine whether to return the error line or error messages by the value of return_error_line Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/error_messages.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/error_messages.rb b/lib/error_messages.rb index 3e1564b..d83d890 100644 --- a/lib/error_messages.rb +++ b/lib/error_messages.rb @@ -40,12 +40,12 @@ class ErrorMessages return @error_messages end - def obtain_error_messages_by_error_id(error_id) + def obtain_error_messages_by_error_id(error_id, return_error_line = false) error_messages_by_error_id = [] error_messages = obtain_error_messages error_messages.each do |k, v| if ('build-pkg.' + build_pkg_error_id(k)) == error_id - error_messages_by_error_id += v.to_a + error_messages_by_error_id += return_error_line ? [k] : v.to_a end end error_messages_by_error_id -- 2.23.0

2 1

[PATCH compass-ci 2/3] qemu/kvm.sh: optimize the prompt when the error occurs
by Yu Chuan 18 Jan '21

18 Jan '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- providers/qemu/kvm.sh | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 35678fbd44ef..f1d65272dbf9 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -52,15 +52,23 @@ parse_ipxe_script() check_option_value() { + [ -n "$kernel" ] || { + echo "[ERROR] Can not find job for current hostname: $hostname." + exit 1 + } + [ -s "$kernel" ] || { - echo "The kernel does not exist: $kernel" - exit + echo "[ERROR] Can not find kernel file or kernel file is empty: $kernel." + exit 1 } # debian has both qemu-system-x86_64 and qemu-system-riscv64 command [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 - [ -n "$initrds" ] || exit + [ -n "$initrds" ] || { + echo "[ERROR] Can not find any initrd for current vm." + exit 1 + } } set_initrd() -- 2.23.0

3 4

[PATCH compass-ci] src/lib/web_backend.rb: fix wrong number of arguments
by Lu Weitao 18 Jan '21

18 Jan '21

[Why] wrong number of arguments (given 3, expected 4) 2021-01-18 11:49:46.000000000 +0800 web-backend: 172.17.0.1 - - [18/Jan/2021:03:49:44 UTC] "GET /compare?suite=iperf&dimension=os HTTP/1.1" 500 13 Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/lib/web_backend.rb | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index ea67064..61e3334 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -186,15 +186,15 @@ end def do_get_groups_matrices(must, dimension, total, size, from) job_list = get_dimension_job_list(dimension, must, size, from) - matrices, suites_hash = combine_group_query_data(job_list, dimension) + matrices, suites_hash, latest_jobs_hash = combine_group_query_data(job_list, dimension) while matrices.empty? from += size break if from > total job_list = get_dimension_job_list(dimension, must, size, from) - matrices, suites_hash = combine_group_query_data(job_list, dimension) + matrices, suites_hash, latest_jobs_hash = combine_group_query_data(job_list, dimension) end - [matrices, suites_hash] + [matrices, suites_hash, latest_jobs_hash] end def get_groups_matrices(conditions, dimension, must, size, from) @@ -209,11 +209,12 @@ end def get_compare_body(params) dimension, conditions = get_dimension_conditions(params) must = get_es_must(params) - groups_matrices, suites_hash = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) + groups_matrices, suites_hash, latest_jobs_hash = + get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) if !groups_matrices || groups_matrices.empty? body = 'No Data.' else - body = compare_group_matrices(groups_matrices, suites_hash, { no_print: true }) + body = compare_group_matrices(groups_matrices, suites_hash, latest_jobs_hash, { no_print: true }) body = 'No Difference.' if !body || body.empty? end return body -- 2.23.0

1 0

[PATCH compass-ci 3/3] my-qemu.sh: add `-h|--help`, `-d|--debug` params for my-qemu.sh
by Yu Chuan 18 Jan '21

18 Jan '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- providers/my-qemu.sh | 32 ++++++++++++++++++++++++++++++++ providers/qemu/kvm.sh | 5 +++-- providers/qemu/pxe.sh | 6 ++++-- 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 8845a346e195..a8f09a610d3f 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -2,6 +2,38 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +help() +{ + cat <<-'EOF' + Usage: ./my-qemu.sh [OPTION]... + Ask scheduer service whether there is a job, and start a local vm if there is a job. + Scheduler api: http://${SCHED_HOST}:${SCHED_PORT}/boot.ipxe/mac/${mac} + + Mandatory arguments to long options are mandatory for short options too. + -h, --help display this help and exit + -d, --debug open the local vm serial port in the current shell + EOF + + exit 0 +} + +while true +do + [ $# -eq 0 ] && break + case "$1" in + -h|--help) + help + shift;; + -d|--debug) + export DEBUG=true + shift;; + *) + echo "[ERROR] Unknown param: $1" + help + exit 1;; + esac +done + [[ $tbox_group ]] || tbox_group=vm-2p8g export hostname=$tbox_group.$USER-$$ diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index f1d65272dbf9..a3cbe4248a3a 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -123,7 +123,7 @@ print_message() echo kernel: $kernel echo initrds: $initrds echo append: $append - echo less $log_file + [ "$DEBUG" == "true" ] || echo less $log_file sleep 5 } @@ -140,9 +140,10 @@ public_option() -k en-us -no-reboot -nographic - -serial file:${log_file} -monitor null ) + + [ "$DEBUG" == "true" ] || kvm=("${kvm[@]}" -serial file:${log_file}) } individual_option() diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index fa688331964a..2e205405c793 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -18,7 +18,7 @@ fi qemu=qemu-system-aarch64 command -v $qemu >/dev/null || qemu=qemu-kvm -echo less $serial_log +[ "$DEBUG" == "true" ] || echo less $serial_log kvm=( $qemu @@ -33,7 +33,9 @@ kvm=( -k en-us -no-reboot -nographic - -serial file:${serial_log} -monitor null ) + +[ "$DEBUG" == "true" ] || kvm=("${kvm[@]}" -serial file:${serial_log}) + "${kvm[@]}" -- 2.23.0

2 2

[PATCH compass-ci] sparrow/0-package: add packages
by Liu Yinsi 18 Jan '21

18 Jan '21

container os-cifs and os-nfs will use sudo command, so add it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index b25dade..b8ef769 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -24,6 +24,7 @@ pkgs=( hostname wget curl + sudo ) yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 4d13413..947cbdc 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -24,6 +24,7 @@ pkgs=( qemu-system-aarch64 qemu-system-x86 qemu-system-riscv64 + sudo ) apt-get update diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 04996f5..61a37c4 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -23,6 +23,7 @@ pkgs=( hostname libvirt curl + sudo ) yum install -y --skip-broken "${pkgs[@]}" -- 2.23.0

1 0

[PATCH compass-ci] container/srv-http: mount /srv/cci/libvirt-xml to srv-http
by Xiao Shenwei 18 Jan '21

18 Jan '21

[why] libvirt support user define domain.xml file to start kvm the /srv/cci/libvirt-xml will save user's xml file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/srv-http/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start b/container/srv-http/start index d9f461b..945cde0 100755 --- a/container/srv-http/start +++ b/container/srv-http/start @@ -16,6 +16,7 @@ cmd=( -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro -v /srv/os:/srv/os:ro -v /srv/git/archlinux:/srv/git/archlinux:ro + -v /srv/cci/libvirt-xml:/srv/cci/libvirt-xml:ro -d srv-http ) -- 2.23.0

1 0

[PATCH compass-ci 1/3] qemu/kvm.sh: fix script will hang when there is no job
by Yu Chuan 18 Jan '21

18 Jan '21

[Error Msg] ++ set_initrd ++ initrd=initrd ++ cat <<< hang here [Root Cause] Function set_initrd() have follow line: cat $initrds > $initrd When there is no job corresponding to current vm in the remote redis queue, var $initrds will be equal to "". And from `cat --help`, we can know the follow: cat Copy standard input to standard output. So the process will get stuck waiting for input. [How] Function check_option_value() is used to check whether $initrds or $kernel is empty. And $initrds and $kernel are assigned in the function parse_ipxe_script(). So move check_option_value() before set_initrd(). Signed-off-by: Yu Chuan <13186087857(a)163.com> --- providers/qemu/kvm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 854e9138a7c0..35678fbd44ef 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -197,8 +197,8 @@ write_logfile parse_ipxe_script -set_options check_option_value +set_options print_message -- 2.23.0

2 6