- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] distro/depends/nginx-server: spelling error
by Wei Jihui 28 Dec '20

28 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/nginx-server | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/depends/nginx-server b/distro/depends/nginx-server index da35d5f6..85e9cc2e 100644 --- a/distro/depends/nginx-server +++ b/distro/depends/nginx-server @@ -1,6 +1,6 @@ gcc glibc -ethool +ethtool libssl-dev zlib1g-dev libpcre3-dev -- 2.23.0

1 0

[PATCH compass-ci 3/3] providers/qemu: adapt kvm.sh for x86_64 and riscv64
by Liu Yinsi 28 Dec '20

28 Dec '20

put same options in public_options, different options in individual_options, merge options according different arch. test result: /srv/result/host-info/2020-12-28/vm-2p8g/openeuler-20.09-aarch64/z9.272042 /srv/result/host-info/2020-12-28/vm-2p8g/openeuler-20.09-x86_64/z9.272046 /srv/result/host-info/2020-12-28/vm-2p8g/debian-sid-riscv64/z9.272055 /srv/result/host-info/2020-12-28/vm-2p8g/debian-sid-x86_64/z9.272057 Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 78 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 70 insertions(+), 8 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 48bacab..83b9606 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -20,8 +20,19 @@ check_logfile() qemu_command() { - qemu=qemu-system-aarch64 - command -v $qemu >/dev/null || qemu=qemu-kvm + qemu=( + qemu-system-aarch64 + qemu-system-x86_64 + qemu-kvm + ) + + for qemu in "${qemu[@]}" + do + [ -n "$(command -v ${qemu})" ] && break + done + + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [ -n "$(echo $kernel | grep "riscv64")" ] && qemu=qemu-system-riscv64 } write_logfile() @@ -31,6 +42,17 @@ write_logfile() cat $ipxe_script >> ${log_file} } +options_value() +{ + bios=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd + ls $bios >/dev/null 2>&1 || bios=/usr/share/ovmf/OVMF.fd + + helper=/usr/libexec/qemu-bridge-helper + ls $helper >/dev/null 2>&1 || helper=/usr/lib/qemu/qemu-bridge-helper + + nic="tap,model=virtio-net-pci,helper=$helper,br=br0,mac=${mac}" +} + get_rootfs() { append= @@ -78,18 +100,14 @@ print_message() sleep 5 } -kvm_command() +public_options() { kvm=( $qemu - -machine virt-4.0,accel=kvm,gic-version=3 -kernel $kernel -initrd $initrd -smp $nr_cpu -m $memory - -cpu Kunpeng-920 - -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd - -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} -k en-us -no-reboot -nographic @@ -98,6 +116,46 @@ kvm_command() ) } +individual_options() +{ + aarch64=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + + x86_64=( + -bios $bios + -nic $nic + ) + + riscv64=( + -machine virt + -device virtio-net-device,netdev=net0,mac=${mac} + -netdev bridge,br=br0,id=net0,helper=${helper} + ) +} + +merge_options() +{ + case $qemu in + qemu-system-aarch64) + kvm=("${kvm[@]}" "${aarch64[@]}") + ;; + qemu-system-x86_64) + kvm=("${kvm[@]}" "${x86_64[@]}") + ;; + qemu-kvm) + [ "$(arch)" == "aarch64" ] && kvm=("${kvm[@]}" "${aarch64[@]}") + [ "$(arch)" == "x86_64" ] && kvm=("${kvm[@]}" "${x86_64[@]}") + ;; + qemu-system-riscv64) + kvm=("${kvm[@]}" "${riscv64[@]}") + ;; + esac +} + run_qemu() { "${kvm[@]}" --append "${append}" @@ -110,8 +168,12 @@ get_rootfs check_rootfs qemu_command +options_value print_message -kvm_command +public_options +individual_options +merge_options + run_qemu -- 2.23.0

1 0

[PATCH v2 compass-ci 2/3] scheduler: idle-job: hand over idle job
by Ren Wen 28 Dec '20

28 Dec '20

When submitting an idle job, the job will be submitted to "queues/sched/<queue>/idle"; When consuming an idle job, the job will be moved to "queues/sched/<queue>/in_process"; It will hand over job from "queues/sched/<queue>/idle/in_process" before. It's wrong for idle job, so fix it. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/close_job.cr | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index 9b4da6a..bce6b33 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -14,8 +14,11 @@ class Sched raise "es set job content fail!" end + subqueue = job.subqueue + queue = (subqueue == "idle" ? job.queue : "#{job.queue}/#{subqueue}") + response = @task_queue.hand_over_task( - "sched/#{job.queue}/#{job.subqueue}", "extract_stats", job_id + "sched/#{queue}", "extract_stats", job_id ) if response[0] != 201 raise "#{response}" -- 2.23.0

1 0

[PATCH v2 compass-ci 1/3] scheduler: idle-job: fix queue when submitting job
by Ren Wen 28 Dec '20

28 Dec '20

[Before]: 1) job has a field 'idle_job' when submitting idle job. 2) field 'uuid' is used with remote user to do a rate limit. [Now]: 1) idle job has a subqueue 'idle'. 2) use 'my_email' instead of 'uuid'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index c322d0f..c09d145 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -114,13 +114,6 @@ class Sched def submit_single_job(job) queue = "#{job.queue}/#{job.subqueue}" - # only single job will has "idle job" and "execute rate limiter" - if job["idle_job"].empty? - queue += "#{job.get_uuid_tag}" - else - queue = "#{queue}/idle" - end - job_id = add_task(queue, job.lab) return [{ "job_id" => "0", -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/es_query.rb: support order by keyword with desc
by Lu Weitao 28 Dec '20

28 Dec '20

[Why] user need get latest doc in ES [Sample] caller(es-find | compare | web-backend/compare | ...) use like: es = ESQuery.new(index: 'jobs') query_result = es.multi_field_query(items, desc_keyword: 'start_time') Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/es_query.rb | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 3bea624..10b9644 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -18,7 +18,7 @@ class ESQuery # Example @items: { key1 => value1, key2 => [value2, value3, ..], ...} # means to query: key1 == value1 && (key2 in [value2, value3, ..]) - def multi_field_query(items, size: 10_000) + def multi_field_query(items, size: 10_000, desc_keyword: nil) unless items warn 'empty filter!' exit @@ -31,6 +31,7 @@ class ESQuery } }, size: size } + query.merge!(assign_desc_body(desc_keyword)) if desc_keyword @client.search index: @index + '*', body: query end @@ -112,3 +113,11 @@ def parse_conditions(items) end items_hash end + +def assign_desc_body(keyword) + { + sort: [{ + keyword => { order: 'desc' } + }] + } +end -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/build-pkg: fix directory cannot access
by Liu Shaofei 28 Dec '20

28 Dec '20

[error] upload_config upload_to_target_dir /cci/build-config/vue . /lkp/lkp/src/lib/upload.sh . /lkp/lkp/src/lib/env.sh '[' -n 1 ']' return local dir=/cci/build-config/vue grep -v @ ls -F /cci/build-config/vue ls: cannot access '/cci/build-config/vue': No such file or directory Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 44bd1c10d..801649cf4 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -48,7 +48,10 @@ get_config() upload_config() { - upload_to_target_dir "${cci_dir}/${pkgname}" + local upload_dir="${cci_dir}/${pkgname}" + [ -d "${upload_dir}" ] || return + + upload_to_target_dir "${upload_dir}" } mount_dest() -- 2.23.0

1 0

[PATCH compass-ci] 3-code/git: add repository version pointing
by Hu Xuejiao 28 Dec '20

28 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/3-code/git | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sparrow/3-code/git b/sparrow/3-code/git index 2159ddc..48741ec 100755 --- a/sparrow/3-code/git +++ b/sparrow/3-code/git @@ -15,7 +15,7 @@ cd /c || exit git clone https://gitee.com/wu_fengguang/compass-ci.git cd /c/compass-ci || exit - [ -n "$compass_commitid" ] && git reset --hard "$compass_commitid" + [ -n "$compass_commit_id" ] && git reset --hard "$compass_commit_id" ln -s compass-ci /c/cci } @@ -24,9 +24,12 @@ cd /c || exit git clone https://gitee.com/wu_fengguang/lkp-tests.git cd /c/lkp-tests || exit - [ -n "$lkp_commitid" ] && git reset --hard "$lkp_commitid" + [ -n "$lkp_commit_id" ] && git reset --hard "$lkp_commit_id" } [ -d "upstream-repos/.git" ] || { git clone https://gitee.com/wu_fengguang/upstream-repos.git + + cd /c/upstream-repos || exit + [ -n "$upstream_commit_id" ] && git reset --hard "$upstream_commit_id" } -- 2.23.0

1 0

[PATCH compass-ci] os/debian: add apt-get update
by Hu Xuejiao 28 Dec '20

28 Dec '20

[why] the url for downloading the package is displayed as 404 Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/0-package/os/debian | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 6a58a0c..c45e8c2 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -22,4 +22,5 @@ pkgs=( hostname ) +apt-get update apt-get install -y --fix-missing "${pkgs[@]}" -- 2.23.0

1 0

[PATCH v3 lkp-tests] daemon: fix ssh login testbox for docker
by Xiao Shenwei 28 Dec '20

28 Dec '20

[why] systemctl is not supported when use docker Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- daemon/sshd | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 41fa99adf..c2404dbfd 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -5,20 +5,35 @@ # sshr_port_len # sshr_port_base -. $LKP_SRC/lib/http.sh +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/http.sh +. $PROJECT_DIR/lib/env.sh sshr_ip=$LKP_SERVER [ -n "$sshr_port" ] || sshr_port=5050 [ -n "$sshr_port_len" ] || sshr_port_len=2000 [ -n "$sshr_port_base" ] || sshr_port_base=50000 +generate_key_file() +{ + echo -e "y\n" | ssh-keygen -t rsa -b 2048 -N '' -f /etc/ssh/ssh_host_rsa_key +} + run_ssh() { [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys - systemctl start sshd + if is_docker; then + generate_key_file + sshd_path=$(cmd_path sshd) + $sshd_path -q + else + systemctl start sshd + fi } data_success() -- 2.23.0

1 0

[PATCH compass-ci 4/4] container: the start script for assist-result
by Cao Xueliang 28 Dec '20

28 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assist-result/start | 45 +++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 container/assist-result/start diff --git a/container/assist-result/start b/container/assist-result/start new file mode 100755 index 0000000..af812aa --- /dev/null +++ b/container/assist-result/start @@ -0,0 +1,45 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +names = Set.new %w[ + ES_HOST + ES_PORT + ASSIST_RESULT_HOST + ASSIST_RESULT_PORT +] + +defaults = relevant_defaults(names) +env = docker_env(defaults) + +DEFAULT_LKP = '/c/lkp-tests' +DEFAULT_CCI = '/c/compass-ci' +DEFAULT_CONFIG_DIR = '/etc/compass-ci/defaults' +ASSIST_RESULT_PORT = defaults['ASSIST_RESULT_PORT'] +docker_rm 'assist-result' + +cmd = %w[ + docker run + --name assist-result + --restart=always + -d +] + env + %W[ + -e LKP_SRC=#{DEFAULT_LKP} + -e CCI_SRC=#{DEFAULT_CCI} + -p #{ASSIST_RESULT_PORT}:#{ASSIST_RESULT_PORT} + -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} + -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} + -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro + -v /etc/localtime:/etc/localtime:ro + -v /srv/result:/srv/result:ro + -w #{DEFAULT_CCI}/container/assist-result/ + assist-result +] + +cmd += ['sh', '-c', 'umask 002 && ruby ./routes.rb'] + +system(*cmd) -- 2.23.0

1 1