December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] lib/job.rb: fix job symbol key check
by Wei Jihui 24 Dec '20

24 Dec '20

load job from yaml maybe has symbol key symbol key does not has start_with function. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 7553312e..35e0ceee 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -254,7 +254,6 @@ class Job @overrides.merge!(hash['override']){ |_key, a, _b| a} hash.delete('override') end - hash.delete_if { |key, _| key.start_with?('#!') } hash.merge!(@overrides) @jobs.concat(multi_args(hash)) # return [hash] or [h1,h2] end @@ -290,6 +289,12 @@ class Job jobs_array = [] hash.each { |key, value| next unless key =~ /^\w.*\|.*\w$/ + next unless key.is_a?(String) + + if key.start_with?('#! ') + hash.delete(key) + next + end key_array = key.split('|').map(&:strip) [value].flatten.each do |v| -- 2.23.0

3 3

[PATCH v2 lkp-tests] jobs: optimize yaml file
by Liu Shaofei 23 Dec '20

23 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- jobs/build-pkg.yaml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/jobs/build-pkg.yaml b/jobs/build-pkg.yaml index b17fd2ade..7a47bd724 100644 --- a/jobs/build-pkg.yaml +++ b/jobs/build-pkg.yaml @@ -1,9 +1,8 @@ -suite: build-pkg -category: functional - -# pkgbuild_repo examples +# the field of "pkgbuild_repo" refers to the path of PKGBUILD, the examples are as follow: # pkgbuild_repo: archlinux/aur-j/java8-openjdk # pkgbuild_repo: archlinux/community//keepalived/trunk # pkgbuild_repo: archlinux/packages//prometheus-node-exporter/trunk +suite: build-pkg +category: functional build-pkg: -- 2.23.0

1 0

[PATCH compass-ci] container/sub-fluentd: the dependency file is missing
by Hu Xuejiao 23 Dec '20

23 Dec '20

[why] container is failed to start in test-machine because of missing file Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- container/sub-fluentd/build-depends | 1 + 1 file changed, 1 insertion(+) create mode 100755 container/sub-fluentd/build-depends diff --git a/container/sub-fluentd/build-depends b/container/sub-fluentd/build-depends new file mode 100755 index 0000000..29ee9d3 --- /dev/null +++ b/container/sub-fluentd/build-depends @@ -0,0 +1 @@ +fluentd-base -- 2.23.0

1 0

[PATCH lkp-tests] sbin/run_job: Automatic task execution
by Hu Xuejiao 23 Dec '20

23 Dec '20

[purpose] Verify that a local job can be consumed properly Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/run_job | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100755 sbin/run_job diff --git a/sbin/run_job b/sbin/run_job new file mode 100755 index 00000000..2b02eb77 --- /dev/null +++ b/sbin/run_job @@ -0,0 +1,76 @@ +#!/bin/bash +# tbox_group +# testbox +# nr_vm +# nr_container +# queues + +cci_makepkg_file="$LKP_SRC/jobs/cci-makepkg.yaml" +grep "atomic" $cci_makepkg_file || sed -i 's|benchmark:|& atomic|g' $cci_makepkg_file + +submit_one_job() +{ + cmd=( + submit -m $yaml + os=${single_arg[0]} + os_arch=${single_arg[1]} + os_version=${single_arg[2]} + os_mount=${single_arg[3]} + testbox=$testbox + queue="$tbox_group.$HOSTNAME" + ) + output=$("${cmd[@]}") + echo "$output" | grep -o -E 'job_state=finished[^\"]*' +} + +submit_job() +{ + yaml_args=("$cci_makepkg_file" "host-info.yaml") + for yaml in "${yaml_args[@]}" + do + submit_one_job & + done +} + +run_qemu() +{ + local nr_vm=4 + local testbox="vm-2p8g" + local tbox_group="vm-2p8g" + local queues="$tbox_group.$HOSTNAME" + + . $LKP_SRC/daemon/multi-qemu + + qemu_args=( + "openeuler aarch64 20.03 cifs" + "centos aarch64 7.6.1810 initramfs" + ) + for qemu_arg in "${qemu_args[@]}" + do + local single_arg=($(echo $qemu_arg)) + submit_job + done +} + +run_docker() +{ + local nr_container=2 + local tbox_group="dc-8g" + local testbox="dc-8g" + local queues="$tbox_group.$HOSTNAME" + + . $LKP_SRC/daemon/multi-docker + + docker_args=( + "centos aarch64 7.6.1810 container" + ) + for docker_arg in "${docker_args[@]}" + do + local single_arg=($(echo $docker_arg)) + submit_job + done +} + +run_qemu +run_docker +wait -- 2.23.0

1 0

[PATCH v3 compass-ci] docker-rootf: add different install methods
by Wang Chenglong 23 Dec '20

23 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/setup-image | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/container/docker-rootfs/setup-image b/container/docker-rootfs/setup-image index 787a42c..05e5a24 100755 --- a/container/docker-rootfs/setup-image +++ b/container/docker-rootfs/setup-image @@ -4,8 +4,15 @@ install_packages() { - yum install -y --skip-broken $(</tmp/packages-to-install) - yum clean all + if [ $(command -v yum) >/dev/null ]; then + yum install -y --skip-broken $(</tmp/packages-to-install) + yum clean all + elif [ $(command -v apt-get) >/dev/null ]; then + apt update + apt-get install -y --fix-missing $(</tmp/packages-to-install) + elif [ $(command -v pacman) >/dev/null ]; then + pacman -Sy --noconfirm $(</tmp/packages-to-install) + fi } remove_file() -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: fix job symbol key check
by Wei Jihui 23 Dec '20

23 Dec '20

load job from yaml maybe has symbol key symbol key does not has start_with function. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 7553312e..5ae67e6b 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -254,7 +254,6 @@ class Job @overrides.merge!(hash['override']){ |_key, a, _b| a} hash.delete('override') end - hash.delete_if { |key, _| key.start_with?('#!') } hash.merge!(@overrides) @jobs.concat(multi_args(hash)) # return [hash] or [h1,h2] end @@ -289,7 +288,12 @@ class Job def multi_args(hash) jobs_array = [] hash.each { |key, value| - next unless key =~ /^\w.*\|.*\w$/ + next unless key =~ /^\w.*\|.*\w$/ || key.is_a?(String) + + if key.start_with?('#! ') + hash.delete(key) + next + end key_array = key.split('|').map(&:strip) [value].flatten.each do |v| -- 2.23.0

1 0

[PATCH v2 compass-ci] docker-rootf: add different install methods
by Wang Chenglong 23 Dec '20

23 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/setup-image | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/container/docker-rootfs/setup-image b/container/docker-rootfs/setup-image index 787a42c..7274e2e 100755 --- a/container/docker-rootfs/setup-image +++ b/container/docker-rootfs/setup-image @@ -4,8 +4,15 @@ install_packages() { - yum install -y --skip-broken $(</tmp/packages-to-install) - yum clean all + if [ $(command -v yum) ]; then + yum install -y --skip-broken $(</tmp/packages-to-install) + yum clean all + elif [ $(command -v apt-get) ]; then + apt update + apt-get install -y --fix-missing $(</tmp/packages-to-install) + elif [ $(command -v pacman) ]; then + pacman -Sy --noconfirm $(</tmp/packages-to-install) + fi } remove_file() -- 2.23.0

1 0

[PATCH compass-ci] docker-rootf: add different install methods
by Wang Chenglong 23 Dec '20

23 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/setup-image | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/container/docker-rootfs/setup-image b/container/docker-rootfs/setup-image index 787a42c..6a121d3 100755 --- a/container/docker-rootfs/setup-image +++ b/container/docker-rootfs/setup-image @@ -4,8 +4,15 @@ install_packages() { - yum install -y --skip-broken $(</tmp/packages-to-install) - yum clean all + if [ $(command -v yum) ]; then + yum install -y --skip-broken $(</tmp/packages-to-install) + yum clean all + elif [ $(command -v apt-get) ]; then + apt update + apt-get install -y --no-install-recommends $(</tmp/packages-to-install) + elif [ $(command -v pacman) ]; then + pacman -Sy --noconfirm $(</tmp/packages-to-install) + fi } remove_file() -- 2.23.0

2 2

[PATCH lkp-tests] lkp-tests: delete useless file
by Wei Jihui 23 Dec '20

23 Dec '20

install.sh is useless now install method is: make install && source ~/.${SHELL##*/}rc Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- install.sh | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 install.sh diff --git a/install.sh b/install.sh deleted file mode 100644 index cea1654f..00000000 --- a/install.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash - -. $LKP_SRC/lib/env.sh - -make - -source $(shell_profile) -- 2.23.0

1 0

[PATCH compass-ci] scheduler: fix block error
by Ren Wen 23 Dec '20

23 Dec '20

[why] Every request to scheduler will initialize a new 'Sched', each 'Sched' will new a 'BlockHelper'. For each node in cluster, it can't release when blocking at scheduler because they use different locks. [how] Set an instance of 'BlockHelper' to class variable. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 3 ++- src/scheduler/request_cluster_state.cr | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index f7220f9..25e3708 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -28,11 +28,12 @@ class Sched property redis property block_helper + @@block_helper = BlockHelper.new + def initialize(env : HTTP::Server::Context) @es = Elasticsearch::Client.new @redis = Redis::Client.new @task_queue = TaskQueueAPI.new - @block_helper = BlockHelper.new @rgc = RemoteGitClient.new @env = env @log = env.log.as(JSONLogger) diff --git a/src/scheduler/request_cluster_state.cr b/src/scheduler/request_cluster_state.cr index 1782797..0703792 100644 --- a/src/scheduler/request_cluster_state.cr +++ b/src/scheduler/request_cluster_state.cr @@ -25,7 +25,7 @@ class Sched update_cluster_state(cluster_id, job_id, {"state" => states[request_state]}) when "wait_ready" update_cluster_state(cluster_id, job_id, {"state" => states[request_state]}) - @block_helper.block_until_finished(cluster_id) { + @@block_helper.block_until_finished(cluster_id) { cluster_state = sync_cluster_state(cluster_id, job_id, states[request_state]) cluster_state == "ready" || cluster_state == "abort" } -- 2.23.0

1 0