- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib/job.cr: add reference url
by Wu Zhende 18 Dec '20

18 Dec '20

submit iperf.yaml testbox=vm-2p8g submit iperf.yaml failed, got job_id=0, error: Missing required job key: 'my_token'. Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco… Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e7b9faf..e046072 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -325,7 +325,11 @@ class Job private def check_required_keys REQUIRED_KEYS.each do |key| if !@hash[key]? - raise "Missing required job key: '#{key}'" + error_msg = "Missing required job key: '#{key}'." + if ["my_email", "my_name", "my_token"].includes?(key) + error_msg += "\nPlease refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" + end + raise error_msg end end end -- 2.23.0

1 0

[PATCH compass-ci] container/initrd-http: support "/srv/cci/" dir download
by Cao Xueliang 18 Dec '20

18 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/initrd-http/root/etc/nginx/conf.d/default.conf | 4 ++++ container/initrd-http/start | 1 + 2 files changed, 5 insertions(+) diff --git a/container/initrd-http/root/etc/nginx/conf.d/default.conf b/container/initrd-http/root/etc/nginx/conf.d/default.conf index f2834c7..18813a2 100644 --- a/container/initrd-http/root/etc/nginx/conf.d/default.conf +++ b/container/initrd-http/root/etc/nginx/conf.d/default.conf @@ -13,4 +13,8 @@ server { location /initrd { autoindex on; } + + location /cci { + autoindex on; + } } diff --git a/container/initrd-http/start b/container/initrd-http/start index ed5d443..8341050 100755 --- a/container/initrd-http/start +++ b/container/initrd-http/start @@ -14,6 +14,7 @@ cmd=( --name initrd-http -p ${INITRD_HTTP_PORT:-8800}:80 -v /srv/initrd:/usr/share/nginx/html/initrd:ro + -v /srv/cci:/usr/share/nginx/html/cci:ro -v /etc/localtime:/etc/localtime:ro -d initrd-http -- 2.23.0

1 0

[PATCH v2 compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 18 Dec '20

18 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 18 Dec '20

18 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 59625299..addff9f7 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -129,7 +129,7 @@ build_source_pkg() create_cgz_name PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 18 Dec '20

18 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dd1ee120..59625299 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -128,8 +128,8 @@ build_source_pkg() create_cgz_name - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: rename my_uuid to my_token
by Wu Zhende 18 Dec '20

18 Dec '20

my_token is more expressive Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d44559..fa8e2dc 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -319,7 +319,7 @@ class Job testbox my_email my_name - my_uuid + my_token ] private def check_required_keys @@ -340,9 +340,10 @@ class Job raise error_msg if @account_info["found"]? == false raise error_msg unless self["my_name"] == @account_info["my_name"].to_s - raise error_msg unless self["my_uuid"] == @account_info["my_uuid"] + raise error_msg unless self["my_token"] == @account_info["my_token"] @hash.delete("my_uuid") + @hash.delete("my_token") end private def get_initialized_keys @@ -366,7 +367,7 @@ class Job "SCHED_HOST", "SCHED_PORT"] - initialized_keys -= ["my_uuid", + initialized_keys -= ["my_token", "kernel_version", "kernel_uri", "kernel_params", -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..dd1ee120f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -97,7 +97,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..9cb51ff6f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -17,7 +17,6 @@ ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, ## make customizations, generate meta-info, and package everything together. ## See: https://wiki.archlinux.org/index.php/Makepkg - . $LKP_SRC/lib/debug.sh . $LKP_SRC/distro/common @@ -97,7 +96,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 1

[PATCH compass-ci] fix: idle job for subqueue: submit idle jobs when finding no job
by Ren Wen 17 Dec '20

17 Dec '20

[Processes] There're 2 types of queues to consume: 1. user queue : sched/<taishan-a12>/<my_email> 2. idle queue : sched/<taishan-a12>/idle Consume Queue 1 at first, next Queue2 if no job found in Queue1. If there's no job in all Queues, auto submit idle jobs to idle queue. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 7565375..567016e 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -51,31 +51,38 @@ class Sched end def get_queues(host) - queues = [] of String + default_queues = [] of String + sub_queues = [] of String queues_str = @redis.hash_get("sched/host2queues", host) - return queues unless queues_str + return default_queues, sub_queues unless queues_str queues_str.split(',', remove_empty: true) do |item| - queues << item.strip + default_queues << item.strip end - sub_queues = [] of String - queues.each do |queue| - matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") + default_queues.each do |queue| + matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") matched_queues.each do |mq| match_data = "#{mq}".match(%r(^#{QUEUE_NAME_BASE}/sched/(#{queue}/.+)/ready$)) sub_queues << match_data[1] if match_data end end - return rand_queues(sub_queues) + return default_queues, sub_queues end def get_job_from_queues(queues, testbox) + default_queues, sub_queues = queues + all_queues = rand_queues(sub_queues) + + default_queues.each do |queue| + all_queues << "#{queue}/idle" + end + job = nil - queues.each do |queue| + all_queues.each do |queue| job = prepare_job("sched/#{queue}", testbox) return job if job end @@ -89,6 +96,8 @@ class Sched if job create_job_cpio(job.dump_to_json_any, Kemal.config.public_folder) + else + spawn { auto_submit_idle_job(host) } end return boot_content(job, boot_type) -- 2.23.0

1 0

[PATCH compass-ci] src/lib/web_backend.rb remove non-existent os_version
by Lu Weitao 17 Dec '20

17 Dec '20

remove non-existent os_version of compare_candidates Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 07e2e9e..cacafde 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -97,7 +97,7 @@ def compare_candidates_body suite: all_suite, OS: [ { os: 'openeuler', os_version: ['1.0', '20.03'] }, - { os: 'centos', os_version: ['7.6', '7.8', '8.1', 'sid'] }, + { os: 'centos', os_version: ['7.6', '7.8', '8.1'] }, { os: 'debian', os_version: %w[10 sid] }, { os: 'archlinux', os_version: ['5.5.0-1'] } ], -- 2.23.0

1 0