- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] result-webdav/nginx.conf: create soft connection after package upload
by Cui Yili 19 Dec '20

19 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/nginx.conf | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/container/result-webdav/nginx.conf b/container/result-webdav/nginx.conf index 8b54314..f077fca 100644 --- a/container/result-webdav/nginx.conf +++ b/container/result-webdav/nginx.conf @@ -37,6 +37,25 @@ http { end) return resultstrlist end + function symbolic_link(path, upload_file) + level_dir = split(path, "/") + if level_dir[1] ~= "initrd" or upload_file == nil then + return + end + if level_dir[2] == "pkg" or level_dir[2] == "build-pkg" then + link_name = "latest" + else + tmp_name = string.match(upload_file, "(.*)_%d+%.cgz") + if not tmp_name then + link_name = tmp_name + else + return + end + end + link_name = link_name..".cgz" + root_path = "/srv/"..path + io.popen("cd "..root_path.."&& ln -sf "..upload_file.." "..link_name) + end redis_host = os.getenv("REDIS_HOST") redis_port = os.getenv("REDIS_PORT") @@ -65,11 +84,13 @@ http { dirs = split(res, ",") for i = 1, #dirs do if string.find(path, dirs[i], 1, true) then + symbolic_link(path, upload_file) return end end else if string.find(path, res, 1, true) then + symbolic_link(path, upload_file) return end end -- 2.23.0

1 0

[PATCH compass-ci] lib/error_messages.rb: fix error when file is not existed
by Lin Jiaxin 19 Dec '20

19 Dec '20

Traceback (most recent call last): 4: from compare_error_messages.rb:63:in `<main>' 3: from compare_error_messages.rb:20:in `get_compare_result' 2: from /home/linjiaxin/compass-ci/lib/error_messages.rb:33:in `obtain_error_messages' 1: from /home/linjiaxin/compass-ci/lib/error_messages.rb:33:in `open' /home/linjiaxin/compass-ci/lib/error_messages.rb:33:in `initialize': No such file or directory @ rb_sysopen - /srv/result/build-pkg/2020-12-16/vm-2p32g.linjiaxin/openeuler-20.03-aarch64/archlinux-community--fcgi-trunk-HEAD/crystal.193087/build-pkg (Errno::ENOENT) Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/error_messages.rb | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/error_messages.rb b/lib/error_messages.rb index 5bbc329..61f8070 100644 --- a/lib/error_messages.rb +++ b/lib/error_messages.rb @@ -30,12 +30,14 @@ class ErrorMessages end def obtain_error_messages - File.open((a)file).each_line do |line| - next if extract_error_message(line) - next unless @in_stderr - next unless @error_message['error_line'] =~ /error|warning/i + if FileTest.exist?(@file) + File.open((a)file).each_line do |line| + next if extract_error_message(line) + next unless @in_stderr + next unless @error_message['error_line'] =~ /error|warning/i - @error_messages[@error_message['error_line']] << @error_message['error_line'] + @error_message['error_details'] + @error_messages[@error_message['error_line']] << @error_message['error_line'] + @error_message['error_details'] + end end return @error_messages end -- 2.23.0

1 0

[PATCH v2 compass-ci] lib: add compare_error_messages.rb
by Lin Jiaxin 19 Dec '20

19 Dec '20

Provide formatted error messages and new error number for git bisect. formatted error messages example: previous error: fcgiapp.c:1717:22: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare] 1717 | if(headerLen < sizeof(header)) { | ^ new error: >> libtool: warning: relinking 'libfcgi++.la' Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 59 +++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 lib/compare_error_messages.rb diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb new file mode 100644 index 0000000..0b51fc3 --- /dev/null +++ b/lib/compare_error_messages.rb @@ -0,0 +1,59 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'terminal-table' +require_relative 'es_query' +require_relative 'error_messages' +require_relative "#{ENV['LKP_SRC']}/lib/common" + +def get_compare_result(previous_job_id, later_job_id) + es = ESQuery.new + + previous_error_ids = es.query_by_id(previous_job_id)['error_ids'] + later_es_result = es.query_by_id(later_job_id) + later_error_ids = later_es_result['error_ids'] + later_result_file = File.join('/srv', later_es_result['result_root'], 'build-pkg') + + new_error_ids = get_new_error_ids(later_error_ids, previous_error_ids) + error_messages = ErrorMessages.new(later_result_file).obtain_error_messages + new_error_number, formatted_error_messages = format_error_messages(error_messages, new_error_ids) + + # new error use ">>" as an identifier in the error header + return new_error_number, formatted_error_messages +end + +def get_new_error_ids(later_error_ids, previous_error_ids) + return nil_to_empty_array(later_error_ids) - nil_to_empty_array(previous_error_ids) +end + +def nil_to_empty_array(array) + array = [] if array.nil? + return array +end + +def format_error_messages(error_messages, new_error_ids) + formatted_error_messages = Terminal::Table.new + formatted_error_messages.style = { border_x: '', border_y: '', border_i: '', padding_left: 0 } + new_error_number = 0 + + error_messages.each do |k, v| + if new_error_ids.include?('build-pkg.' + build_pkg_error_id(k)) + new_error_number += 1 + formatted_error_messages = add_sign(formatted_error_messages, '>>', v) + else + formatted_error_messages = add_sign(formatted_error_messages, ' ', v) + end + end + + return new_error_number, formatted_error_messages.to_s +end + +def add_sign(formatted_error_messages, sign, set) + set.each do |value| + formatted_error_messages.add_row([sign, value]) + end + + return formatted_error_messages +end -- 2.23.0

1 0

[PATCH compass-ci] lib: add compare_error_messages.rb
by Lin Jiaxin 19 Dec '20

19 Dec '20

Provide formatted error messages and new error number for git bisect. formatted error messages example: previous error: fcgiapp.c:1717:22: warning: comparison of integer expressions of different signedness: ‘int’ and ‘long unsigned int’ [-Wsign-compare] 1717 | if(headerLen < sizeof(header)) { | ^ new error: >> libtool: warning: relinking 'libfcgi++.la' Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 50 +++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 lib/compare_error_messages.rb diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb new file mode 100644 index 0000000..c96edef --- /dev/null +++ b/lib/compare_error_messages.rb @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'terminal-table' +require_relative 'es_query' +require_relative 'error_messages' +require_relative "#{ENV['LKP_SRC']}/lib/common" + +def get_compare_result(previous_job_id, later_job_id) + es = ESQuery.new + + previous_error_ids = es.query_by_id(previous_job_id)['error_ids'] + later_es_result = es.query_by_id(later_job_id) + later_error_ids = later_es_result['error_ids'] + later_result_file = File.join('/srv', later_es_result['result_root'], 'build-pkg') + + new_error_ids = later_error_ids - previous_error_ids + error_messages = ErrorMessages.new(later_result_file).obtain_error_messages + new_error_number, formatted_error_messages = format_error_messages(error_messages, new_error_ids) + + # new error use ">>" as an identifier in the error header + return new_error_number, formatted_error_messages +end + +def format_error_messages(error_messages, new_error_ids) + formatted_error_messages = Terminal::Table.new + formatted_error_messages.style = { border_x: '', border_y: '', border_i: '', padding_left: 0 } + new_error_number = 0 + + error_messages.each do |k, v| + if new_error_ids.include?('build-pkg.' + build_pkg_error_id(k)) + new_error_number += 1 + formatted_error_messages = add_sign(formatted_error_messages, '>>', v) + else + formatted_error_messages = add_sign(formatted_error_messages, ' ', v) + end + end + + return new_error_number, formatted_error_messages.to_s +end + +def add_sign(formatted_error_messages, sign, set) + set.each do |value| + formatted_error_messages.add_row([sign, value]) + end + + return formatted_error_messages +end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/2] container: http-initrd container support job read /srv/cci/build-config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/initrd-http/root/etc/nginx/conf.d/default.conf | 4 ++++ container/initrd-http/start | 1 + 2 files changed, 5 insertions(+) diff --git a/container/initrd-http/root/etc/nginx/conf.d/default.conf b/container/initrd-http/root/etc/nginx/conf.d/default.conf index f2834c7..18813a2 100644 --- a/container/initrd-http/root/etc/nginx/conf.d/default.conf +++ b/container/initrd-http/root/etc/nginx/conf.d/default.conf @@ -13,4 +13,8 @@ server { location /initrd { autoindex on; } + + location /cci { + autoindex on; + } } diff --git a/container/initrd-http/start b/container/initrd-http/start index ed5d443..8341050 100755 --- a/container/initrd-http/start +++ b/container/initrd-http/start @@ -14,6 +14,7 @@ cmd=( --name initrd-http -p ${INITRD_HTTP_PORT:-8800}:80 -v /srv/initrd:/usr/share/nginx/html/initrd:ro + -v /srv/cci:/usr/share/nginx/html/cci:ro -v /etc/localtime:/etc/localtime:ro -d initrd-http -- 2.23.0

1 0

[PATCH v5 compass-ci 1/2] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/build-config dir write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..2852f24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/build-config/:/srv/cci/build-config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..4d0b5df 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,33 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? + mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + if @hash["pkgbuild_repo"].to_s =~ /(packages|community)\/\// + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-2] + else + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] + end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" + package_dir += ",/cci/build-config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 0

[PATCH v4 compass-ci 2/2] container: http-initrd container support job write cci/build-config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/initrd-http/root/etc/nginx/conf.d/default.conf | 4 ++++ container/initrd-http/start | 1 + 2 files changed, 5 insertions(+) diff --git a/container/initrd-http/root/etc/nginx/conf.d/default.conf b/container/initrd-http/root/etc/nginx/conf.d/default.conf index f2834c7..546e824 100644 --- a/container/initrd-http/root/etc/nginx/conf.d/default.conf +++ b/container/initrd-http/root/etc/nginx/conf.d/default.conf @@ -13,4 +13,8 @@ server { location /initrd { autoindex on; } + + location /build-config { + autoindex on; + } } diff --git a/container/initrd-http/start b/container/initrd-http/start index ed5d443..ac8506e 100755 --- a/container/initrd-http/start +++ b/container/initrd-http/start @@ -14,6 +14,7 @@ cmd=( --name initrd-http -p ${INITRD_HTTP_PORT:-8800}:80 -v /srv/initrd:/usr/share/nginx/html/initrd:ro + -v /srv/cci/build-config:/usr/share/nginx/html/build-config:ro -v /etc/localtime:/etc/localtime:ro -d initrd-http -- 2.23.0

1 0

[PATCH v4 compass-ci 1/2] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/build-config dir write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..2852f24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/build-config/:/srv/cci/build-config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..4d0b5df 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,33 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? + mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + if @hash["pkgbuild_repo"].to_s =~ /(packages|community)\/\// + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-2] + else + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] + end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" + package_dir += ",/cci/build-config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 0

[PATCH compass-ci] doc: update documents
by Luan Shengde 19 Dec '20

19 Dec '20

update documents: my_uuid => my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/mail-robot.md | 6 +++--- doc/manual/apply-account.md | 2 +- ...43\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index 06e4e87..1e8d618 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -58,14 +58,14 @@ attahcment first attachment - apply_my_account - my_uuid - generate uuid + my_token + generate my_token apply_account invoke ApplyJumperAccount to apply new account with my_info and pub_key my_info: - my_email - my_name - - my_uuid + - my_token apply_jumper_account required data: pub_key complete my_info diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index d9a03c8..8ae6c6d 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -25,7 +25,7 @@ Which contains account information for you: my_email my_name - my_uuid + my_token ## step 3: one-time setup diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 5a02431..160fdd0 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -14,7 +14,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 ```SHELL mkdir -p ~/.config/compass-ci/defaults/ cat >> ~/.config/compass-ci/defaults/${USER}.yaml <<-EOF - my_uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx + my_token: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx my_email: xxx my_name: xx EOF -- 2.23.0

2 2

[PATCH v3 compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/build-config dir write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..2852f24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/build-config/:/srv/cci/build-config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..d5e8ad5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,33 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? + mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + if @hash["pkgbuild_repo"].to_s =~ /(packages|community)\/\// + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-2] + else + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] + end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 1

[PATCH v2 compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/build-config dir write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..d5e8ad5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,33 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? + mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + if @hash["pkgbuild_repo"].to_s =~ /(packages|community)\/\// + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-2] + else + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] + end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 1

[PATCH compass-ci] modify options for auto submit api-avx2neon.yaml
by Li Ping 19 Dec '20

19 Dec '20

Signed-off-by: Li Ping <1477412247(a)qq.com> --- sbin/auto_submit.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index d0f4fbe..e1eb39b 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -1,5 +1,5 @@ a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 api-avx2neon.yaml +- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- 2.23.0

1 0

[PATCH v1 compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/config dir read and write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 19 ++++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..85159f5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,27 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? or @hash["cci-depends"]? or @hash["build-pkg"]? + common_dir = "#{os_mount}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + package_dir = ",/initrd/build-pkg/#{common_dir}/#{@hash["pkgbuild_repo"].split("/")[-1]}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

2 3

[PATCH lkp-tests] lib/monitor: get host from job
by Wu Zhende 19 Dec '20

19 Dec '20

If @job is exist and monitor_url is not configured, set monitor_url based on SCHED_HOST Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index f65cfccf1..283cd788d 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -24,7 +24,6 @@ class Monitor # 1 means timeout exit @exit_status_code = 0 @defaults = {} - load_default @result = [] @stop_query = {} @reason = nil @@ -33,6 +32,10 @@ class Monitor def load_default return unless @monitor_url == '' + if host = @job["SCHED_HOST"] + return @monitor_url = "ws://#{host}:11310/filter" + end + Dir.glob(['/etc/compass-ci/monitor/*.yaml', "#{ENV['HOME']}/.config/compass-ci/monitor/*.yaml"]).each do |file| next unless File.exist? file @@ -98,6 +101,7 @@ class Monitor def run(timeout = nil) merge_overrides + load_default field_check @query.each do |k, v| -- 2.23.0

1 0

[PATCH compass-ci] modify assign-account-email
by Li Ping 19 Dec '20

19 Dec '20

[why]: source /root/.bashrc && /root/.bash_profile follow this step can not success. Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 77ed4e8..dccbd40 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -37,7 +37,7 @@ def build_apply_account_email(my_info) git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ${HOME}/.bashrc && source ${HOME}/.bash_profile + source ~/.${SHELL##*/}rc 3) submit job reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md -- 2.23.0

2 1

[PATCH compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/config dir read and write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 17 ++++++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..5644ddf 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,25 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? or @hash["cci-depends"]? or @hash["build-pkg"]? + common_dir = "#{os_mount}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + package_dir = ",/initrd/build-pkg/#{common_dir}/#{@hash["pkgbuild_repo"].split("/")[-1]}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end + end - self["upload_dirs"] = "#{result_root}#{package_dir}" + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 1

[PATCH compass-ci 3/3] rename my_uuid to my_token
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/mail-robot.md | 6 +++--- doc/manual/apply-account.md | 4 ++-- sbin/build-my-info.rb | 6 +++--- sbin/es-accounts-mapping.sh | 2 +- src/delimiter/utils.rb | 4 ++-- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index 06e4e87..d3778e7 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -58,14 +58,14 @@ attahcment first attachment - apply_my_account - my_uuid + my_token generate uuid apply_account invoke ApplyJumperAccount to apply new account with my_info and pub_key my_info: - - my_email - my_name - - my_uuid + - my_email + - my_token apply_jumper_account required data: pub_key complete my_info diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index d9a03c8..7e1a6a6 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -23,9 +23,9 @@ Email template: Which contains account information for you: - my_email my_name - my_uuid + my_email + my_token ## step 3: one-time setup diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 73f7390..d75f767 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -12,12 +12,12 @@ print 'email: ' my_email = $stdin.echo = gets.chomp print 'name: ' my_name = $stdin.echo = gets.chomp -my_uuid = %x(uuidgen).chomp +my_token = %x(uuidgen).chomp my_info = { - 'my_email' => my_email, 'my_name' => my_name, - 'my_uuid' => my_uuid + 'my_email' => my_email, + 'my_token' => my_token } def store_account_info(my_info) diff --git a/sbin/es-accounts-mapping.sh b/sbin/es-accounts-mapping.sh index d5566dc..d04b0f3 100755 --- a/sbin/es-accounts-mapping.sh +++ b/sbin/es-accounts-mapping.sh @@ -18,7 +18,7 @@ else "my_commit_url": { "type": "keyword" }, - "my_uuid": { + "my_token": { "type": "keyword" }, "my_email": { diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index e87bbbe..41803f3 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -101,9 +101,9 @@ module Utils raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info job['suite'] = 'bisect' - job['my_uuid'] = account_info['my_uuid'] job['my_name'] = account_info['my_name'] job['my_email'] = account_info['my_email'] + job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id job.delete('error_ids') @@ -127,7 +127,7 @@ module Utils def create_bisect_log(git_dir) FileUtils.mkdir_p TMP_RESULT_ROOT unless File.exist? TMP_RESULT_ROOT - log_file = File.join(TMP_RESULT_ROOT, "bisect.log") + log_file = File.join(TMP_RESULT_ROOT, 'bisect.log') log_content = parse_bisect_log(git_dir) File.open(log_file, 'w') do |f| log_content.each { |line| f.puts(line) } -- 2.23.0

1 0

[PATCH compass-ci 2/3] assign-account-email: remove .bash_profile usage
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 77ed4e8..ee3b574 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -37,7 +37,7 @@ def build_apply_account_email(my_info) git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ${HOME}/.bashrc && source ${HOME}/.bash_profile + source ${HOME}/.${SHELL##*/}rc 3) submit job reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md -- 2.23.0

1 0

[PATCH compass-ci 1/3] doc/manual: improve install client document
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- ...ci\345\256\242\346\210\267\347\253\257.md" | 27 +++++++------------ 1 file changed, 10 insertions(+), 17 deletions(-) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 5a02431..dd115f7 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -2,32 +2,25 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客户端，通过本地安装 lkp-tests 可以手动提交测试任务 +## 申请帐号 + :exclamation: 前提条件：按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… 完成如下动作： - send apply account email - receive email from compass-ci(a)qq.com +- get reply email and follow its instructions to + - setup default config -## Getting started +## 下载安装 lkp-tests -1. setup default config +Run the following command to install/setup lkp-test: - run the following command to add the below setup to default config file -```SHELL - mkdir -p ~/.config/compass-ci/defaults/ - cat >> ~/.config/compass-ci/defaults/${USER}.yaml <<-EOF - my_uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx - my_email: xxx - my_name: xx - EOF -``` -2. download lkp-tests and dependencies - - run the following command to install the lkp-test and effect the configuration ```SHELL git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ~/.bashrc && source ~/.bash_profile + source ~/.${SHELL##*/}rc ``` -3. submit job - Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +## submit job + +Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

1 0

[PATCH compass-ci] fetch-mail: update Dockerfile
by Luan Shengde 19 Dec '20

19 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile index 0434e09..99abf86 100644 --- a/container/fetch-mail/Dockerfile +++ b/container/fetch-mail/Dockerfile @@ -5,6 +5,7 @@ FROM centos:latest MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive +RUN groupadd -g 1147 mailer RUN useradd -u 1144 -g 1147 mailer RUN yum update -y RUN yum install -y procmail fetchmail -- 2.23.0

1 0

[PATCH lkp-tests] daemon: fix ssh login testbox for docker
by Xiao Shenwei 18 Dec '20

18 Dec '20

[why] systemctl is not supported when use docker [how] use the sshd command Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- daemon/sshd | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/daemon/sshd b/daemon/sshd index 41fa99adf..6bfda467e 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -6,19 +6,31 @@ # sshr_port_base . $LKP_SRC/lib/http.sh +. $LKP_SRC/lib/env.sh sshr_ip=$LKP_SERVER [ -n "$sshr_port" ] || sshr_port=5050 [ -n "$sshr_port_len" ] || sshr_port_len=2000 [ -n "$sshr_port_base" ] || sshr_port_base=50000 +generate_key_file() +{ + echo -e "y\n" | ssh-keygen -t rsa -b 2048 -N '' -f /etc/ssh/ssh_host_rsa_key +} + run_ssh() { [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys - systemctl start sshd + if is_docker; then + generate_key_file + sshd_path=$(cmd_path sshd) + $sshd_path -q + else + systemctl start sshd + fi } data_success() -- 2.23.0

1 0

[PATCH compass-ci] providers/docker: set the private key of the testbox
by Xiao Shenwei 18 Dec '20

18 Dec '20

[why] ssh password-free login for testbox of docker [how] Add ssh private key for docker, for automatically establish the ssh reverse tunnel between sut and port map container. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 63cb605..feb5edf 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -2,4 +2,6 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /root/.ssh +cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap -- 2.23.0

1 0

help star
by Wu Fengguang 18 Dec '20

18 Dec '20

Wei Jihui is our help star, thanks Wei Jihui! 24.0 Wu Fengguang 22.8 Wei Jihui 21.4 Yu Chuan 18.7 Cao Xueliang 12.0 Wu Zhende 10.6 Xiao Shenwei 9.2 Lu Weitao 8.5 Hu Xuejiao 7.8 Wang Yong 7.4 Liu Shaofei 6.4 Luan Shengde 5.9 Xu Xijian 5.7 Cui Yili 5.3 Zhang Yu 5.3 Zhang Dewan 5.2 Zhang Yale 5.0 Bai Jing 4.8 Liu Yinsi 4.4 Lin Jiaxin 4.3 Li Ping 4.0 Wang Chenglong 2.7 Sun Yukui 2.6 Lu Kaiyi 2.4 Ren Wen 2.2 Du Kaitian 1.3 Li Yuanchao 0.0 Tong Qunfeng 0.0 Wang Shuai 0.0 Cui Xucui

1 0

git count
by Wu Fengguang 18 Dec '20

18 Dec '20

Shengde is our code star again, thanks Shengde! wfg@crystal ~/todo% bin/git-count 14 Luan Shengde 13 Cao Xueliang 12 Wu Fengguang 10 Liu Yinsi 10 Sun Yukui 10 Wu Zhende 8 Ren Wen 7 Zhang Yu 7 Li Yuanchao 6 Cui Yili 6 Wang Chenglong 6 Lu Kaiyi 5 Liu Shaofei 5 Lu Weitao 5 Wei Jihui 5 Yu Chuan 5 Wang Yong 4 Hu Xuejiao 3 Bai Jing 3 Xu Xijian 3 Zhang Yale 3 Lin Jiaxin 3 Li Ping 2 Xiao Shenwei 0 Tong Qunfeng 0 Du Kaitian 0 Zhang Dewan

1 0

review star
by Wu Fengguang 18 Dec '20

18 Dec '20

Xueliang is our review star, thanks Xueliang! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 121 21 24 24 Cao Xueliang 48 30 57 20 Wu Fengguang 74 16 23 15 Wei Jihui 69 7 20 10 Luan Shengde 40 13 11 7 Liu Shaofei 42 14 20 7 Liu Yinsi 29 4 13 5 Yu Chuan 8 2 6 4 Xu Xijian 36 8 7 4 Lu Kaiyi 26 4 5 4 Xiao Shenwei 36 6 4 3 Zhang Yu 22 6 7 3 Lin Jiaxin 13 4 12 2 Sun Yukui 21 5 12 2 Ren Wen 14 4 2 2 Cui Yili 6 2 1 1 Bai Jing 18 3 2 1 Hu Xuejiao 30 4 4 1 Wang Chenglong 38 1 1 1 Wu Zhende 7 2 0 0 Zhang Yale 15 3 4 0 Li Ping 9 0 0 0 Li Yuanchao 6 2 2 0 Zhang Dewan 10 0 0 0 Wang Yong 25 2 4 0 Lu Weitao

1 0

[PATCH compass-ci] lib/es_query: add query filed from index
by Lu Weitao 18 Dec '20

18 Dec '20

[Why] We need only get specific field instead of entire document. it's like SQL: SELECT DISTINCT column FROM table Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/es_query.rb | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index 3bea624..86c078f 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -61,6 +61,30 @@ class ESQuery return result[0]['_source'] end + + # select doc_field from index + # input: + # eg: suite (@index: jobs) + # output: + # [ + # {"key"=>"build-pkg", "doc_count"=>90841}, + # {"key"=>"cci-depends", "doc_count"=>4636}, + # {"key"=>"cci-makepkg", "doc_count"=>3647}, + # ... + # ] + def query_specific_fields(field) + query = { + aggs: { + "all_#{field}" => { + terms: { field: field, size: 1000 } + } + }, + size: 0 + } + result = @client.search index: @index + '*', body: query + + result ['aggregations']["all_#{field}"]['buckets'] + end end # Range Query Example: -- 2.23.0

1 1

[PATCH compass-ci] mail-robot/assign-account-email: add config for lab
by Wei Jihui 18 Dec '20

18 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 834e96f..77ed4e8 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -24,6 +24,7 @@ def build_apply_account_email(my_info) cat >> ~/.config/compass-ci/defaults/account.yaml <<-EOF my_email: #{my_info['my_email']} my_name: #{my_info['my_name']} + lab: #{ENV['lab']} EOF mkdir -p ~/.config/compass-ci/include/lab cat >> ~/.config/compass-ci/include/lab/#{ENV['lab']}.yaml <<-EOF -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] distro/adaptation: add sshd mapping
by Wang Yong 18 Dec '20

18 Dec '20

add sshd mapping in different OS Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/archlinux | 2 ++ distro/adaptation/centos | 1 + distro/adaptation/fedora | 1 + distro/adaptation/kylin | 2 ++ distro/adaptation/openeuler | 2 ++ distro/adaptation/opensuse | 1 + distro/adaptation/oracle | 1 + 7 files changed, 10 insertions(+) diff --git a/distro/adaptation/archlinux b/distro/adaptation/archlinux index 53474d6c4..89163cf41 100644 --- a/distro/adaptation/archlinux +++ b/distro/adaptation/archlinux @@ -70,3 +70,5 @@ xutils-dev: imake xz-utils: xz libtirpc: libtirpc1 initramfs-tools: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/centos b/distro/adaptation/centos index e7fabf36d..2cfe3877c 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -334,3 +334,4 @@ junit4: junit liboath-dev: liboath-devel lsb-release: redhat-lsb linux-base: +openssh-client: openssh-clients diff --git a/distro/adaptation/fedora b/distro/adaptation/fedora index 38b7e9dc4..6686bc8c2 100644 --- a/distro/adaptation/fedora +++ b/distro/adaptation/fedora @@ -162,3 +162,4 @@ network-manager: NetworkManager javahelper: junit4: junit linux-base: +openssh-client: openssh-clients diff --git a/distro/adaptation/kylin b/distro/adaptation/kylin index f143fde96..e9286effd 100644 --- a/distro/adaptation/kylin +++ b/distro/adaptation/kylin @@ -147,3 +147,5 @@ lsb-release: openjdk: java-1.8.0-openjdk ruby-full: ruby ruby-devel linux-base: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/openeuler b/distro/adaptation/openeuler index e72f34fd2..09120489b 100644 --- a/distro/adaptation/openeuler +++ b/distro/adaptation/openeuler @@ -293,3 +293,5 @@ lsb-release: openjdk: java-1.8.0-openjdk ruby-full: ruby ruby-devel rubygem-rdoc linux-base: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/opensuse b/distro/adaptation/opensuse index 868d06b0d..f061c0894 100644 --- a/distro/adaptation/opensuse +++ b/distro/adaptation/opensuse @@ -13,3 +13,4 @@ libtirpc: libtirpc3 # Go get it yourself: https://github.com/schacon/ruby-git # We have no package for it yet ruby-git: +openssh-client: openssh-clients diff --git a/distro/adaptation/oracle b/distro/adaptation/oracle index af8415ec3..522b3d869 100644 --- a/distro/adaptation/oracle +++ b/distro/adaptation/oracle @@ -135,3 +135,4 @@ initramfs-tools: init-system-helpers: javahelper: linux-base: +openssh-client: openssh-clients -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] distro/depends: add sshd depends file
by Wang Yong 18 Dec '20

18 Dec '20

add depends file for sshd Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/sshd | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 distro/depends/sshd diff --git a/distro/depends/sshd b/distro/depends/sshd new file mode 100644 index 000000000..85ba955ae --- /dev/null +++ b/distro/depends/sshd @@ -0,0 +1,2 @@ +openssh-server +openssh-client -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: fix error for submit remote job
by Wu Zhende 18 Dec '20

18 Dec '20

[Error] submit iperf.yaml failed, got job_id=0, error: Missing uuid for remote job [How] Don't need to check uuid for remote job now. All jobs use my_token Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e7b9faf..dc43cbf 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -185,18 +185,6 @@ class Job # handle by me, then keep connect to me self["LKP_SERVER"] = SCHED_HOST self["LKP_CGI_PORT"] = SCHED_PORT.to_s - - # need further uuid check (validate? exist? no need?) - if self["SCHED_HOST"] != SCHED_HOST # remote submited job - # ?further fix to 127.0.0.1 (from remote ssh port forwarding) - # ?even set self["SCHED_HOST"] and self["SCHED_PORT"] - - if self["uuid"] == "" - puts "Job's SCHED_HOST is #{self["SCHED_HOST"]}, " + - "current scheduler IP is: #{SCHED_HOST}" - raise "Missing uuid for remote job" - end - end end private def set_sshr_info -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: add reference url
by Wu Zhende 18 Dec '20

18 Dec '20

submit iperf.yaml testbox=vm-2p8g submit iperf.yaml failed, got job_id=0, error: Missing required job key: 'my_token'. Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco… Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e7b9faf..e046072 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -325,7 +325,11 @@ class Job private def check_required_keys REQUIRED_KEYS.each do |key| if !@hash[key]? - raise "Missing required job key: '#{key}'" + error_msg = "Missing required job key: '#{key}'." + if ["my_email", "my_name", "my_token"].includes?(key) + error_msg += "\nPlease refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" + end + raise error_msg end end end -- 2.23.0

1 0

[PATCH compass-ci] container/initrd-http: support "/srv/cci/" dir download
by Cao Xueliang 18 Dec '20

18 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/initrd-http/root/etc/nginx/conf.d/default.conf | 4 ++++ container/initrd-http/start | 1 + 2 files changed, 5 insertions(+) diff --git a/container/initrd-http/root/etc/nginx/conf.d/default.conf b/container/initrd-http/root/etc/nginx/conf.d/default.conf index f2834c7..18813a2 100644 --- a/container/initrd-http/root/etc/nginx/conf.d/default.conf +++ b/container/initrd-http/root/etc/nginx/conf.d/default.conf @@ -13,4 +13,8 @@ server { location /initrd { autoindex on; } + + location /cci { + autoindex on; + } } diff --git a/container/initrd-http/start b/container/initrd-http/start index ed5d443..8341050 100755 --- a/container/initrd-http/start +++ b/container/initrd-http/start @@ -14,6 +14,7 @@ cmd=( --name initrd-http -p ${INITRD_HTTP_PORT:-8800}:80 -v /srv/initrd:/usr/share/nginx/html/initrd:ro + -v /srv/cci:/usr/share/nginx/html/cci:ro -v /etc/localtime:/etc/localtime:ro -d initrd-http -- 2.23.0

1 0

[PATCH v2 compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 18 Dec '20

18 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 18 Dec '20

18 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 59625299..addff9f7 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -129,7 +129,7 @@ build_source_pkg() create_cgz_name PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 18 Dec '20

18 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dd1ee120..59625299 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -128,8 +128,8 @@ build_source_pkg() create_cgz_name - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: rename my_uuid to my_token
by Wu Zhende 18 Dec '20

18 Dec '20

my_token is more expressive Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d44559..fa8e2dc 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -319,7 +319,7 @@ class Job testbox my_email my_name - my_uuid + my_token ] private def check_required_keys @@ -340,9 +340,10 @@ class Job raise error_msg if @account_info["found"]? == false raise error_msg unless self["my_name"] == @account_info["my_name"].to_s - raise error_msg unless self["my_uuid"] == @account_info["my_uuid"] + raise error_msg unless self["my_token"] == @account_info["my_token"] @hash.delete("my_uuid") + @hash.delete("my_token") end private def get_initialized_keys @@ -366,7 +367,7 @@ class Job "SCHED_HOST", "SCHED_PORT"] - initialized_keys -= ["my_uuid", + initialized_keys -= ["my_token", "kernel_version", "kernel_uri", "kernel_params", -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..dd1ee120f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -97,7 +97,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..9cb51ff6f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -17,7 +17,6 @@ ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, ## make customizations, generate meta-info, and package everything together. ## See: https://wiki.archlinux.org/index.php/Makepkg - . $LKP_SRC/lib/debug.sh . $LKP_SRC/distro/common @@ -97,7 +96,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 1

[PATCH compass-ci] fix: idle job for subqueue: submit idle jobs when finding no job
by Ren Wen 17 Dec '20

17 Dec '20

[Processes] There're 2 types of queues to consume: 1. user queue : sched/<taishan-a12>/<my_email> 2. idle queue : sched/<taishan-a12>/idle Consume Queue 1 at first, next Queue2 if no job found in Queue1. If there's no job in all Queues, auto submit idle jobs to idle queue. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 7565375..567016e 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -51,31 +51,38 @@ class Sched end def get_queues(host) - queues = [] of String + default_queues = [] of String + sub_queues = [] of String queues_str = @redis.hash_get("sched/host2queues", host) - return queues unless queues_str + return default_queues, sub_queues unless queues_str queues_str.split(',', remove_empty: true) do |item| - queues << item.strip + default_queues << item.strip end - sub_queues = [] of String - queues.each do |queue| - matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") + default_queues.each do |queue| + matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") matched_queues.each do |mq| match_data = "#{mq}".match(%r(^#{QUEUE_NAME_BASE}/sched/(#{queue}/.+)/ready$)) sub_queues << match_data[1] if match_data end end - return rand_queues(sub_queues) + return default_queues, sub_queues end def get_job_from_queues(queues, testbox) + default_queues, sub_queues = queues + all_queues = rand_queues(sub_queues) + + default_queues.each do |queue| + all_queues << "#{queue}/idle" + end + job = nil - queues.each do |queue| + all_queues.each do |queue| job = prepare_job("sched/#{queue}", testbox) return job if job end @@ -89,6 +96,8 @@ class Sched if job create_job_cpio(job.dump_to_json_any, Kemal.config.public_folder) + else + spawn { auto_submit_idle_job(host) } end return boot_content(job, boot_type) -- 2.23.0

1 0

[PATCH compass-ci] src/lib/web_backend.rb remove non-existent os_version
by Lu Weitao 17 Dec '20

17 Dec '20

remove non-existent os_version of compare_candidates Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 07e2e9e..cacafde 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -97,7 +97,7 @@ def compare_candidates_body suite: all_suite, OS: [ { os: 'openeuler', os_version: ['1.0', '20.03'] }, - { os: 'centos', os_version: ['7.6', '7.8', '8.1', 'sid'] }, + { os: 'centos', os_version: ['7.6', '7.8', '8.1'] }, { os: 'debian', os_version: %w[10 sid] }, { os: 'archlinux', os_version: ['5.5.0-1'] } ], -- 2.23.0

1 0

[PATCH lab-z9] hosts: add taishan200-2280-2s64p-256g--a121
by Wei Jihui 17 Dec '20

17 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- hosts/taishan200-2280-2s64p-256g--a121 | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a121 diff --git a/hosts/taishan200-2280-2s64p-256g--a121 b/hosts/taishan200-2280-2s64p-256g--a121 new file mode 100644 index 0000000..9c87315 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a121 @@ -0,0 +1,22 @@ +memory: 256G +nr_ssd_partitions: 1 +ssd_partitions: + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43943 +mac_addr: + - 44:67:47:e9:76:95 + - 44:67:47:e9:76:96 + - 44:67:47:e9:76:97 + - 44:67:47:e9:76:98 + - ac:b3:b5:1e:d3:ab + - ac:b3:b5:1e:d3:ac + - ac:b3:b5:1e:d3:ad + - ac:b3:b5:1e:d3:ae + - ac:b3:b5:1e:d6:0b + - ac:b3:b5:1e:d6:0c + - ac:b3:b5:1e:d6:0d + - ac:b3:b5:1e:d6:0e +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.12.2 -- 2.23.0

1 0

[PATCH compass-ci] container/web-backend: install terminal-table
by Li Yuanchao 17 Dec '20

17 Dec '20

As code changes, 'terminal-table' is needed in this container. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/web-backend/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/web-backend/Dockerfile b/container/web-backend/Dockerfile index 4fb6610..2ac5148 100644 --- a/container/web-backend/Dockerfile +++ b/container/web-backend/Dockerfile @@ -11,7 +11,7 @@ COPY conf/sources.list /etc/apt/ RUN apt-get update && \ apt-get install -y git ruby-json ruby-rails ruby-sinatra && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install elasticsearch redis git + gem install elasticsearch redis git terminal-table COPY web-backend /usr/local/bin/ -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: also hand over job from idle queue
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/close_job.cr | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index d8f80d7..a389a5c 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -14,8 +14,11 @@ class Sched raise "es set job content fail!" end + subqueue = job.subqueue + queue = subqueue == "idle" ? job.queue : "#{job.queue}/#{job.subqueue}" + response = @task_queue.hand_over_task( - "sched/#{job.queue}/#{job.subqueue}", "extract_stats", job_id + "sched/#{queue}", "extract_stats", job_id ) if response[0] != 201 raise "#{response}" -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: "idle" is a value of subqueue not a field
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index c322d0f..c09d145 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -114,13 +114,6 @@ class Sched def submit_single_job(job) queue = "#{job.queue}/#{job.subqueue}" - # only single job will has "idle job" and "execute rate limiter" - if job["idle_job"].empty? - queue += "#{job.get_uuid_tag}" - else - queue = "#{queue}/idle" - end - job_id = add_task(queue, job.lab) return [{ "job_id" => "0", -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: subqueue can be "idle"
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d44559..9fc3bf7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -274,7 +274,7 @@ class Job end private def set_subqueue - self["subqueue"] = self["my_email"] + self["subqueue"] = self["my_email"] unless self["subqueue"] == "idle" end # if not assign tbox_group, set it to a match result from testbox -- 2.23.0

1 0

[PATCH compass-ci] lib: optimize es query_by_id to adapt to change the es mapping
by Cao Xueliang 17 Dec '20

17 Dec '20

When we change es mapping, we can't use es.get_source, because the jobs is a alias for new jobs index, the old data at the old index. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/es_query.rb | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 68c70b5..3bea624 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -54,9 +54,12 @@ class ESQuery end def query_by_id(id) - @client.get_source({ index: @index, type: '_doc', id: id }) - rescue Elasticsearch::Transport::Transport::Errors::NotFound - nil + result = @client.search(index: @index + '*', + body: { query: { bool: { must: { term: { '_id' => id } } } }, + size: 1 })['hits']['hits'] + return nil unless result.size == 1 + + return result[0]['_source'] end end -- 2.23.0

1 0

[PATCH compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 17 Dec '20

17 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: add sshd depends file
by Wang Yong 17 Dec '20

17 Dec '20

add depends file for sshd Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/sshd | 1 + 1 file changed, 1 insertion(+) create mode 100644 distro/depends/sshd diff --git a/distro/depends/sshd b/distro/depends/sshd new file mode 100644 index 000000000..dd21b5aa9 --- /dev/null +++ b/distro/depends/sshd @@ -0,0 +1 @@ +openssh-server -- 2.23.0

2 2

[PATCH v2 compass-ci] scheduler: submit cluster job to "queue/subqueue"
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 4303b1d..c322d0f 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -37,6 +37,7 @@ class Sched def submit_cluster_job(job, cluster_config) job_messages = Array(Hash(String, String)).new lab = job.lab + subqueue = job.subqueue # collect all job ids job_ids = [] of String @@ -52,7 +53,7 @@ class Sched # steps for each host cluster_config["nodes"].as_h.each do |host, config| queue = host.to_s - job_id = add_task(queue, lab) + job_id = add_task("#{queue}/#{subqueue}", lab) # return when job_id is '0' # 2 Questions: -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add runtime field in borrow-*.yaml
by Zhang Yale 17 Dec '20

17 Dec '20

[why] 1. We need "$runtime" field to determine "Due time" value in email template received by users. 2. And the "$runtime" field is used in sleep script too. Therefore, sleep can be left blank, and pp.sleep can be parse successfully now. 3. User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/borrow-10d.yaml | 3 ++- jobs/borrow-1d.yaml | 3 ++- jobs/borrow-1h.yaml | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/jobs/borrow-10d.yaml b/jobs/borrow-10d.yaml index e0e35e17..88a137e0 100644 --- a/jobs/borrow-10d.yaml +++ b/jobs/borrow-10d.yaml @@ -1,6 +1,7 @@ suite: borrow testcase: borrow +runtime: 10d ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -10,4 +11,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 10d +sleep: diff --git a/jobs/borrow-1d.yaml b/jobs/borrow-1d.yaml index c31d7ebd..f79b440b 100644 --- a/jobs/borrow-1d.yaml +++ b/jobs/borrow-1d.yaml @@ -3,6 +3,7 @@ testcase: borrow timeout: 1d +runtime: 1d ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -12,4 +13,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 1d +sleep: diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml index d6b72c43..d13af910 100644 --- a/jobs/borrow-1h.yaml +++ b/jobs/borrow-1h.yaml @@ -1,6 +1,7 @@ suite: borrow testcase: borrow +runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -10,4 +11,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 1h +sleep: -- 2.23.0

2 2