- Compass-ci - mailweb.openeuler.org

review star
by Wu Fengguang 18 Dec '20

18 Dec '20

Xueliang is our review star, thanks Xueliang! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 121 21 24 24 Cao Xueliang 48 30 57 20 Wu Fengguang 74 16 23 15 Wei Jihui 69 7 20 10 Luan Shengde 40 13 11 7 Liu Shaofei 42 14 20 7 Liu Yinsi 29 4 13 5 Yu Chuan 8 2 6 4 Xu Xijian 36 8 7 4 Lu Kaiyi 26 4 5 4 Xiao Shenwei 36 6 4 3 Zhang Yu 22 6 7 3 Lin Jiaxin 13 4 12 2 Sun Yukui 21 5 12 2 Ren Wen 14 4 2 2 Cui Yili 6 2 1 1 Bai Jing 18 3 2 1 Hu Xuejiao 30 4 4 1 Wang Chenglong 38 1 1 1 Wu Zhende 7 2 0 0 Zhang Yale 15 3 4 0 Li Ping 9 0 0 0 Li Yuanchao 6 2 2 0 Zhang Dewan 10 0 0 0 Wang Yong 25 2 4 0 Lu Weitao

1 0

[PATCH compass-ci] lib/es_query: add query filed from index
by Lu Weitao 18 Dec '20

18 Dec '20

[Why] We need only get specific field instead of entire document. it's like SQL: SELECT DISTINCT column FROM table Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/es_query.rb | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index 3bea624..86c078f 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -61,6 +61,30 @@ class ESQuery return result[0]['_source'] end + + # select doc_field from index + # input: + # eg: suite (@index: jobs) + # output: + # [ + # {"key"=>"build-pkg", "doc_count"=>90841}, + # {"key"=>"cci-depends", "doc_count"=>4636}, + # {"key"=>"cci-makepkg", "doc_count"=>3647}, + # ... + # ] + def query_specific_fields(field) + query = { + aggs: { + "all_#{field}" => { + terms: { field: field, size: 1000 } + } + }, + size: 0 + } + result = @client.search index: @index + '*', body: query + + result ['aggregations']["all_#{field}"]['buckets'] + end end # Range Query Example: -- 2.23.0

1 1

[PATCH compass-ci] mail-robot/assign-account-email: add config for lab
by Wei Jihui 18 Dec '20

18 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 834e96f..77ed4e8 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -24,6 +24,7 @@ def build_apply_account_email(my_info) cat >> ~/.config/compass-ci/defaults/account.yaml <<-EOF my_email: #{my_info['my_email']} my_name: #{my_info['my_name']} + lab: #{ENV['lab']} EOF mkdir -p ~/.config/compass-ci/include/lab cat >> ~/.config/compass-ci/include/lab/#{ENV['lab']}.yaml <<-EOF -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] distro/adaptation: add sshd mapping
by Wang Yong 18 Dec '20

18 Dec '20

add sshd mapping in different OS Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/archlinux | 2 ++ distro/adaptation/centos | 1 + distro/adaptation/fedora | 1 + distro/adaptation/kylin | 2 ++ distro/adaptation/openeuler | 2 ++ distro/adaptation/opensuse | 1 + distro/adaptation/oracle | 1 + 7 files changed, 10 insertions(+) diff --git a/distro/adaptation/archlinux b/distro/adaptation/archlinux index 53474d6c4..89163cf41 100644 --- a/distro/adaptation/archlinux +++ b/distro/adaptation/archlinux @@ -70,3 +70,5 @@ xutils-dev: imake xz-utils: xz libtirpc: libtirpc1 initramfs-tools: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/centos b/distro/adaptation/centos index e7fabf36d..2cfe3877c 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -334,3 +334,4 @@ junit4: junit liboath-dev: liboath-devel lsb-release: redhat-lsb linux-base: +openssh-client: openssh-clients diff --git a/distro/adaptation/fedora b/distro/adaptation/fedora index 38b7e9dc4..6686bc8c2 100644 --- a/distro/adaptation/fedora +++ b/distro/adaptation/fedora @@ -162,3 +162,4 @@ network-manager: NetworkManager javahelper: junit4: junit linux-base: +openssh-client: openssh-clients diff --git a/distro/adaptation/kylin b/distro/adaptation/kylin index f143fde96..e9286effd 100644 --- a/distro/adaptation/kylin +++ b/distro/adaptation/kylin @@ -147,3 +147,5 @@ lsb-release: openjdk: java-1.8.0-openjdk ruby-full: ruby ruby-devel linux-base: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/openeuler b/distro/adaptation/openeuler index e72f34fd2..09120489b 100644 --- a/distro/adaptation/openeuler +++ b/distro/adaptation/openeuler @@ -293,3 +293,5 @@ lsb-release: openjdk: java-1.8.0-openjdk ruby-full: ruby ruby-devel rubygem-rdoc linux-base: +openssh-server: openssh +openssh-client: openssh diff --git a/distro/adaptation/opensuse b/distro/adaptation/opensuse index 868d06b0d..f061c0894 100644 --- a/distro/adaptation/opensuse +++ b/distro/adaptation/opensuse @@ -13,3 +13,4 @@ libtirpc: libtirpc3 # Go get it yourself: https://github.com/schacon/ruby-git # We have no package for it yet ruby-git: +openssh-client: openssh-clients diff --git a/distro/adaptation/oracle b/distro/adaptation/oracle index af8415ec3..522b3d869 100644 --- a/distro/adaptation/oracle +++ b/distro/adaptation/oracle @@ -135,3 +135,4 @@ initramfs-tools: init-system-helpers: javahelper: linux-base: +openssh-client: openssh-clients -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] distro/depends: add sshd depends file
by Wang Yong 18 Dec '20

18 Dec '20

add depends file for sshd Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/sshd | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 distro/depends/sshd diff --git a/distro/depends/sshd b/distro/depends/sshd new file mode 100644 index 000000000..85ba955ae --- /dev/null +++ b/distro/depends/sshd @@ -0,0 +1,2 @@ +openssh-server +openssh-client -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: fix error for submit remote job
by Wu Zhende 18 Dec '20

18 Dec '20

[Error] submit iperf.yaml failed, got job_id=0, error: Missing uuid for remote job [How] Don't need to check uuid for remote job now. All jobs use my_token Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e7b9faf..dc43cbf 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -185,18 +185,6 @@ class Job # handle by me, then keep connect to me self["LKP_SERVER"] = SCHED_HOST self["LKP_CGI_PORT"] = SCHED_PORT.to_s - - # need further uuid check (validate? exist? no need?) - if self["SCHED_HOST"] != SCHED_HOST # remote submited job - # ?further fix to 127.0.0.1 (from remote ssh port forwarding) - # ?even set self["SCHED_HOST"] and self["SCHED_PORT"] - - if self["uuid"] == "" - puts "Job's SCHED_HOST is #{self["SCHED_HOST"]}, " + - "current scheduler IP is: #{SCHED_HOST}" - raise "Missing uuid for remote job" - end - end end private def set_sshr_info -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: add reference url
by Wu Zhende 18 Dec '20

18 Dec '20

submit iperf.yaml testbox=vm-2p8g submit iperf.yaml failed, got job_id=0, error: Missing required job key: 'my_token'. Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco… Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e7b9faf..e046072 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -325,7 +325,11 @@ class Job private def check_required_keys REQUIRED_KEYS.each do |key| if !@hash[key]? - raise "Missing required job key: '#{key}'" + error_msg = "Missing required job key: '#{key}'." + if ["my_email", "my_name", "my_token"].includes?(key) + error_msg += "\nPlease refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" + end + raise error_msg end end end -- 2.23.0

1 0

[PATCH compass-ci] container/initrd-http: support "/srv/cci/" dir download
by Cao Xueliang 18 Dec '20

18 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/initrd-http/root/etc/nginx/conf.d/default.conf | 4 ++++ container/initrd-http/start | 1 + 2 files changed, 5 insertions(+) diff --git a/container/initrd-http/root/etc/nginx/conf.d/default.conf b/container/initrd-http/root/etc/nginx/conf.d/default.conf index f2834c7..18813a2 100644 --- a/container/initrd-http/root/etc/nginx/conf.d/default.conf +++ b/container/initrd-http/root/etc/nginx/conf.d/default.conf @@ -13,4 +13,8 @@ server { location /initrd { autoindex on; } + + location /cci { + autoindex on; + } } diff --git a/container/initrd-http/start b/container/initrd-http/start index ed5d443..8341050 100755 --- a/container/initrd-http/start +++ b/container/initrd-http/start @@ -14,6 +14,7 @@ cmd=( --name initrd-http -p ${INITRD_HTTP_PORT:-8800}:80 -v /srv/initrd:/usr/share/nginx/html/initrd:ro + -v /srv/cci:/usr/share/nginx/html/cci:ro -v /etc/localtime:/etc/localtime:ro -d initrd-http -- 2.23.0

1 0

[PATCH v2 compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 18 Dec '20

18 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 18 Dec '20

18 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 59625299..addff9f7 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -129,7 +129,7 @@ build_source_pkg() create_cgz_name PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 18 Dec '20

18 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dd1ee120..59625299 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -128,8 +128,8 @@ build_source_pkg() create_cgz_name - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: rename my_uuid to my_token
by Wu Zhende 18 Dec '20

18 Dec '20

my_token is more expressive Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d44559..fa8e2dc 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -319,7 +319,7 @@ class Job testbox my_email my_name - my_uuid + my_token ] private def check_required_keys @@ -340,9 +340,10 @@ class Job raise error_msg if @account_info["found"]? == false raise error_msg unless self["my_name"] == @account_info["my_name"].to_s - raise error_msg unless self["my_uuid"] == @account_info["my_uuid"] + raise error_msg unless self["my_token"] == @account_info["my_token"] @hash.delete("my_uuid") + @hash.delete("my_token") end private def get_initialized_keys @@ -366,7 +367,7 @@ class Job "SCHED_HOST", "SCHED_PORT"] - initialized_keys -= ["my_uuid", + initialized_keys -= ["my_token", "kernel_version", "kernel_uri", "kernel_params", -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..dd1ee120f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -97,7 +97,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix incorrect git_tag value
by Liu Shaofei 18 Dec '20

18 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index fd2697e25..9cb51ff6f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -17,7 +17,6 @@ ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, ## make customizations, generate meta-info, and package everything together. ## See: https://wiki.archlinux.org/index.php/Makepkg - . $LKP_SRC/lib/debug.sh . $LKP_SRC/distro/common @@ -97,7 +96,7 @@ replace_source() echo "$url" | egrep 'git\+|\.git$' && { [ -n "$repo_dir" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}" - [ -n ${git_tag} ] && { + [ -n "$git_tag" ] && { url="${repo_dir}git://${LKP_SERVER}/${upstream_repo}#tag=${git_tag}" } } -- 2.23.0

1 1

[PATCH compass-ci] fix: idle job for subqueue: submit idle jobs when finding no job
by Ren Wen 17 Dec '20

17 Dec '20

[Processes] There're 2 types of queues to consume: 1. user queue : sched/<taishan-a12>/<my_email> 2. idle queue : sched/<taishan-a12>/idle Consume Queue 1 at first, next Queue2 if no job found in Queue1. If there's no job in all Queues, auto submit idle jobs to idle queue. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 7565375..567016e 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -51,31 +51,38 @@ class Sched end def get_queues(host) - queues = [] of String + default_queues = [] of String + sub_queues = [] of String queues_str = @redis.hash_get("sched/host2queues", host) - return queues unless queues_str + return default_queues, sub_queues unless queues_str queues_str.split(',', remove_empty: true) do |item| - queues << item.strip + default_queues << item.strip end - sub_queues = [] of String - queues.each do |queue| - matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") + default_queues.each do |queue| + matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") matched_queues.each do |mq| match_data = "#{mq}".match(%r(^#{QUEUE_NAME_BASE}/sched/(#{queue}/.+)/ready$)) sub_queues << match_data[1] if match_data end end - return rand_queues(sub_queues) + return default_queues, sub_queues end def get_job_from_queues(queues, testbox) + default_queues, sub_queues = queues + all_queues = rand_queues(sub_queues) + + default_queues.each do |queue| + all_queues << "#{queue}/idle" + end + job = nil - queues.each do |queue| + all_queues.each do |queue| job = prepare_job("sched/#{queue}", testbox) return job if job end @@ -89,6 +96,8 @@ class Sched if job create_job_cpio(job.dump_to_json_any, Kemal.config.public_folder) + else + spawn { auto_submit_idle_job(host) } end return boot_content(job, boot_type) -- 2.23.0

1 0

[PATCH compass-ci] src/lib/web_backend.rb remove non-existent os_version
by Lu Weitao 17 Dec '20

17 Dec '20

remove non-existent os_version of compare_candidates Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 07e2e9e..cacafde 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -97,7 +97,7 @@ def compare_candidates_body suite: all_suite, OS: [ { os: 'openeuler', os_version: ['1.0', '20.03'] }, - { os: 'centos', os_version: ['7.6', '7.8', '8.1', 'sid'] }, + { os: 'centos', os_version: ['7.6', '7.8', '8.1'] }, { os: 'debian', os_version: %w[10 sid] }, { os: 'archlinux', os_version: ['5.5.0-1'] } ], -- 2.23.0

1 0

[PATCH lab-z9] hosts: add taishan200-2280-2s64p-256g--a121
by Wei Jihui 17 Dec '20

17 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- hosts/taishan200-2280-2s64p-256g--a121 | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a121 diff --git a/hosts/taishan200-2280-2s64p-256g--a121 b/hosts/taishan200-2280-2s64p-256g--a121 new file mode 100644 index 0000000..9c87315 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a121 @@ -0,0 +1,22 @@ +memory: 256G +nr_ssd_partitions: 1 +ssd_partitions: + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43943 +mac_addr: + - 44:67:47:e9:76:95 + - 44:67:47:e9:76:96 + - 44:67:47:e9:76:97 + - 44:67:47:e9:76:98 + - ac:b3:b5:1e:d3:ab + - ac:b3:b5:1e:d3:ac + - ac:b3:b5:1e:d3:ad + - ac:b3:b5:1e:d3:ae + - ac:b3:b5:1e:d6:0b + - ac:b3:b5:1e:d6:0c + - ac:b3:b5:1e:d6:0d + - ac:b3:b5:1e:d6:0e +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.12.2 -- 2.23.0

1 0

[PATCH compass-ci] container/web-backend: install terminal-table
by Li Yuanchao 17 Dec '20

17 Dec '20

As code changes, 'terminal-table' is needed in this container. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/web-backend/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/web-backend/Dockerfile b/container/web-backend/Dockerfile index 4fb6610..2ac5148 100644 --- a/container/web-backend/Dockerfile +++ b/container/web-backend/Dockerfile @@ -11,7 +11,7 @@ COPY conf/sources.list /etc/apt/ RUN apt-get update && \ apt-get install -y git ruby-json ruby-rails ruby-sinatra && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install elasticsearch redis git + gem install elasticsearch redis git terminal-table COPY web-backend /usr/local/bin/ -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: also hand over job from idle queue
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/close_job.cr | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index d8f80d7..a389a5c 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -14,8 +14,11 @@ class Sched raise "es set job content fail!" end + subqueue = job.subqueue + queue = subqueue == "idle" ? job.queue : "#{job.queue}/#{job.subqueue}" + response = @task_queue.hand_over_task( - "sched/#{job.queue}/#{job.subqueue}", "extract_stats", job_id + "sched/#{queue}", "extract_stats", job_id ) if response[0] != 201 raise "#{response}" -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: "idle" is a value of subqueue not a field
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index c322d0f..c09d145 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -114,13 +114,6 @@ class Sched def submit_single_job(job) queue = "#{job.queue}/#{job.subqueue}" - # only single job will has "idle job" and "execute rate limiter" - if job["idle_job"].empty? - queue += "#{job.get_uuid_tag}" - else - queue = "#{queue}/idle" - end - job_id = add_task(queue, job.lab) return [{ "job_id" => "0", -- 2.23.0

1 0

[PATCH compass-ci] fix: idle job for subqueue: subqueue can be "idle"
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d44559..9fc3bf7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -274,7 +274,7 @@ class Job end private def set_subqueue - self["subqueue"] = self["my_email"] + self["subqueue"] = self["my_email"] unless self["subqueue"] == "idle" end # if not assign tbox_group, set it to a match result from testbox -- 2.23.0

1 0

[PATCH compass-ci] lib: optimize es query_by_id to adapt to change the es mapping
by Cao Xueliang 17 Dec '20

17 Dec '20

When we change es mapping, we can't use es.get_source, because the jobs is a alias for new jobs index, the old data at the old index. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/es_query.rb | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 68c70b5..3bea624 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -54,9 +54,12 @@ class ESQuery end def query_by_id(id) - @client.get_source({ index: @index, type: '_doc', id: id }) - rescue Elasticsearch::Transport::Transport::Errors::NotFound - nil + result = @client.search(index: @index + '*', + body: { query: { bool: { must: { term: { '_id' => id } } } }, + size: 1 })['hits']['hits'] + return nil unless result.size == 1 + + return result[0]['_source'] end end -- 2.23.0

1 0

[PATCH compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 17 Dec '20

17 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: add sshd depends file
by Wang Yong 17 Dec '20

17 Dec '20

add depends file for sshd Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/sshd | 1 + 1 file changed, 1 insertion(+) create mode 100644 distro/depends/sshd diff --git a/distro/depends/sshd b/distro/depends/sshd new file mode 100644 index 000000000..dd21b5aa9 --- /dev/null +++ b/distro/depends/sshd @@ -0,0 +1 @@ +openssh-server -- 2.23.0

2 2

[PATCH v2 compass-ci] scheduler: submit cluster job to "queue/subqueue"
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 4303b1d..c322d0f 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -37,6 +37,7 @@ class Sched def submit_cluster_job(job, cluster_config) job_messages = Array(Hash(String, String)).new lab = job.lab + subqueue = job.subqueue # collect all job ids job_ids = [] of String @@ -52,7 +53,7 @@ class Sched # steps for each host cluster_config["nodes"].as_h.each do |host, config| queue = host.to_s - job_id = add_task(queue, lab) + job_id = add_task("#{queue}/#{subqueue}", lab) # return when job_id is '0' # 2 Questions: -- 2.23.0

1 0