October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 2/2] delete the field: testcase
by Li Ping 20 Oct '20

20 Oct '20

[why] there is no need to add the field: testcase in the $job.yaml Signed-off-by: Li Ping <15396232681(a)163.com> --- bin/lkp-setup-rootfs | 2 +- lib/bootstrap.sh | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 09f7e7eb..05a3325c 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -51,7 +51,7 @@ job_does_not_need_reboot() } grep -q "^do_not_reboot_for_same_kernel: 1$" $job && - is_same_kernel_and_rootfs && is_same_testcase && is_same_bp_memmap && { + is_same_kernel_and_rootfs && is_same_suite && is_same_bp_memmap && { echo "LKP: [do_not_reboot_for_same_kernel] start a new job without rebooting" return 0 } diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 2506c021..a7776d68 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -631,12 +631,12 @@ is_same_kernel_and_rootfs() return 1 } -is_same_testcase() +is_same_suite() { - local current_testcase=$testcase - local next_testcase=$(awk '/^testcase: /{print $2}' $job | tr -d '"') + local current_suite=$suite + local next_suite=$(awk '/^suite: /{print $2}' $job | tr -d '"') - [ "$current_testcase" = "$next_testcase" ] + [ "$current_suite" = "$next_suite" ] } is_same_bp_memmap() -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests: add get_ipmi_ip function to host-info
by Zhang Yu 20 Oct '20

20 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/host-info b/tests/host-info index a57855dc..9ca06235 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,12 @@ get_partitions() fi } +get_ipmi_ip() +{ + imp_ip=$(ipmitool lan print |awk 'NR==3{print $4}') + [ $? -eq 0 ] && echo "ipmi_ip:" $imp_ip +} + main() { get_node @@ -87,6 +93,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

4 3

[PATCH lkp-tests] sshd: send emails to login services for users
by Zhang Yale 20 Oct '20

20 Oct '20

[Why] Provide a login port for user by email, determine whether to send the ssh_port message based on the uuid. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 99 +++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 74 insertions(+), 25 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..cd57be4f 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key -# email +# my_email +# my_uuid +# ssh_port_len +# ssh_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +sshr_port=5050 +[ -n "$ssh_port_len" ] || ssh_port_len=2000 +[ -n "$ssh_port_base" ] || ssh_port_base=50000 run_ssh() { @@ -11,40 +21,79 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { - deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") - data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh - -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: - - Login: - ssh root@$PUB_IP - Due time: - $deadline - - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch + data="To: $my_email +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_username: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: + + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch + +Regards +Compass-ci +" +} + +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Applying $testbox failed + +Dear $my_username: + Sorry to inform you that your application failed. + You may need to wait for a while and then try again. Regards -compass-ci +Compass-ci " } +compose_email() +{ + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$my_uuid" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$ssh_port_len"+"$ssh_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$my_email" ] && [ -n "$my_uuid" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

3 5

[PATCH v9 lkp-tests] sshd: send emails to login services for users
by Zhang Yale 19 Oct '20

19 Oct '20

Provide a login port for user by email, determine whether to send the ssh_port message based on the uuid. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 89 +++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 20 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..55015b40 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key # email +# my_uuid +# ssh_port_len +# ssh_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=30000 run_ssh() { @@ -11,40 +21,79 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { - deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_name: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch - Login: - ssh root@$PUB_IP - Due time: - $deadline +Regards +Compass-ci +" +} + +data_failure() +{ + data="To: $email +Subject: [NOTIFY Compass-ci] Applying $testbox failed - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch +Dear $my_name: + Sorry to inform you that your application failed. + You may need to wait for a while and then try again. Regards -compass-ci +Compass-ci " } +compose_email() +{ + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$my_uuid" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$email" ] && [ -n "$my_uuid" ] || return 0 + my_name=$(echo "$email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

3 3

[PATCH compass-ci] job.cr: initramfs and nfs use the same kernel file
by Xu Xijian 19 Oct '20

19 Oct '20

Actually the vmlinuz file in SRV_INITRD is copy from SRV_OS, so they are the same file, just use the same path is enough. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1e44d15..fc15441 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -272,18 +272,9 @@ class Job return true end - private def vmlinuz - if os_mount == "initramfs" - return "#{INITRD_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{os_dir}/vmlinuz")}" - else - return "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" - end - end - private def set_kernel_uri - self["kernel_uri"] = "kernel #{vmlinuz()}" + self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" end private def kernel_common_params -- 2.23.0

1 0

[PATCH lab-z9] allot/idle: remove the field -hi1620 from tbox_group
by Li Ping 19 Oct '20

19 Oct '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- allot/idle/taishan200-2280-2s48p-256g--a60/multi-qemu.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/allot/idle/taishan200-2280-2s48p-256g--a60/multi-qemu.yaml b/allot/idle/taishan200-2280-2s48p-256g--a60/multi-qemu.yaml index 4525fcd..513b631 100644 --- a/allot/idle/taishan200-2280-2s48p-256g--a60/multi-qemu.yaml +++ b/allot/idle/taishan200-2280-2s48p-256g--a60/multi-qemu.yaml @@ -8,11 +8,11 @@ os_mount: initramfs simplify-ci: multi-qemu-0: nr_vm: 20 - tbox_group: vm-hi1620-2p8g + tbox_group: vm-2p8g multi-qemu-1: nr_vm: 2 - tbox_group: vm-hi1620-2p4g + tbox_group: vm-2p4g multi-qemu-2: nr_vm: 2 - tbox_group: vm-hi1620-1p1g + tbox_group: vm-1p1g sleep: 100d -- 2.23.0

1 0

[PATCH v7 compass-ci 1/6] container/mail-robot: mail-robot.rb
by Luan Shengde 19 Oct '20

19 Oct '20

mail robot for email application monitor mailbox check new added email file in new check new mail's subject return if unmatched subject call answerback-email check to apply uuid/account Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/mail-robot.rb | 43 ++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100755 container/mail-robot/mail-robot.rb diff --git a/container/mail-robot/mail-robot.rb b/container/mail-robot/mail-robot.rb new file mode 100755 index 0000000..70b8b25 --- /dev/null +++ b/container/mail-robot/mail-robot.rb @@ -0,0 +1,43 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' +require_relative 'answerback-email.rb' + +mail_inbox = '/srv/cci/Maildir/.compass-ci/new/' +mail_drafts = '/srv/cci/Maildir/.compass-ci/cur/' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + unless added.empty? + added.each do |mail_file| + begin + check_to_send_account(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + puts e.backtrace + end + end + end + end + listener.start + sleep +end + +def check_to_send_account(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + subject = mail_content.subject + return unless subject =~ /apply account/i + + assign_uuid = AssignUuid.new(mail_content) + assign_uuid.send_account + FileUtils.mv(mail_file, mail_drafts) +end + +monitor_dir(mail_inbox, mail_drafts) -- 2.23.0

3 4

[PATCH lkp-tests] lib/job.rb: load hosts file for atomic job
by Wei Jihui 19 Oct '20

19 Oct '20

[why] for commit c725e0f022b7: def each_jobs(&block) each_job_init - load_hosts_config job = deepcopy(@job) @job2 = {} load_defaults + load_hosts_config each_job_init each_job(&block) @jobs.each do |hash| job = deepcopy(@job) this @job has no load hosts config, so it should do it under "@jobs.each do |hash|" for atomic job. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index 14946f47..6359254e 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -696,9 +696,11 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| + @load_hosts_done = false @job = deepcopy(job) @job2 = hash load_defaults + load_hosts_config each_job_init each_job(&block) end -- 2.23.0

2 2

[PATCH v7 compass-ci 5/6] container/mail-robot: start file
by Luan Shengde 19 Oct '20

19 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/start | 39 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..108ed83 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,39 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +docker_rm 'mail-robot' + +names = Set.new %w[ + JUMPER_HOST + JUMPER_PORT + SEND_INTERNET_MAIL_PORT +] + +defaults = relevant_defaults(names) + +JUMPER_HOST = defaults['JUMPER_HOST'] +JUMPER_PORT = defaults['JUMPER_PORT'] +SEND_MAIL_PORT = defaults['SEND_INTERNET_MAIL_PORT'] + +cmd = %W[ + docker run + --restart=always + --name=mail-robot + -d + -e J_HOST=#{JUMPER_HOST} + -e J_PORT=#{JUMPER_PORT} + -e S_M_PORT=#{SEND_MAIL_PORT} + -v /c/upstream-repos:/c/upstream-repos:ro + -v /srv/cci/Maildir:/srv/cci/Maildir:rw + -v /c/compass-ci:/c/compass-ci:ro + mail-robot +] + +cmd += ['bash', '-c', 'umask 002 && ruby ./mail-robot.rb'] + +system(*cmd) -- 2.23.0

2 2

[PATCH v2 compass-ci] sched: throw an exception when deps/pkg does not exist
by Cao Xueliang 19 Oct '20

19 Oct '20

When submitting a job, an exception is thrown when deps/pkg does not exist on the server. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 32aa662..7785c2a 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -389,10 +389,12 @@ class Job program_params.keys.each do |program| deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}.cgz" pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}.cgz" - if File.exists?("#{ENV["LKP_SRC"]}/distro/depends/#{program}") && File.exists?(deps_dest_file) + + if File.exists?("#{ENV["LKP_SRC"]}/distro/depends/#{program}") initrd_deps_arr << "#{initrd_http_prefix}" + JobHelper.service_path(deps_dest_file) end - if File.exists?("#{ENV["LKP_SRC"]}/pkg/#{program}") && File.exists?(pkg_dest_file) + + if File.exists?("#{ENV["LKP_SRC"]}/pkg/#{program}") initrd_pkg_arr << "#{initrd_http_prefix}" + JobHelper.service_path(pkg_dest_file) end end -- 2.23.0

2 5