- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] lib: fix btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 21 Oct '20

21 Oct '20

When job.yaml container "fs: **", and /srv/initrd/deps/**/**/fs.cgz(btrfs-progs), it will execute mkfs.btrfs -f $rootfs_partition. but job.yaml don't container "fs: **" to appear warning: mkfs.btrfs command not found Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/bootstrap.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..16789c5c 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,6 +456,7 @@ mount_rootfs()… [View More]

2 1

[PATCH compass-ci] auto_submit: add upstream_tag if the commit is tagged
by Li Yuanchao 21 Oct '20

21 Oct '20

sometimes the testcase needs a specific version software, add upstream_tag can help find the specific version. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sbin/auto_submit b/sbin/auto_submit index 1f55701..88ec560 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -28,12 +28,25 @@ class AutoSubmit return pkgbuild_repo end + def if_tagged(tag_list, commit_id) + tag_list.… [View More]

7 17

[PATCH v5 lkp-tests] tests: add get_ipmi_ip function to host-info
by Zhang Yu 21 Oct '20

21 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tests/host-info b/tests/host-info index a57855dc..99b14083 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,25 @@ get_partitions() fi } +get_ipmi_ip() +{ + # root@localhost ~# ipmitool lan print + # Set in Progress : Set Complete + # IP Address Source : Static Address + # IP Address : 9.3.4.1 + # Subnet … [View More]

3 2

[PATCH compass-ci] fix: scheduler: cut down log
by Tong Qunfeng 21 Oct '20

21 Oct '20

[why] when consume with "No job now", scheduler output 14 lines log. when submit 1 job and consume it, scheduler output 78 lines log. it is a bit of heavy for log collect and some logs are redundancy. [how] 1.merge 4 lines to 1 line (form to json) for debug_message. 2.remove output "Done", it's same with kemal-frame meesage "200 GET/PUT..." 3.only 0 | 1 extra message for 1 API: if there has an active message, do not call debug_message as default, there has 1 kemal-frame message for each API … [View More] 2020-10-20 09:40:28 UTC 200 GET /boot.ipxe/mac/0a-4c-34-18-0a-17 7.18ms <time in> <response status> <----request API----> <response time span> [now] this patch cut 92 lines log to 26 line, interactive with 15 API. examples: --- <4 to 1> and <remove single "Done"> >> 172.168.131.113:29720 << Done 2020-10... 200 PUT /set_host_mac?hostname=vm... 289.2µs --> 2020-10... 200 PUT /set_host_mac?hostname=vm... 977.56µs --- <0 | 1 extra message> >> 172.18.222.81:44210 << Done {"job_state":"running","job_id":"crystal.105070"} 2020-10... 200 GET /~lkp/cgi-bin/lkp-jobfile-append-var?... 2.36ms --> {"job_state":"running","job_id":"crystal.105071"} 2020-10... UTC 200 GET /~lkp/cgi-bin/lkp-jobfile-append-var?... 2.8ms Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/sched.cr | 9 +++------ src/scheduler/scheduler.cr | 30 ++++++++++-------------------- 2 files changed, 13 insertions(+), 26 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index e6f353a..7215df5 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -317,7 +317,6 @@ class Sched JobHelper.service_path("#{SRV_INITRD}/lkp/#{job.lkp_initrd_user}/lkp-#{job.arch}.cgz") response["job"] = "http://#{SCHED_HOST}:#{SCHED_PORT}/job_initrd_tmpfs/#{job.id}/job.cgz" - puts %({"job_id": "#{job.id}", "job_state": "boot"}) return response.to_json end @@ -339,7 +338,6 @@ class Sched response += "boot\n" - puts %({"job_id": "#{job.id}", "job_state": "boot"}) return response end @@ -481,7 +479,6 @@ class Sched response += job.kernel_params response += "\nboot\n" - puts %({"job_id": "#{job.id}", "job_state": "boot"}) return response end @@ -512,7 +509,7 @@ class Sched # json log log = job_content.dup log["job_id"] = log.delete("id").not_nil! - puts log.to_json + return log.to_json end def update_tbox_wtmp(env : HTTP::Server::Context) @@ -541,7 +538,7 @@ class Sched # json log hash["testbox"] = testbox - puts hash.to_json + return hash.to_json end def report_ssh_port(testbox : String, ssh_port : String) @@ -572,6 +569,6 @@ class Sched @redis.remove_finished_job(job_id) - puts %({"job_id": "#{job_id}", "job_state": "complete"}) + return %({"job_id": "#{job_id}", "job_state": "complete"}) end end diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index e38d15c..5befcf9 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -34,9 +34,7 @@ module Scheduler # for debug (maybe kemal debug|logger does better) def self.debug_message(env, response) - puts "\n\n" - puts ">> #{env.request.remote_address}" - puts "<< #{response}" + puts %({"from": "#{env.request.remote_address}", "response": #{response.to_json}}) end # echo alive @@ -53,7 +51,8 @@ module Scheduler get "/boot.:boot_type/:parameter/:value" do |env| response = sched.find_job_boot(env) - debug_message(env, response) + job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? + puts %({"job_id": "#{job_id}", "job_state": "boot"}) if job_id response end @@ -62,7 +61,8 @@ module Scheduler get "/~lkp/cgi-bin/gpxelinux.cgi" do |env| response = sched.find_next_job_boot(env) - debug_message(env, response) + job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? + puts %({"job_id": "#{job_id}", "job_state": "boot"}) if job_id response end @@ -87,7 +87,6 @@ module Scheduler file_path = ::File.join [Kemal.config.public_folder, job_id, job_package] puts %({"job_id": "#{job_id}", "job_state": "download"}) - debug_message(env, file_path) send_file env, file_path end @@ -102,8 +101,6 @@ module Scheduler if (client_hostname = env.params.query["hostname"]?) && (client_mac = env.params.query["mac"]?) sched.set_host_mac(client_mac, client_hostname) - debug_message(env, "Done") - "Done" else "No yet!" @@ -115,8 +112,6 @@ module Scheduler if client_mac = env.params.query["mac"]? sched.del_host_mac(client_mac) - debug_message(env, "Done") - "Done" else "No yet!" @@ -129,10 +124,8 @@ module Scheduler # ?job_file=/lkp/scheduled/job.yaml&job_state=post_run&job_id=10 # ?job_file=/lkp/scheduled/job.yaml&loadavg=0.28 0.82 0.49 1/105 3389&start_time=1587725398&end_time=1587725698&job_id=10 get "/~lkp/cgi-bin/lkp-jobfile-append-var" do |env| - # get job_id from request - debug_message(env, "Done") + puts sched.update_job_parameter(env) - sched.update_job_parameter(env) "Done" end @@ -168,17 +161,15 @@ module Scheduler # get job_id from request job_id = env.params.query["job_id"]? if job_id - debug_message(env, "Done") - - sched.close_job(job_id) + puts sched.close_job(job_id) end + "Done" end get "/~lkp/cgi-bin/lkp-wtmp" do |env| - debug_message(env, "Done") + puts sched.update_tbox_wtmp(env) - sched.update_tbox_wtmp(env) "Done" end @@ -188,12 +179,11 @@ module Scheduler job_id = env.params.query["job_id"].to_s if testbox && ssh_port - debug_message(env, "Done") - sched.report_ssh_port(testbox, ssh_port) end puts %({"job_id": "#{job_id}", "state": "set ssh port", "ssh_port": "#{ssh_port}", "tbox_name": "#{testbox}"}) + "Done" end end -- 2.23.0 [View Less]

1 0

[PATCH lkp-tests] job: add monitors to job
by Wei Jihui 21 Oct '20

21 Oct '20

find all monitors and add to job scheduler will use it to get monitors program's cgz. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 7 +++++++ sbin/submit | 1 + 2 files changed, 8 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index caa76c4b..e2416eb8 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -564,6 +564,13 @@ class Job return option_value end + def add_monitors + @job['monitors'] = {} + for_each_in(@job, monitor_params) do |_pk, _h, m_n, … [View More]

2 1

[PATCH v4 compass-ci 1/2] container/assign-account: answerback-email.rb
by Luan Shengde 21 Oct '20

21 Oct '20

add parameters for answerback-email -e,--email email_addr: add email address -s,--ssh-pubkey pub_key: add a ssh login pub_key -f,--raw-email mail_file: add email address[, pub_key] via email file why: easier for administrator to manully assign jumper account for user with: - user's email - user's email and a pub_key - user's email_file Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/answerback-email.rb | 173 ++++++++----------… [View More]- 1 file changed, 74 insertions(+), 99 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index b5283ba..5a5234c 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -3,46 +3,18 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -=begin - -repo_list: - all repos url list from upstream-repos.git - -API: -call graph: -read_mail_content -send_account_request - email_addr - get email address - email_message_id - get email message_id - check_email_available - check email available - pub_key_value - get pub key - account - send apply account request and return account info - build_message - build email message - send_mail - call send_mail to send mail whit build message - -the returned data for account_info like: -{ - "account" => "guest", - "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", - "jumper_port" => "10000" -} -=end +# xx --email xxx --login --ssh-pubkey xxx --raw-email email-file +# samba mount +# ssh logshn (huawei, ) (install pubkey / send password) require 'json' require 'mail' require 'set' -require_relative '../defconfig.rb' +require 'optparse' +require_relative '../defconfig' names = Set.new %w[ - JUMPER_IP + JUMPER_HOST JUMPER_PORT CRYSTAL_INTRANET SEND_MAIL_PORT @@ -50,27 +22,71 @@ names = Set.new %w[ defaults = relevant_defaults(names) -JUMPER_IP = defaults['JUMPER_IP'] -JUMPER_PORT = defaults['JUMPER_PORT'] -CRYSTAL_INTRANET = defaults['CRYSTAL_INTRANET'] -SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] +JUMPER_HOST = defaults['JUMPER_HOST'] || '183.134.196.212' +JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 +CRYSTAL_INTRANET = defaults['CRYSTAL_INTRANET'] || localhost +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 11312 + +apply_info = { + 'email_addr' => nil, + 'pub_key' => nil +} + +def init_info(email_file, apply_info) + mail_content = Mail.read(email_file) + + email_addr = mail_content.from[0] + pub_key = mail_content.part[1].body.decoded.gsub(/\r|\n/, '') if mail_content.part[1].filename == 'id_rsa.pub' + + apply_info['email_addr'] = email_addr + apply_info['pub_key'] = pub_key + + apply_info +end + +options = OptionParser.new do |opts| + opts.banner = "Usage: answerback-mail.rb [--email email] [--ssh-pubkey pub_key] [--raw-email email_file]\n" + opts.banner += " -e or -f is required\n" + opts.banner += ' -s is optional when use -e' + + opts.separator '' + opts.separator 'options:' + + opts.on('-e email_addr', '--email email_addr', 'appoint email address') do |email_addr| + apply_info['email_addr'] = email_addr + end + + opts.on('-s pub_key', '--ssh-pubkey pub_key', 'ssh pub_key to enable keyless login') do |pub_key| + apply_info['pub_key'] = pub_key + end + + opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + init_info(email_file, apply_info) + end + + opts.on_tail('-h', '--help', 'show this message') do + puts opts + exit + end +end -def build_message(email, message_id, infos) +options.parse!(ARGV) + +def build_message(email, acct_infos) message = <<~EMAIL_MESSAGE To: #{email} - Message-ID: #{message_id} Subject: jumper account is ready - Dear #{email} + Dear user: Thank you for joining us. You can use the following command to login the jumper server: login command: - ssh -p #{infos['jumper_port']} #{infos['account']}@#{infos['jumper_ip']} + ssh -p #{acct_infos['jumper_port']} #{acct_infos['account']}@#{acct_infos['jumper_ip']} - account passwd: - account_password: #{infos['passwd']} + account password: + #{acct_infos['passwd']} regards compass-ci @@ -79,67 +95,26 @@ def build_message(email, message_id, infos) return message end -def email_addr(mail_content) - msg = 'not an applying account email' - - raise msg unless mail_content.subject =~ /apply ssh account/i - - email = mail_content.from.join(',') - - return email -end - -# def check_email_available(mail_content, email) -# oos_list = File.read('/c/upstream-repos/repo_list').split(/\n/) -# url = mail_content.body.decoded.split(/\n/).find { |line| line =~ /https?:\/\// } -# base_url = url.split('/')[0,5].join('/') -# message = 'The url is not in upstream repo_list' -# -# raise message unless oos_list.include? base_url -# -# url_fdback = %x(curl #{url}) -# email_index = url_fdback.index email -# -# message = 'No commit info found from the url for the email' -# raise message unless email_index -# end - -def email_message_id(mail_content) - message_id = mail_content.message_id - return message_id -end - -def pub_key_value(mail_content) - pub_key = mail_content.body.decoded.split(/\n/).find { |line| line =~ /ssh-rsa/ } - return pub_key -end - def account_info(pub_key) - account_info_str = %x(curl -XGET '#{JUMPER_IP}:#{JUMPER_PORT}/assign_account' -d "pub_key: #{pub_key}") - account_info = JSON.parse account_info_str - - return account_info + account_info_str = if pub_key.nil? + %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account') + else + %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d "pub_key: #{pub_key}") + end + JSON.parse account_info_str end -def send_account(mail_content) - email = email_addr(mail_content) - message_id = email_message_id(mail_content) - # check_email_available(mail_content, email) +def send_account(apply_info) + message = "No email address specified\n" + message += "use -e email_addr add a email address\n" + message += 'or use -f to add a mail file' + raise message if apply_info['email_addr'].nil? - pub_key = pub_key_value(mail_content) - acct_info = account_info(pub_key) + acct_info = account_info(apply_info['pub_key']) - message = build_message(email, message_id, acct_info) + message = build_message(apply_info['email_addr'], acct_info) %x(curl -XPOST '#{CRYSTAL_INTRANET}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end -def read_mail_content(mail_file) - mail_content = Mail.read(mail_file) - - return mail_content -end - -mail_file = ARGV[0] -mail_content = read_mail_content(mail_file) -send_account(mail_content) +send_account(apply_info) -- 2.23.0 [View Less]

2 3

[PATCH v3 compass-ci] /src/lib/web_backend.rb: fix "/compare_candidates" tbox_group regex bug
by Zhang Yuhang 21 Oct '20

21 Oct '20

Info: "/compare_canditates" error, due to fault regular expression. Detail: Miss a couple brackets, so will match /\d+$/. Error Flow: 1. like "xxx123" will be matched 2. index = "xxx123".index('--') || "xxx123".rindex('-') 3. index equal "nil" 4. r = r[0, nil] will raise an error "TypeError (no implicit conversion from nil to integer)" Solution: Fill a couple brackets, so will match "vm-abc-0abc-abc" or "vm-abc-0abc-012". Signed-off-by: Zhang Yuhang <… [View More]

1 1

[PATCH v4 compass-ci 2/2] container/assign-account: answerback-email.rb
by Luan Shengde 21 Oct '20

21 Oct '20

add parameters for answerback-email -e,--email email_addr: add email address -s,--ssh-pubkey pub_key: add a ssh login pub_key -f,--raw-email mail_file: add email address[, pub_key] via email file why: easier for administrator to manully assign jumper account for user with the parameters Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/answerback-email.rb | 42 ++++++++++---------- 1 file changed, 21 insertions(+), 21 deletions(-)… [View More] diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 5a5234c..a3108cb 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -16,48 +16,48 @@ require_relative '../defconfig' names = Set.new %w[ JUMPER_HOST JUMPER_PORT - CRYSTAL_INTRANET - SEND_MAIL_PORT + SEND_MAIL_HOST + SEND_INTERNET_MAIL_PORT ] defaults = relevant_defaults(names) JUMPER_HOST = defaults['JUMPER_HOST'] || '183.134.196.212' JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 -CRYSTAL_INTRANET = defaults['CRYSTAL_INTRANET'] || localhost -SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 11312 +SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || localhost +SEND_MAIL_PORT = defaults['SEND_INTERNET_MAIL_PORT'] || 11312 apply_info = { - 'email_addr' => nil, - 'pub_key' => nil + 'my_email' => nil, + 'my_pub_key' => nil } def init_info(email_file, apply_info) mail_content = Mail.read(email_file) - email_addr = mail_content.from[0] - pub_key = mail_content.part[1].body.decoded.gsub(/\r|\n/, '') if mail_content.part[1].filename == 'id_rsa.pub' + my_email = mail_content.from[0] + my_pub_key = mail_content.part[1].body.decoded.gsub(/\r|\n/, '') if mail_content.part[1].filename == 'id_rsa.pub' - apply_info['email_addr'] = email_addr - apply_info['pub_key'] = pub_key + apply_info['my_email'] = my_email + apply_info['my_pub_key'] = my_pub_key apply_info end options = OptionParser.new do |opts| - opts.banner = "Usage: answerback-mail.rb [--email email] [--ssh-pubkey pub_key] [--raw-email email_file]\n" + opts.banner = "Usage: answerback-mail.rb [--email email] [--ssh-pubkey pub_key_file] [--raw-email email_file]\n" opts.banner += " -e or -f is required\n" opts.banner += ' -s is optional when use -e' opts.separator '' opts.separator 'options:' - opts.on('-e email_addr', '--email email_addr', 'appoint email address') do |email_addr| - apply_info['email_addr'] = email_addr + opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + apply_info['my_email'] = email_address end - opts.on('-s pub_key', '--ssh-pubkey pub_key', 'ssh pub_key to enable keyless login') do |pub_key| - apply_info['pub_key'] = pub_key + opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', 'ssh pub_key file, enable keyless login') do |pub_key_file| + apply_info['my_pub_key'] = File.read(pub_key_file) end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| @@ -106,15 +106,15 @@ end def send_account(apply_info) message = "No email address specified\n" - message += "use -e email_addr add a email address\n" - message += 'or use -f to add a mail file' - raise message if apply_info['email_addr'].nil? + message += "use -e email_address add a email address\n" + message += 'or use -f to add a email file' + raise message if apply_info['my_email'].nil? - acct_info = account_info(apply_info['pub_key']) + acct_info = account_info(apply_info['my_pub_key']) - message = build_message(apply_info['email_addr'], acct_info) + message = build_message(apply_info['my_email'], acct_info) - %x(curl -XPOST '#{CRYSTAL_INTRANET}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") + %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end send_account(apply_info) -- 2.23.0 [View Less]

1 1

[PATCH v4 compass-ci 2/6] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 21 Oct '20

21 Oct '20

setup smtp for email server Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..3f181ba --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,20 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All … [View More]

2 2

[PATCH v2 compass-ci] /src/lib/web_backend.rb: fix "/compare_candidates" tbox_group regex bug
by Zhang Yuhang 21 Oct '20

21 Oct '20

info: "/compare_canditates" error, due to fault regular expression before: /^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$/ afeter: /^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$/ before miss a couple brackets, so will match /\d+$/. It will cause an exception. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 48cfadb..fda01ca 100644 --- a/src/lib/web_backend.… [View More]

2 2