November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v7 compass-ci] answerback-email: fix bugs and optimize
by Luan Shengde 30 Nov '20

30 Nov '20

1. set default option '-h' [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined 'nil' for my_email when execute answerback-email with no '-e' or '-f', it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' 3. option parse error [why] if add no required value for option, the option will use the field that follows as its value. example: ./answerback-email.rb -e -n name Not a standard format email: -n. [how] add value check for options that need a value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 49 ++++++++++++++++++-- 1 file changed, 44 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..825d815 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -94,19 +94,45 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + unless email_address =~ /[^@]+@[\d\w]+\.[\w\d]+/ + message = "Not a standard format email: #{email_address}.\n\n" + puts message + + return false + end + my_info['my_email'] = email_address end opts.on('-n name', '--name name', 'appoint name') do |name| + unless name =~ /^\w[\w\d ]+/ + message = "Name should only contains letters, digits and spaces\n\n" + puts message + + return false + end + stdin_info['my_name'] = name end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| + unless File.exist? pub_key_file + message = "File not found: #{pub_key_file}.\n\n" + puts message + + return false + end stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + unless File.exist? email_file + message = "Email file not found: #{email_file}.\n\n" + puts message + + return false + end mail_content = Mail.read(email_file) init_info(mail_content, email_info, my_info) end @@ -121,12 +147,16 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + message = "-l: bad value #{value}, please use y|n\n\n" + puts message + + return false end end @@ -136,6 +166,15 @@ options = OptionParser.new do |opts| end end +# if no option specified, set default option '-h' +# if both '-e' and '-f' are not specified, will clear the ARGV, +# and use '-h' instead +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +191,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v2 lkp-tests] spec/submit_spec.rb: update submit_job
by Hu Xuejiao 30 Nov '20

30 Nov '20

[why] Originally submit_job is too complicated, it need to be simplified. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit_spec.rb | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 4096cb73..2ea689ec 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -20,10 +20,12 @@ def traverse_file(output_dir) end end -def submit_job(submit_item) - submit_yaml_path = File.join("#{LKP_SRC}/spec/submit", submit_item) - Dir.glob("#{submit_yaml_path}/*.yaml").each do |yaml_file| - output_dir = File.join(submit_yaml_path, File.basename(yaml_file, '.yaml')) +def submit_job() + Dir.glob("#{LKP_SRC}/spec/submit/*/*.yaml").each do |yaml_file| + next unless File.exist?(yaml_file) + + path = File.dirname(yaml_file) + output_dir = File.join(path, File.basename(yaml_file, '.yaml')) submit_cmd = [ "#{LKP_SRC}/sbin/submit", '-o', output_dir, @@ -36,24 +38,4 @@ def submit_job(submit_item) end end -describe 'submit job spec' do - it 'link jobs spec' do - submit_job('link_jobs') - end - - it 'link matrix' do - submit_job('matrix') - end - - it 'separate yaml spec' do - submit_job('separate_yaml') - end - - it 'job on fail' do - submit_job('job_on_fail') - end - - it 'merge yaml' do - submit_job('merge_yaml') - end -end +submit_job -- 2.23.0

2 1

[PATCH v6 compass-ci] answerback-email: fix bugs and optimizi
by Luan Shengde 30 Nov '20

30 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined nil for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' 3. option parse error [why] if add no required value for option, it will use the field that follow as the value, example: ./answerback-email.rb -e -n name Not a standard format email: -n. [how] add value check for options that need a value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 49 ++++++++++++++++++-- 1 file changed, 44 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..825d815 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -94,19 +94,45 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + unless email_address =~ /[^@]+@[\d\w]+\.[\w\d]+/ + message = "Not a standard format email: #{email_address}.\n\n" + puts message + + return false + end + my_info['my_email'] = email_address end opts.on('-n name', '--name name', 'appoint name') do |name| + unless name =~ /^\w[\w\d ]+/ + message = "Name should only contains letters, digits and spaces\n\n" + puts message + + return false + end + stdin_info['my_name'] = name end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| + unless File.exist? pub_key_file + message = "File not found: #{pub_key_file}.\n\n" + puts message + + return false + end stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + unless File.exist? email_file + message = "Email file not found: #{email_file}.\n\n" + puts message + + return false + end mail_content = Mail.read(email_file) init_info(mail_content, email_info, my_info) end @@ -121,12 +147,16 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + message = "-l: bad value #{value}, please use y|n\n\n" + puts message + + return false end end @@ -136,6 +166,15 @@ options = OptionParser.new do |opts| end end +# if no option specified, set default option '-h' +# if both '-e' and '-f' are not specified, will clear the ARGV, +# and use '-h' instead +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +191,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v3 compass-ci 4/4] user-client/maintain: send report mail
by Liu Yinsi 30 Nov '20

30 Nov '20

call $CCI_SRC/lib/mail_client.rb to mail. 'LOG' is a file => walk-test.report, job result will write in it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 42a8b97..2ec95ce 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,17 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def mail_report + message = File.read(LOG).chomp + data = " + subject: os rootfs test report + to: #{EMAIL_ADDRESS} + body: #{message}" + + email = MailClient.new + email.send_mail(data) +end + def write_report(report_all) File.open(LOG, 'a') do |f| log = Logger.new(f, 'weekly') -- 2.23.0

1 0

[PATCH v3 compass-ci 3/4] user-client/maintain: write job result into report
by Liu Yinsi 30 Nov '20

30 Nov '20

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 4f3bc88..42a8b97 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,16 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def write_report(report_all) + File.open(LOG, 'a') do |f| + log = Logger.new(f, 'weekly') + log.formatter = proc { |_severity, datetime, _progname, msg| + "[#{datetime}]: #{msg}\n" + } + log.info(report_all) + end +end + def monitor(query, actions, timeout) monitor = Monitor.new monitor.overrides = query -- 2.23.0

1 0

[PATCH v3 compass-ci 2/4] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 30 Nov '20

30 Nov '20

1. monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2. query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 091b01b..4f3bc88 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,46 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + + %w[boot extract_finished].each do |state| + result_array = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_array.empty? + end + + time_list +end + +def es_search(job_id) + es_list = [] + + es = ESQuery.new + result_dict = es.query_by_id(job_id) + %w[job_state result_root].each do |field| + es_list.push(result_dict[field]) unless result_dict.nil? + end + + es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

1 0

[PATCH v2 compass-ci] service/scheduler: initialize a Sched for each API
by Wu Zhende 30 Nov '20

30 Nov '20

So can use Sched's instance variable in Sched's any interface Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 7 +++--- src/scheduler/find_job_boot.cr | 6 ++--- src/scheduler/find_next_job_boot.cr | 6 ++--- src/scheduler/request_cluster_state.cr | 14 ++++++------ src/scheduler/scheduler.cr | 31 +++++++++++++++----------- src/scheduler/submit_job.cr | 4 ++-- src/scheduler/update_job_parameter.cr | 6 ++--- 7 files changed, 40 insertions(+), 34 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2be7d41..26ddd92 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -25,12 +25,13 @@ class Sched property redis property block_helper - def initialize + def initialize(env : HTTP::Server::Context) @es = Elasticsearch::Client.new @redis = Redis::Client.new @task_queue = TaskQueueAPI.new @block_helper = BlockHelper.new @rgc = RemoteGitClient.new + @env = env end def normalize_mac(mac : String) @@ -67,7 +68,7 @@ class Sched extra_job_fields) if Dir.glob(full_path_patterns).size > 0 end - def update_tbox_wtmp(env : HTTP::Server::Context) + def update_tbox_wtmp testbox = "" hash = Hash(String, String).new @@ -75,7 +76,7 @@ class Sched hash["time"] = time %w(mac ip job_id tbox_name tbox_state).each do |parameter| - if (value = env.params.query[parameter]?) + if (value = @env.params.query[parameter]?) case parameter when "tbox_name" testbox = value diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index b5a23c5..9805464 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -2,9 +2,9 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. class Sched - def find_job_boot(env : HTTP::Server::Context) - value = env.params.url["value"] - boot_type = env.params.url["boot_type"] + def find_job_boot + value = @env.params.url["value"] + boot_type = @env.params.url["boot_type"] case boot_type when "ipxe" diff --git a/src/scheduler/find_next_job_boot.cr b/src/scheduler/find_next_job_boot.cr index 807a9ae..09abbd5 100644 --- a/src/scheduler/find_next_job_boot.cr +++ b/src/scheduler/find_next_job_boot.cr @@ -2,9 +2,9 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. class Sched - def find_next_job_boot(env) - hostname = env.params.query["hostname"]? - mac = env.params.query["mac"]? + def find_next_job_boot + hostname = @env.params.query["hostname"]? + mac = @env.params.query["mac"]? if !hostname && mac hostname = @redis.hash_get("sched/mac2host", normalize_mac(mac)) end diff --git a/src/scheduler/request_cluster_state.cr b/src/scheduler/request_cluster_state.cr index 72e3810..79a13cc 100644 --- a/src/scheduler/request_cluster_state.cr +++ b/src/scheduler/request_cluster_state.cr @@ -6,9 +6,9 @@ class Sched # all request states: # wait_ready | abort | failed | finished | wait_finish | # write_state | roles_ip - def request_cluster_state(env) - request_state = env.params.query["state"] - job_id = env.params.query["job_id"] + def request_cluster_state + request_state = @env.params.query["state"] + job_id = @env.params.query["job_id"] cluster_id = @redis.hash_get("sched/id2cluster", job_id).not_nil! cluster_state = "" @@ -41,10 +41,10 @@ class Sched return cluster_state when "write_state" - node_roles = env.params.query["node_roles"] - node_ip = env.params.query["ip"] - direct_ips = env.params.query["direct_ips"] - direct_macs = env.params.query["direct_macs"] + node_roles = @env.params.query["node_roles"] + node_ip = @env.params.query["ip"] + direct_ips = @env.params.query["direct_ips"] + direct_macs = @env.params.query["direct_macs"] job_info = {"roles" => node_roles, "ip" => node_ip, diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 5f97567..98b8e2e 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -30,12 +30,17 @@ require "../lib/sched" module Scheduler VERSION = "0.2.0" - sched = Sched.new + add_context_storage_type(Sched) before_all do |env| + env.set "sched", Sched.new(env) env.response.headers["Connection"] = "close" end + def self.get_sched_from_env(env) + env.get("sched").as(Sched) + end + # for debug (maybe kemal debug|logger does better) def self.debug_message(env, response) puts %({"from": "#{env.request.remote_address}", "response": #{response.to_json}}) @@ -53,7 +58,7 @@ module Scheduler # /boot.xxx/host/${hostname} # /boot.yyy/mac/${mac} get "/boot.:boot_type/:parameter/:value" do |env| - response = sched.find_job_boot(env) + response = get_sched_from_env(env).find_job_boot job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? puts %({"job_id": "#{job_id}", "job_state": "boot"}) if job_id @@ -63,7 +68,7 @@ module Scheduler # /~lkp/cgi-bin/gpxelinux.cgi?hostname=:hostname&mac=:mac&last_kernel=:last_kernel get "/~lkp/cgi-bin/gpxelinux.cgi" do |env| - response = sched.find_next_job_boot(env) + response = get_sched_from_env(env).find_next_job_boot job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? puts %({"job_id": "#{job_id}", "job_state": "boot"}) if job_id @@ -75,7 +80,7 @@ module Scheduler # - echo job_id to caller # -- job_id = "0" ? means failed post "/submit_job" do |env| - job_messages = sched.submit_job(env) + job_messages = get_sched_from_env(env).submit_job job_messages.each do |job_message| puts job_message.to_json @@ -103,7 +108,7 @@ module Scheduler # curl -X PUT "http://localhost:3000/set_host_mac?hostname=wfg&mac=00-01-02-03-04-05" put "/set_host_mac" do |env| if (client_hostname = env.params.query["hostname"]?) && (client_mac = env.params.query["mac"]?) - sched.set_host_mac(client_mac, client_hostname) + get_sched_from_env(env).set_host_mac(client_mac, client_hostname) "Done" else @@ -114,7 +119,7 @@ module Scheduler # curl -X PUT "http://localhost:3000/set_host2queues?queues=vm-2p8g.aarch64&host=vm-2p8g.a…" put "/set_host2queues" do |env| if (client_queues = env.params.query["queues"]?) && (client_host = env.params.query["host"]?) - sched.set_host2queues(client_host, client_queues) + get_sched_from_env(env).set_host2queues(client_host, client_queues) "Done" else @@ -125,7 +130,7 @@ module Scheduler # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| if client_mac = env.params.query["mac"]? - sched.del_host_mac(client_mac) + get_sched_from_env(env).del_host_mac(client_mac) "Done" else @@ -136,7 +141,7 @@ module Scheduler # curl -X PUT "http://localhost:3000/del_host2queues?host=vm-2p8g.aarch64" put "/del_host2queues" do |env| if client_host = env.params.query["host"]? - sched.del_host2queues(client_host) + get_sched_from_env(env).del_host2queues(client_host) "Done" else @@ -150,7 +155,7 @@ module Scheduler # ?job_file=/lkp/scheduled/job.yaml&job_state=post_run&job_id=10 # ?job_file=/lkp/scheduled/job.yaml&loadavg=0.28 0.82 0.49 1/105 3389&start_time=1587725398&end_time=1587725698&job_id=10 get "/~lkp/cgi-bin/lkp-jobfile-append-var" do |env| - puts sched.update_job_parameter(env) + puts get_sched_from_env(env).update_job_parameter "Done" end @@ -173,7 +178,7 @@ module Scheduler # response: get "server ip" from cluster state, # return "server=<server ip>". get "/~lkp/cgi-bin/lkp-cluster-sync" do |env| - response = sched.request_cluster_state(env) + response = get_sched_from_env(env).request_cluster_state debug_message(env, response) @@ -187,14 +192,14 @@ module Scheduler # get job_id from request job_id = env.params.query["job_id"]? if job_id - puts sched.close_job(job_id) + puts get_sched_from_env(env).close_job(job_id) end "Done" end get "/~lkp/cgi-bin/lkp-wtmp" do |env| - puts sched.update_tbox_wtmp(env) + puts get_sched_from_env(env).update_tbox_wtmp "Done" end @@ -205,7 +210,7 @@ module Scheduler job_id = env.params.query["job_id"].to_s if testbox && ssh_port - sched.report_ssh_port(testbox, ssh_port) + get_sched_from_env(env).report_ssh_port(testbox, ssh_port) end puts %({"job_id": "#{job_id}", "state": "set ssh port", "ssh_port": "#{ssh_port}", "tbox_name": "#{testbox}"}) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 2836275..4e78102 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -2,8 +2,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. class Sched - def submit_job(env : HTTP::Server::Context) - body = env.request.body.not_nil!.gets_to_end + def submit_job + body = @env.request.body.not_nil!.gets_to_end job_content = JSON.parse(body) job = Job.new(job_content, job_content["id"]?) diff --git a/src/scheduler/update_job_parameter.cr b/src/scheduler/update_job_parameter.cr index 0692025..cee8b9b 100644 --- a/src/scheduler/update_job_parameter.cr +++ b/src/scheduler/update_job_parameter.cr @@ -2,8 +2,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. class Sched - def update_job_parameter(env : HTTP::Server::Context) - job_id = env.params.query["job_id"]? + def update_job_parameter + job_id = @env.params.query["job_id"]? if !job_id return false end @@ -13,7 +13,7 @@ class Sched job_content["id"] = job_id (%w(start_time end_time loadavg job_state)).each do |parameter| - value = env.params.query[parameter]? + value = @env.params.query[parameter]? if !value || value == "" next end -- 2.23.0

2 1

[PATCH v5 compass-ci] answerback-email: fix bugs and optimizi
by Luan Shengde 30 Nov '20

30 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined nil for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' 3. option parse error [why] if add no required value for option, it will use the field that follow as the value, example: ./answerback-email.rb -e -n name Not a standard format email: -n. [how] add value check for options that need a value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 51 +++++++++++++++++--- 1 file changed, 45 insertions(+), 6 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..ec1f5b7 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -94,19 +94,45 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + unless email_address =~ /[^@]+@[\d\w]+\.[\w\d]+/ + message = "Not a standard format email: #{email_address}.\n\n" + puts message + + return false + end + my_info['my_email'] = email_address end opts.on('-n name', '--name name', 'appoint name') do |name| + unless name =~ /^\w[\w\d ]+/ + message = "Name should only contains letters, digits and spaces\n\n" + puts message + + return false + end + stdin_info['my_name'] = name end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| + unless File.exist? pub_key_file + message = "File not found: #{pub_key_file}.\n\n" + puts message + + return false + end stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + unless File.exist? email_file + message = "Email file not found: #{email_file}.\n\n" + puts message + + return false + end mail_content = Mail.read(email_file) init_info(mail_content, email_info, my_info) end @@ -121,12 +147,16 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + message = "-l: bad value #{value}, please use y|n\n\n" + puts message + + return false end end @@ -136,6 +166,15 @@ options = OptionParser.new do |opts| end end +# if no option specified, set default option '-h' +# if both '-e' and '-f' are not specified, will clear the ARGV, +# and use '-h' instead +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +191,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) @@ -210,4 +249,4 @@ def store_account_info(my_info) es.put_source_by_id(my_info['my_email'], my_info) end -send_account(my_info, conf_info, email_info, my_info_es, stdin_info) +# send_account(my_info, conf_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 1

[PATCH compass-ci 4/5] user-client/maintain: write job result into report
by Liu Yinsi 30 Nov '20

30 Nov '20

close the file every time after open and write it, avoid duplicate printing of logs. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index d9d6bd6..7ed07c5 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -34,6 +34,16 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def write_report(report_all) + file = File.open(LOG, 'a') + log = Logger.new(file, 'weekly') + log.formatter = proc { |_severity, datetime, _progname, msg| + "[#{datetime}]: #{msg}\n" + } + log.info(report_all) + file.close +end + def monitor(query, actions, timeout) monitor = Monitor.new monitor.overrides = query -- 2.23.0

3 6

Re: [PATCH lkp-tests] job.rb: disable submit warn
by Ren Wen 30 Nov '20

30 Nov '20

> unless File.executable?(path) >+ next if File.symlink?(path) >+ Skip all symlinks here? How about skipping 'wait' only? Thanks, RenWen > log_warn "skip non-executable #{path}" unless path =~ /\.cr$/ > next > end >-- >2.23.0 >

1 0