December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] scheduler: prints logs in Sched class
by Wu Zhende 12 Dec '20

12 Dec '20

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 24 ++++++++-- src/scheduler/close_job.cr | 7 ++- src/scheduler/download_file.cr | 14 ++++++ src/scheduler/find_job_boot.cr | 6 ++- src/scheduler/find_next_job_boot.cr | 6 ++- src/scheduler/scheduler.cr | 64 +++++---------------------- src/scheduler/submit_job.cr | 10 +++-- src/scheduler/update_job_parameter.cr | 2 +- 8 files changed, 68 insertions(+), 65 deletions(-) create mode 100644 src/scheduler/download_file.cr diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6225ac2..10a51ab 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -21,6 +21,7 @@ require "../scheduler/close_job" require "../scheduler/request_cluster_state" require "../scheduler/update_job_parameter" require "../scheduler/create_job_cpio" +require "../scheduler/download_file" class Sched property es @@ -37,6 +38,15 @@ class Sched @log = env.log.as(JSONLogger) end + def debug_message(response) + @log.info(%({"from": "#{(a)env.request.remote_address}", "response": #{response.to_json}})) + end + + def alive(version) + debug_message("Env= {\n#{`export`}}") + "LKP Alive! The time is #{Time.local}, version = #{version}" + end + def normalize_mac(mac : String) mac.gsub(":", "-") end @@ -97,11 +107,19 @@ class Sched # json log hash["testbox"] = testbox - return hash.to_json + @log.info(hash.to_json) end - def report_ssh_port(testbox : String, ssh_port : String) - @redis.hash_set("sched/tbox2ssh_port", testbox, ssh_port) + def report_ssh_port + testbox = @env.params.query["tbox_name"] + ssh_port = @env.params.query["ssh_port"].to_s + job_id = @env.params.query["job_id"].to_s + + if testbox && ssh_port + @redis.hash_set("sched/tbox2ssh_port", testbox, ssh_port) + end + + @log.info(%({"job_id": "#{job_id}", "state": "set ssh port", "ssh_port": "#{ssh_port}", "tbox_name": "#{testbox}"})) end private def query_consumable_keys(shortest_queue_name) diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index 2c33ffd..579c2c0 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -2,7 +2,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. class Sched - def close_job(job_id : String) + def close_job + job_id = @env.params.query["job_id"]? + return unless job_id + job = @redis.get_job(job_id) delete_access_key_file(job) if job @@ -22,7 +25,7 @@ class Sched @redis.remove_finished_job(job_id) - return %({"job_id": "#{job_id}", "job_state": "complete"}) + @log.info(%({"job_id": "#{job_id}", "job_state": "complete"})) end def delete_access_key_file(job : Job) diff --git a/src/scheduler/download_file.cr b/src/scheduler/download_file.cr new file mode 100644 index 0000000..e1f0c9a --- /dev/null +++ b/src/scheduler/download_file.cr @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Sched + def download_file + job_id = @env.params.url["job_id"] + job_package = @env.params.url["job_package"] + file_path = ::File.join [Kemal.config.public_folder, job_id, job_package] + + @log.info(%({"job_id": "#{job_id}", "job_state": "download"})) + + send_file @env, file_path + end +end diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index fb94e91..5a11523 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -16,7 +16,11 @@ class Sched host = value end - get_job_boot(host, boot_type) + response = get_job_boot(host, boot_type) + job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? + @log.info(%({"job_id": "#{job_id}", "job_state": "boot"})) if job_id + + response end # auto submit a job to collect the host information diff --git a/src/scheduler/find_next_job_boot.cr b/src/scheduler/find_next_job_boot.cr index 09abbd5..0bdfe5b 100644 --- a/src/scheduler/find_next_job_boot.cr +++ b/src/scheduler/find_next_job_boot.cr @@ -9,6 +9,10 @@ class Sched hostname = @redis.hash_get("sched/mac2host", normalize_mac(mac)) end - get_job_boot(hostname, "ipxe") + response = get_job_boot(hostname, "ipxe") + job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? + @log.info(%({"job_id": "#{job_id}", "job_state": "boot"})) if job_id + + response end end diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 4fde6e3..7ec149c 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -41,15 +41,9 @@ module Scheduler env.create_sched end - # for debug (maybe kemal debug|logger does better) - def self.debug_message(env, response) - env.log.info(%({"from": "#{env.request.remote_address}", "response": #{response.to_json}})) - end - # echo alive get "/" do |env| - debug_message(env, "Env= {\n#{`export`}}") - "LKP Alive! The time is #{Time.local}, version = #{VERSION}" + env.sched.alive(VERSION) end # for XXX_runner get job @@ -58,46 +52,24 @@ module Scheduler # /boot.xxx/host/${hostname} # /boot.yyy/mac/${mac} get "/boot.:boot_type/:parameter/:value" do |env| - response = env.sched.find_job_boot - - job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? - env.log.info(%({"job_id": "#{job_id}", "job_state": "boot"})) if job_id - - response + env.sched.find_job_boot end # /~lkp/cgi-bin/gpxelinux.cgi?hostname=:hostname&mac=:mac&last_kernel=:last_kernel get "/~lkp/cgi-bin/gpxelinux.cgi" do |env| - response = env.sched.find_next_job_boot - - job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? - env.log.info(%({"job_id": "#{job_id}", "job_state": "boot"})) if job_id - - response + env.sched.find_next_job_boot end # enqueue # - echo job_id to caller # -- job_id = "0" ? means failed post "/submit_job" do |env| - job_messages = env.sched.submit_job - - job_messages.each do |job_message| - env.log.info(job_message.to_json) - end - - job_messages.to_json + env.sched.submit_job.to_json end # file download server get "/job_initrd_tmpfs/:job_id/:job_package" do |env| - job_id = env.params.url["job_id"] - job_package = env.params.url["job_package"] - file_path = ::File.join [Kemal.config.public_folder, job_id, job_package] - - env.log.info(%({"job_id": "#{job_id}", "job_state": "download"})) - - send_file env, file_path + env.sched.download_file end # client(runner) report its hostname and mac @@ -155,7 +127,7 @@ module Scheduler # ?job_file=/lkp/scheduled/job.yaml&job_state=post_run&job_id=10 # ?job_file=/lkp/scheduled/job.yaml&loadavg=0.28 0.82 0.49 1/105 3389&start_time=1587725398&end_time=1587725698&job_id=10 get "/~lkp/cgi-bin/lkp-jobfile-append-var" do |env| - env.log.info(env.sched.update_job_parameter) + env.sched.update_job_parameter "Done" end @@ -178,42 +150,26 @@ module Scheduler # response: get "server ip" from cluster state, # return "server=<server ip>". get "/~lkp/cgi-bin/lkp-cluster-sync" do |env| - response = env.sched.request_cluster_state - - debug_message(env, response) - - response + env.sched.request_cluster_state end # client(runner) report job post_run finished # /~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job.yaml&job_id=40 # curl "http://localhost:3000/~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job…" get "/~lkp/cgi-bin/lkp-post-run" do |env| - # get job_id from request - job_id = env.params.query["job_id"]? - if job_id - env.log.info(env.sched.close_job(job_id)) - end + env.sched.close_job "Done" end get "/~lkp/cgi-bin/lkp-wtmp" do |env| - env.log.info(env.sched.update_tbox_wtmp) + env.sched.update_tbox_wtmp "Done" end get "/~lkp/cgi-bin/report_ssh_port" do |env| - testbox = env.params.query["tbox_name"] - ssh_port = env.params.query["ssh_port"].to_s - job_id = env.params.query["job_id"].to_s - - if testbox && ssh_port - env.sched.report_ssh_port(testbox, ssh_port) - end - - env.log.info(%({"job_id": "#{job_id}", "state": "set ssh port", "ssh_port": "#{ssh_port}", "tbox_name": "#{testbox}"})) + env.sched.report_ssh_port "Done" end diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 3909cb7..4303b1d 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -11,20 +11,24 @@ class Sched cluster_file = job["cluster"] if cluster_file.empty? || cluster_file == "cs-localhost" - return submit_single_job(job) + response = submit_single_job(job) else cluster_config = get_cluster_config(cluster_file, job.lkp_initrd_user, job.os_arch).not_nil! - return submit_cluster_job(job, cluster_config) + response = submit_cluster_job(job, cluster_config) end rescue ex @log.warn(ex.inspect_with_backtrace) - return [{ + response = [{ "job_id" => "0", "message" => ex.to_s, "job_state" => "submit", }] + ensure + response.each do |job_message| + @log.info(job_message.to_json) + end end # return: diff --git a/src/scheduler/update_job_parameter.cr b/src/scheduler/update_job_parameter.cr index cee8b9b..da528c1 100644 --- a/src/scheduler/update_job_parameter.cr +++ b/src/scheduler/update_job_parameter.cr @@ -29,6 +29,6 @@ class Sched # json log log = job_content.dup log["job_id"] = log.delete("id").not_nil! - return log.to_json + @log.info(log.to_json) end end -- 2.23.0

1 0

[PATCH compass-ci 2/2] mail-robot: update email message
by Luan Shengde 12 Dec '20

12 Dec '20

update email message with new key: my_uuid => my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index a88626c..834e96f 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -27,7 +27,7 @@ def build_apply_account_email(my_info) EOF mkdir -p ~/.config/compass-ci/include/lab cat >> ~/.config/compass-ci/include/lab/#{ENV['lab']}.yaml <<-EOF - my_uuid: #{my_info['my_uuid']} + my_token: #{my_info['my_token']} EOF 2) download lkp-tests and dependencies -- 2.23.0

1 0

[PATCH compass-ci 1/2] mail-robot: use new key my_token for apply account
by Luan Shengde 12 Dec '20

12 Dec '20

[why] key my_uuid is renamed to my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb index 0fff3b5..93920f4 100755 --- a/container/mail-robot/lib/apply-account.rb +++ b/container/mail-robot/lib/apply-account.rb @@ -31,6 +31,7 @@ SEND_MAIL_PORT = ENV['SEND_MAIL_PORT'] || 49000 # - my_email # - my_name # - my_uuid +# - my_token # my_ssh_pubkey # store_account_info # call ESClient to store my_info @@ -38,6 +39,7 @@ SEND_MAIL_PORT = ENV['SEND_MAIL_PORT'] || 49000 # - my_email # - my_name # - my_uuid +# - my_token # - my_commit_url # - my_login_name # - my_ssh_pubkey @@ -62,6 +64,7 @@ class ApplyAccount # in order to successfully send email for failed parsing # firstly get my_email before execute parse_mail_content is needed @my_info['my_email'] = @mail_content.from[0] + @my_info['lab'] = ENV['lab'] parse_mail_content apply_my_account @@ -92,14 +95,17 @@ class ApplyAccount apply_info = {} if my_account_es my_ssh_pubkey_new = @my_info.delete('my_ssh_pubkey') + # my_token will take the place of my_uuid. + # to make the change with no sense, will keep the my_uuid for now + apply_info['my_token'] = my_account_es['my_uuid'] if my_account_es['my_token'].nil? apply_info.update my_account_es apply_info.update @my_info apply_info['my_ssh_pubkey'] = (apply_info['my_ssh_pubkey'] + my_ssh_pubkey_new).uniq @my_info.update apply_info apply_info['is_update_account'] = true else - my_uuid = %x(uuidgen).chomp - @my_info['my_uuid'] = my_uuid + my_token = %x(uuidgen).chomp + @my_info['my_token'] = my_token apply_info.update @my_info end apply_new_account(apply_info, my_account_es) @@ -112,10 +118,6 @@ class ApplyAccount @my_info['my_login_name'] = acct_info['my_login_name'] unless my_account_es end - def check_account_es - ESQuery.new(index: 'accounts').query_by_id(@my_info['my_email']) - end - def store_account_info es = ESClient.new(index: 'accounts') es.put_source_by_id(@my_info['my_email'], @my_info) -- 2.23.0

1 0

[PATCH compass-ci 3/3] answerback-email: rename key my_uuid with my_token
by Luan Shengde 12 Dec '20

12 Dec '20

key my_uuid is renamed to my_token. when applying/updating account, will use my_token instead for this change with no sense for old user(update account) keep key my_uuid and add key my_token for new user delete my_uuid and just add key my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c08f10e..401bd8c 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -29,12 +29,13 @@ JUMPER_HOST = defaults['JUMPER_HOST'] JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 +LAB = 'z9' my_info = { 'my_email' => nil, 'my_name' => nil, 'my_commit_url' => nil, - 'my_uuid' => nil, + 'my_token' => nil, 'my_login_name' => nil, 'my_ssh_pubkey' => [] } @@ -200,6 +201,7 @@ def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) new_stdin_pubkey = stdin_info.delete 'new_ssh_pubkey' new_pubkey = new_stdin_pubkey || new_email_pubkey + my_info['my_token'] = my_info_es['my_uuid'] if my_info['my_token'].nil? my_info.update my_info_es unless my_info_es.empty? my_info.update email_info unless email_info.empty? my_info.update stdin_info unless stdin_info.empty? @@ -221,7 +223,7 @@ def build_my_info_from_account_info(my_info, account_info, conf_info) end def check_server - return true if ENV['HOSTNAME'] == 'z9' + return true if ENV['HOSTNAME'] == LAB message = 'please run the tool on z9 server' puts message @@ -260,7 +262,7 @@ def send_account(my_info, conf_info, email_info, my_info_es, stdin_info) return unless check_my_email(my_info) return unless check_email_assigned_account(conf_info, my_info_es) - my_info['my_uuid'] = %x(uuidgen).chomp unless conf_info['is_update_account'] + my_info['my_token'] = %x(uuidgen).chomp unless conf_info['is_update_account'] build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) return unless check_my_name_exist(my_info) -- 2.23.0

1 0

[PATCH v2 compass-ci] logging: fix time difference of eight hours
by Wu Zhende 12 Dec '20

12 Dec '20

If the time zone information is not contained, the ES saves time as the UTC time by default. As a result, the kibana display time is eight hours later. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/master-fluentd/docker-fluentd.conf | 2 +- src/lib/json_logger.cr | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/container/master-fluentd/docker-fluentd.conf b/container/master-fluentd/docker-fluentd.conf index b3daeb4..7bf2229 100644 --- a/container/master-fluentd/docker-fluentd.conf +++ b/container/master-fluentd/docker-fluentd.conf @@ -11,7 +11,7 @@ @type record_transformer enable_ruby <record> - time ${time.strftime('%Y-%m-%dT%H:%M:%S')} + time ${time.strftime('%Y-%m-%dT%H:%M:%S.%3N+0800')} </record> </filter> diff --git a/src/lib/json_logger.cr b/src/lib/json_logger.cr index dd87020..6d61197 100644 --- a/src/lib/json_logger.cr +++ b/src/lib/json_logger.cr @@ -18,7 +18,7 @@ class JSONLogger < Logger Logger::Formatter.new do | severity, datetime, progname, msg, io| get_env_info(@env.as(HTTP::Server::Context)) if @env level_num = severity.to_i32 - datetime = datetime.to_s("%Y-%m-%dT%H:%M:%S") + datetime = datetime.to_s("%Y-%m-%dT%H:%M:%S.%3N+0800") logger_hash = JSON.parse(%({"level_num": #{level_num}, "level": "#{severity}", "time": "#{datetime}" -- 2.23.0

1 0

[PATCH compass-ci] README.md: modify the link address
by Li Ping 11 Dec '20

11 Dec '20

1 0

[PATCH compass-ci] container/defconfig.rb: split functions
by Liu Yinsi 11 Dec '20

11 Dec '20

split into 2 functions. function 1 -> meminfo_hash: create a more generic function by returning a hash, function 2 -> get_available_memory: create a dedicated function for use by es and logging-es container Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index d353bb3..482dbc8 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -34,10 +34,14 @@ def docker_rm(container) system "docker stop #{container} && docker rm -f #{container}" end +def meminfo_hash + File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } +end + def get_available_memory - memory = File.readlines('/proc/meminfo')[0].chomp.split[1].to_f / 1048576 + memtotal = meminfo_hash[0]['MemTotal'].to_f / 1048576 # set container available memory size, minimum size is 1024m, maximum size is 30720m, # take the middle value according to the system memory size. - [1024, 30720, Math.sqrt(memory) * 1024].sort[1].to_i + [1024, 30720, Math.sqrt(memtotal) * 1024].sort[1].to_i end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/4] container/defconfig.rb: limit container memory usage
by Liu Yinsi 11 Dec '20

11 Dec '20

use Math.sqrt(memory) to set container occupied memory according to system memory size, instead of setting fixed size, more reasonable. In addition, '-Xms' not support floating number, so use to_i. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index 05c56db..026a183 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -35,3 +35,11 @@ def docker_rm(container) system "docker stop #{container} && docker rm -f #{container}" end + +def set_available_memory + memory = `awk '($1 == "MemTotal:"){print $2/1048576}' /proc/meminfo`.to_f + + # set container available memory size, minimum size is 1g, maximum size is 30g, + # take the middle value according to the system memory size. + [1, 30, Math.sqrt(memory)].sort[1].to_i +end -- 2.23.0

2 2

[PATCH compass-ci] README.zh.md: modify the link address
by Li Ping 11 Dec '20

11 Dec '20

1 0

[PATCH lkp-tests] bin/lkp-setup-rootfs: modify test-machine is not restarted
by Hu Xuejiao 11 Dec '20

11 Dec '20

[why] When force_reboot is not added to the job.yaml, the test-machine is "no KERNEL found". Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- bin/lkp-setup-rootfs | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 0eabea75..cca1b53b 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -61,7 +61,10 @@ job_does_not_need_reboot() job_force_reboot() { - grep -q "^force_reboot: \+1$" $job + grep -q "^force_reboot: 1$" $job && return 0 + grep -q "^do_not_reboot: 1$" $job || return 0 + + return 1 } reboot_for_next_job() -- 2.23.0

1 0