- Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 27 Oct '20

27 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..c5f29d74 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n "$block_size" ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 27 Oct '20

27 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..c5f29d74 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n "$block_size" ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH v1 compass-ci] based on queue field add the job into specify queue
by Xiao Shenwei 27 Oct '20

27 Oct '20

use queue replace tbox_group Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 692180f..d4929cd 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -30,12 +30,15 @@ class Sched mac.gsub(":", "-") end - def set_host_mac(mac : String, hostname : String) + def set_host_mac(mac : String, hostname : String, queues : String) + puts "queues: #{queues}" @redis.hash_set("sched/mac2host", normalize_mac(mac), hostname) + @redis.hash_set("sched/mac2queues", normalize_mac(mac), queues) end def del_host_mac(mac : String) @redis.hash_del("sched/mac2host", normalize_mac(mac)) + @redis.hash_del("sched/mac2queues", normalize_mac(mac)) end # return: @@ -251,24 +254,24 @@ class Sched # success: [{"job_id" => job_id, "message" => "", job_state => "submit"}] # failure: [{"job_id" => "0", "message" => err_msg, job_state => "submit"}] def submit_single_job(job) - tbox_group = job.tbox_group + queue = job.queue return [{ "job_id" => "0", - "message" => "get tbox group failed", + "message" => "get queue failed", "job_state" => "submit", - }] unless tbox_group + }] unless queue # only single job will has "idle job" and "execute rate limiter" if job["idle_job"].empty? - tbox_group += "#{job.get_uuid_tag}" + queue += "#{job.get_uuid_tag}" else - tbox_group = "#{tbox_group}/idle" + queue = "#{queue}/idle" end - job_id = add_task(tbox_group, job.lab) + job_id = add_task(queue, job.lab) return [{ "job_id" => "0", - "message" => "add task queue sched/#{tbox_group} failed", + "message" => "add task queue sched/#{queue} failed", "job_state" => "submit", }] unless job_id @@ -372,7 +375,7 @@ class Sched hostname = @redis.hash_get("sched/mac2host", normalize_mac(api_param)) if hostname.nil? # auto name new/unknown machine hostname = "sut-#{api_param}" - set_host_mac(api_param, hostname) + set_host_mac(api_param, hostname, "") # auto submit a job to collect the host information # grub hostname is link with ":", like "00:01:02:03:04:05" -- 2.23.0

2 2

[PATCH compass-ci] sched: mount default config dir for idle job
by Cao Xueliang 27 Oct '20

27 Oct '20

[why] There no default config yaml in scheduler container, when submit idle job use the submit script. [how] Mount default config dir in scheduler container. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/scheduler/start | 3 +++ src/scheduler/jobfile_operate.cr | 10 +++------- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/container/scheduler/start b/container/scheduler/start index d61616a..6a2a4b0 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -45,6 +45,8 @@ SCHED_HOST = defaults['SCHED_HOST'] || '172.17.0.1' FLUENTD_HOST = defaults['FLUENTD_HOST'] || '172.17.0.1' FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' FLUENTD_SERVER_PORT = defaults['FLUENTD_SERVER_PORT'] || '24224' +DEFAULT_CONFIG_DIR = '/etc/compass-ci/defaults' + docker_rm "s001-alpine-#{SCHED_PORT}" bind_list = [] @@ -70,6 +72,7 @@ cmd = %W[ -e FLUENTD_SERVER_PORT=#{FLUENTD_SERVER_PORT} -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} + -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro -v #{CCI_REPOS}/lab-#{LAB}:#{CCI_REPOS}/lab-#{LAB} -v /srv/cci/scheduler/alpine:/srv/cci/scheduler -v /etc/localtime:/etc/localtime:ro diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index e0e298f..d3c26c5 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -202,14 +202,10 @@ module Jobfile::Operate end def self.auto_submit_job(job_file, override_parameter, other_parameters = nil) - job_fields = ["SCHED_HOST=#{ENV["SCHED_HOST"]}", - "SCHED_PORT=#{ENV["SCHED_PORT"]}", - "GIT_SERVER=#{ENV["GIT_SERVER"]}", - "lab=#{ENV["lab"]}"] - - job_fields.concat(other_parameters) if other_parameters cmd = "#{ENV["LKP_SRC"]}/sbin/submit " - cmd += job_fields.join(" ") + if other_parameters + cmd += other_parameters.join(" ") + end cmd += " -s '#{override_parameter}' #{job_file}" puts `#{cmd}` end -- 2.23.0

1 0

[PATCH v3 compass-ci] add queue field into job.yaml
by Xiao Shenwei 27 Oct '20

27 Oct '20

[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/job.cr | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..0376f9f 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -73,6 +73,7 @@ class Job tbox_group testbox lab + queue initrd_pkg initrd_deps initrds_uri @@ -139,6 +140,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end private def append_init_field @@ -199,6 +201,16 @@ class Job self["result_service"] = "raw_upload" end + private def set_queue + return if @hash.has_key?("queue") + + # set default value + self["queue"] = tbox_group + if tbox_group.to_s.starts_with?(/(vm|dc).*/) + self["queue"] = "#{tbox_group}.#{arch}" + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0

3 4

[PATCH compass-ci] fix: taskqueue api: use same client for 1 server
by Tong Qunfeng 27 Oct '20

27 Oct '20

[why] interaction to taskqueue will be frequently, so keep same client as possible. - btw: caller, that do not need keep-alive with taskqueue, should set header["Connection"]="close", or active close the connection. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/taskqueue_api.cr | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/src/lib/taskqueue_api.cr b/src/lib/taskqueue_api.cr index 5a83a15..a8bbf16 100644 --- a/src/lib/taskqueue_api.cr +++ b/src/lib/taskqueue_api.cr @@ -4,25 +4,33 @@ require "json" require "http/client" +def loop_till_connectable + loop do + begin + return yield + rescue ex + puts "#{ex}, retrying in 5s..." + sleep(5) + end + end +end + class TaskQueueAPI def initialize - @port = ENV.has_key?("TASKQUEUE_PORT") ? ENV["TASKQUEUE_PORT"].to_i32 : 3060 - @host = ENV.has_key?("TASKQUEUE_HOST") ? ENV["TASKQUEUE_HOST"] : "172.17.0.1" + port = ENV.has_key?("TASKQUEUE_PORT") ? ENV["TASKQUEUE_PORT"].to_i32 : 3060 + host = ENV.has_key?("TASKQUEUE_HOST") ? ENV["TASKQUEUE_HOST"] : "172.17.0.1" + @client = HTTP::Client.new(host, port: port) end def add_task(service_queue_path : String, task : JSON::Any) params = HTTP::Params.encode({"queue" => service_queue_path}) - client = HTTP::Client.new(@host, port: @port) - response = client.post("/add?" + params, body: task.to_json) - client.close + response = loop_till_connectable { @client.post("/add?" + params, body: task.to_json) } arrange_response(response) end def query_keys(service_key_with_wild_char : String) params = HTTP::Params.encode({"queue" => service_key_with_wild_char}) - client = HTTP::Client.new(@host, port: @port) - response = client.get("/keys?" + params) - client.close + response = loop_till_connectable { @client.get("/keys?" + params) } arrange_response(response) end @@ -42,9 +50,7 @@ class TaskQueueAPI end private def response_put_api(cmd : String, params : String) - client = HTTP::Client.new(@host, port: @port) - response = client.put("/#{cmd}?" + params) - client.close + response = loop_till_connectable { @client.put("/#{cmd}?" + params) } arrange_response(response) end -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: close connection after API call
by Tong Qunfeng 27 Oct '20

27 Oct '20

[why] most client call to scheduler do not need keep-alive. when API call is finished, close the connection. [how] set response header["Connection"]="close" Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/scheduler/scheduler.cr | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 5befcf9..371e4ed 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -39,6 +39,7 @@ module Scheduler # echo alive get "/" do |env| + env.response.headers["Connection"] = "close" debug_message(env, "Env= {\n#{`export`}}") "LKP Alive! The time is #{Time.local}, version = #{VERSION}" end @@ -49,6 +50,7 @@ module Scheduler # /boot.xxx/host/${hostname} # /boot.yyy/mac/${mac} get "/boot.:boot_type/:parameter/:value" do |env| + env.response.headers["Connection"] = "close" response = sched.find_job_boot(env) job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? @@ -59,6 +61,7 @@ module Scheduler # /~lkp/cgi-bin/gpxelinux.cgi?hostname=:hostname&mac=:mac&last_kernel=:last_kernel get "/~lkp/cgi-bin/gpxelinux.cgi" do |env| + env.response.headers["Connection"] = "close" response = sched.find_next_job_boot(env) job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? @@ -71,6 +74,7 @@ module Scheduler # - echo job_id to caller # -- job_id = "0" ? means failed post "/submit_job" do |env| + env.response.headers["Connection"] = "close" job_messages = sched.submit_job(env) job_messages.each do |job_message| @@ -82,6 +86,7 @@ module Scheduler # file download server get "/job_initrd_tmpfs/:job_id/:job_package" do |env| + env.response.headers["Connection"] = "close" job_id = env.params.url["job_id"] job_package = env.params.url["job_package"] file_path = ::File.join [Kemal.config.public_folder, job_id, job_package] @@ -98,6 +103,7 @@ module Scheduler # # curl -X PUT "http://localhost:3000/set_host_mac?hostname=wfg&mac=00-01-02-03-04-05" put "/set_host_mac" do |env| + env.response.headers["Connection"] = "close" if (client_hostname = env.params.query["hostname"]?) && (client_mac = env.params.query["mac"]?) sched.set_host_mac(client_mac, client_hostname) @@ -109,6 +115,7 @@ module Scheduler # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| + env.response.headers["Connection"] = "close" if client_mac = env.params.query["mac"]? sched.del_host_mac(client_mac) @@ -124,6 +131,7 @@ module Scheduler # ?job_file=/lkp/scheduled/job.yaml&job_state=post_run&job_id=10 # ?job_file=/lkp/scheduled/job.yaml&loadavg=0.28 0.82 0.49 1/105 3389&start_time=1587725398&end_time=1587725698&job_id=10 get "/~lkp/cgi-bin/lkp-jobfile-append-var" do |env| + env.response.headers["Connection"] = "close" puts sched.update_job_parameter(env) "Done" @@ -147,6 +155,7 @@ module Scheduler # response: get "server ip" from cluster state, # return "server=<server ip>". get "/~lkp/cgi-bin/lkp-cluster-sync" do |env| + env.response.headers["Connection"] = "close" response = sched.request_cluster_state(env) debug_message(env, response) @@ -158,6 +167,7 @@ module Scheduler # /~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job.yaml&job_id=40 # curl "http://localhost:3000/~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job…" get "/~lkp/cgi-bin/lkp-post-run" do |env| + env.response.headers["Connection"] = "close" # get job_id from request job_id = env.params.query["job_id"]? if job_id @@ -168,12 +178,14 @@ module Scheduler end get "/~lkp/cgi-bin/lkp-wtmp" do |env| + env.response.headers["Connection"] = "close" puts sched.update_tbox_wtmp(env) "Done" end get "/~lkp/cgi-bin/report_ssh_port" do |env| + env.response.headers["Connection"] = "close" testbox = env.params.query["tbox_name"] ssh_port = env.params.query["ssh_port"].to_s job_id = env.params.query["job_id"].to_s -- 2.23.0

1 0

[PATCH v2 compass-ci] /lib/es_query.rb: add range query
by Zhang Yuhang 27 Oct '20

27 Oct '20

[why] Some queries need to limit the range of fields. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- lib/es_query.rb | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index aaaadc2..037213d 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -36,12 +36,27 @@ class ESQuery end end +# Range Query Example: +# range = { +# start_time: { +# gte: '2020-09-10 01:50:00', +# lte: '2020-09-10 01:53:00' +# }, +# end_time: { +# gt: '2020-09-10 01:52:00', +# lt: '2020-09-10 01:54:00' +# } +# } +# items['range'] = range +# build_mutli_field_subquery_body(items) def build_mutli_field_subquery_body(items) query_fields = [] items.each do |key, value| if value.is_a?(Array) inner_query = build_multi_field_or_query_body(key, value) query_fields.push({ bool: { should: inner_query } }) + elsif key.to_s == 'range' + query_fields.concat(value.map { |k, v| { range: { k => v } } }) else query_fields.push({ term: { key => value } }) end -- 2.23.0

2 2

[PATCH lab-z9 3/3] rename 3 files for memory change to 512G
by Zhang Dewan 27 Oct '20

27 Oct '20

--- ...200-2280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} | 0 ...200-2280-2s48p-256g--a105 => taishan200-2280-2s48p-512g--a105} | 0 ...an200-2280-2s48p-256g--a95 => taishan200-2280-2s48p-512g--a95} | 0 3 files changed, 0 insertions(+), 0 deletions(-) rename hosts/{taishan200-2280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} (100%) rename hosts/{taishan200-2280-2s48p-256g--a105 => taishan200-2280-2s48p-512g--a105} (100%) rename hosts/{taishan200-2280-2s48p-256g--a95 => taishan200-2280-2s48p-512g--a95} (100%) diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-512g--a102 similarity index 100% rename from hosts/taishan200-2280-2s48p-256g--a102 rename to hosts/taishan200-2280-2s48p-512g--a102 diff --git a/hosts/taishan200-2280-2s48p-256g--a105 b/hosts/taishan200-2280-2s48p-512g--a105 similarity index 100% rename from hosts/taishan200-2280-2s48p-256g--a105 rename to hosts/taishan200-2280-2s48p-512g--a105 diff --git a/hosts/taishan200-2280-2s48p-256g--a95 b/hosts/taishan200-2280-2s48p-512g--a95 similarity index 100% rename from hosts/taishan200-2280-2s48p-256g--a95 rename to hosts/taishan200-2280-2s48p-512g--a95 -- 2.23.0

1 0

[PATCH lab-z9 2/3] add new host file
by Zhang Dewan 27 Oct '20

27 Oct '20

--- hosts/m | 43 ++------------------------ hosts/taishan200-2280-2s48p-256g--a102 | 1 - hosts/taishan200-2280-2s48p-256g--a103 | 24 ++++++++++++++ hosts/taishan200-2280-2s48p-256g--a104 | 23 ++++++++++++++ hosts/taishan200-2280-2s48p-256g--a105 | 24 ++++++++++++++ hosts/taishan200-2280-2s48p-256g--a106 | 24 ++++++++++++++ hosts/taishan200-2280-2s48p-256g--a107 | 24 ++++++++++++++ hosts/taishan200-2280-2s48p-256g--a16 | 20 ++++++++++++ hosts/taishan200-2280-2s48p-256g--a35 | 21 +++++++++++++ hosts/taishan200-2280-2s48p-256g--a63 | 19 ++++++++++++ hosts/taishan200-2280-2s48p-256g--a87 | 20 ++++++++++++ hosts/taishan200-2280-2s48p-256g--a95 | 1 - 12 files changed, 202 insertions(+), 42 deletions(-) create mode 100644 hosts/taishan200-2280-2s48p-256g--a103 create mode 100644 hosts/taishan200-2280-2s48p-256g--a104 create mode 100644 hosts/taishan200-2280-2s48p-256g--a105 create mode 100644 hosts/taishan200-2280-2s48p-256g--a106 create mode 100644 hosts/taishan200-2280-2s48p-256g--a107 create mode 100644 hosts/taishan200-2280-2s48p-256g--a16 create mode 100644 hosts/taishan200-2280-2s48p-256g--a35 create mode 100644 hosts/taishan200-2280-2s48p-256g--a63 create mode 100644 hosts/taishan200-2280-2s48p-256g--a87 diff --git a/hosts/m b/hosts/m index b73dfb7..73ba010 100644 --- a/hosts/m +++ b/hosts/m @@ -1,40 +1,3 @@ -taishan200-2280-2s48p-256g--a11 a11 -taishan200-2280-2s48p-256g--a13 a13 -taishan200-2280-2s48p-256g--a14 a14 -taishan200-2280-2s48p-256g--a15 a15 -taishan200-2280-2s48p-256g--a17 a17 -taishan200-2280-2s48p-256g--a18 a18 -taishan200-2280-2s48p-256g--a19 a19 -taishan200-2280-2s48p-256g--a21 a21 -taishan200-2280-2s48p-256g--a22 a22 -taishan200-2280-2s48p-256g--a24 a24 -taishan200-2280-2s48p-256g--a25 a25 -taishan200-2280-2s48p-256g--a26 a26 -taishan200-2280-2s48p-256g--a27 a27 -taishan200-2280-2s48p-256g--a28 a28 -taishan200-2280-2s48p-256g--a29 a29 -taishan200-2280-2s48p-256g--a30 a30 -taishan200-2280-2s48p-256g--a31 a31 -taishan200-2280-2s48p-256g--a32 a32 -taishan200-2280-2s48p-256g--a33 a33 -taishan200-2280-2s48p-256g--a34 a34 -taishan200-2280-2s48p-256g--a52 a52 -taishan200-2280-2s48p-256g--a53 a53 -taishan200-2280-2s48p-256g--a60 a60 -taishan200-2280-2s48p-256g--a61 a61 -taishan200-2280-2s48p-256g--a62 a62 -taishan200-2280-2s48p-256g--a66 a66 -taishan200-2280-2s48p-256g--a67 a67 -taishan200-2280-2s48p-256g--a68 a68 -taishan200-2280-2s48p-256g--a69 a69 -taishan200-2280-2s48p-256g--a70 a70 -taishan200-2280-2s48p-256g--a71 a71 -taishan200-2280-2s64p-256g--a10 a10 -taishan200-2280-2s64p-256g--a3 a3 -taishan200-2280-2s64p-256g--a36 a36 -taishan200-2280-2s64p-256g--a4 a4 -taishan200-2280-2s64p-256g--a40 a40 -taishan200-2280-2s64p-256g--a41 a41 -taishan200-2280-2s64p-256g--a42 a42 -taishan200-2280-2s64p-256g--a8 a8 -taishan200-2280-2s64p-256g--a9 a9 +taishan200-2280-2s48p-256g--a16 a16 +taishan200-2280-2s48p-256g--a35 a35 +taishan200-2280-2s48p-256g--a87 a87 diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-256g--a102 index e1d3f10..267a962 100644 --- a/hosts/taishan200-2280-2s48p-256g--a102 +++ b/hosts/taishan200-2280-2s48p-256g--a102 @@ -1,4 +1,3 @@ -arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G diff --git a/hosts/taishan200-2280-2s48p-256g--a103 b/hosts/taishan200-2280-2s48p-256g--a103 new file mode 100644 index 0000000..4afef3c --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a103 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bdc69 + - /dev/disk/by-id/scsi-350000399b88bdd85 + - /dev/disk/by-id/scsi-350000399b88bdc4d + - /dev/disk/by-id/scsi-350000399b88bdc7d +mac_addr: + - 44:67:47:c9:d8:b8 + - 44:67:47:c9:d8:b9 + - 44:67:47:c9:d8:ba + - 44:67:47:c9:d8:bb + - 44:67:47:c9:d8:bc + - 44:67:47:c9:d8:bd + - 44:67:47:c9:d8:be + - 44:67:47:c9:d8:bf + - cc:05:77:ee:b5:4f + - cc:05:77:ee:b5:50 + - cc:05:77:ee:b5:51 + - cc:05:77:ee:b5:52 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.8 diff --git a/hosts/taishan200-2280-2s48p-256g--a104 b/hosts/taishan200-2280-2s48p-256g--a104 new file mode 100644 index 0000000..3c5b807 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a104 @@ -0,0 +1,23 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8892431 + - /dev/disk/by-id/scsi-350000399b8892f81 + - /dev/disk/by-id/scsi-350000399b8891c49 + - /dev/disk/by-id/scsi-350000399b88930d1 +mac_addr: + - 44:67:47:d7:6d:f0 + - 44:67:47:d7:6d:f1 + - 44:67:47:d7:6d:f2 + - 44:67:47:d7:6d:f3 + - 44:67:47:d7:6d:f4 + - 44:67:47:d7:6d:f5 + - 44:67:47:d7:6d:f6 + - 44:67:47:d7:6d:f7 + - cc:05:77:ee:b9:6f + - cc:05:77:ee:b9:70 + - cc:05:77:ee:b9:71 + - cc:05:77:ee:b9:72 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a105 b/hosts/taishan200-2280-2s48p-256g--a105 new file mode 100644 index 0000000..3c28d2d --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a105 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 512G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b891945d + - /dev/disk/by-id/scsi-350000399b8919135 + - /dev/disk/by-id/scsi-350000399b8919699 + - /dev/disk/by-id/scsi-350000399b89190d5 +mac_addr: + - 44:67:47:d7:76:c4 + - 44:67:47:d7:76:c5 + - 44:67:47:d7:76:c6 + - 44:67:47:d7:76:c7 + - 44:67:47:d7:76:c8 + - 44:67:47:d7:76:c9 + - 44:67:47:d7:76:ca + - 44:67:47:d7:76:cb + - cc:05:77:ee:b2:8f + - cc:05:77:ee:b2:90 + - cc:05:77:ee:b2:91 + - cc:05:77:ee:b2:92 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.10 diff --git a/hosts/taishan200-2280-2s48p-256g--a106 b/hosts/taishan200-2280-2s48p-256g--a106 new file mode 100644 index 0000000..a7b44a0 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a106 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b883bf99 + - /dev/disk/by-id/scsi-350000399b883ba89 + - /dev/disk/by-id/scsi-350000399b883b94d + - /dev/disk/by-id/scsi-350000399b883b981 +mac_addr: + - 44:67:47:86:1c:30 + - 44:67:47:86:1c:31 + - 44:67:47:86:1c:32 + - 44:67:47:86:1c:33 + - 44:67:47:86:1c:34 + - 44:67:47:86:1c:35 + - 44:67:47:86:1c:36 + - 44:67:47:86:1c:37 + - cc:05:77:ee:b7:df + - cc:05:77:ee:b7:e0 + - cc:05:77:ee:b7:e1 + - cc:05:77:ee:b7:e2 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.11 diff --git a/hosts/taishan200-2280-2s48p-256g--a107 b/hosts/taishan200-2280-2s48p-256g--a107 new file mode 100644 index 0000000..9d20e57 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a107 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8917c39 + - /dev/disk/by-id/scsi-350000399b8918005 + - /dev/disk/by-id/scsi-350000399b8900a4d + - /dev/disk/by-id/scsi-350000399b8900799 +mac_addr: + - 84:46:fe:73:a2:71 + - 84:46:fe:73:a2:72 + - 84:46:fe:73:a2:73 + - 84:46:fe:73:a2:74 + - 84:46:fe:73:a2:75 + - 84:46:fe:73:a2:76 + - 84:46:fe:73:a2:77 + - 84:46:fe:73:a2:78 + - 44:a1:91:35:1a:70 + - 44:a1:91:35:1a:71 + - 44:a1:91:35:1a:72 + - 44:a1:91:35:1a:73 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.12 diff --git a/hosts/taishan200-2280-2s48p-256g--a16 b/hosts/taishan200-2280-2s48p-256g--a16 new file mode 100644 index 0000000..62cb124 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a16 @@ -0,0 +1,20 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bf271 + - /dev/disk/by-id/scsi-350000399b88bf1a9 + - /dev/disk/by-id/scsi-350000399b88bf0d5 + - /dev/disk/by-id/scsi-350000399b88bf385 +mac_addr: + - 84:46:fe:3d:a2:17 + - 84:46:fe:3d:a2:18 + - 84:46:fe:3d:a2:19 + - 84:46:fe:3d:a2:1a + - 84:46:fe:3d:a2:1b + - 84:46:fe:3d:a2:1c + - 84:46:fe:3d:a2:1d + - 84:46:fe:3d:a2:1e +model_name: Kunpeng-920 +ipmi_ip: 9.3.5.6 diff --git a/hosts/taishan200-2280-2s48p-256g--a35 b/hosts/taishan200-2280-2s48p-256g--a35 new file mode 100644 index 0000000..ec1e524 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a35 @@ -0,0 +1,21 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 3 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886a71a5 + - /dev/disk/by-id/scsi-350000399886b66dd-part1 + - /dev/disk/by-id/scsi-350000399886b66dd-part2 + - /dev/disk/by-id/scsi-350000399886b66dd-part3 + - /dev/disk/by-id/scsi-362c97b1c6eedb00026e527a2b8b998f2 +mac_addr: + - 44:67:47:c9:ec:ec + - 44:67:47:c9:ec:ed + - 44:67:47:c9:ec:ee + - 44:67:47:c9:ec:ef + - 44:67:47:c9:ec:f0 + - 44:67:47:c9:ec:f1 + - 44:67:47:c9:ec:f2 + - 44:67:47:c9:ec:f3 +model_name: Kunpeng-920 +ipmi_ip: 9.3.9.12 diff --git a/hosts/taishan200-2280-2s48p-256g--a63 b/hosts/taishan200-2280-2s48p-256g--a63 new file mode 100644 index 0000000..6d4b636 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a63 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886b0431 + - /dev/disk/by-id/scsi-350000399886adc71 + - /dev/disk/by-id/scsi-350000399886b66f5 + - /dev/disk/by-id/scsi-350000399886b6b75 +mac_addr: + - 44:67:47:c9:e5:44 + - 44:67:47:c9:e5:45 + - 44:67:47:c9:e5:46 + - 44:67:47:c9:e5:47 + - 44:67:47:c9:e5:48 + - 44:67:47:c9:e5:49 + - 44:67:47:c9:e5:4a + - 44:67:47:c9:e5:4b +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a87 b/hosts/taishan200-2280-2s48p-256g--a87 new file mode 100644 index 0000000..b87417b --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a87 @@ -0,0 +1,20 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886a6b55 + - /dev/disk/by-id/scsi-350000399886ad4a5 + - /dev/disk/by-id/scsi-350000399886a7139 + - /dev/disk/by-id/scsi-350000399886acb21 +mac_addr: + - 44:67:47:c9:eb:66 + - 44:67:47:c9:eb:67 + - 44:67:47:c9:eb:68 + - 44:67:47:c9:eb:69 + - 44:67:47:c9:eb:6a + - 44:67:47:c9:eb:6b + - 44:67:47:c9:eb:6c + - 44:67:47:c9:eb:6d +model_name: Kunpeng-920 +ipmi_ip: 9.3.8.4 diff --git a/hosts/taishan200-2280-2s48p-256g--a95 b/hosts/taishan200-2280-2s48p-256g--a95 index 213d10f..3fd8124 100644 --- a/hosts/taishan200-2280-2s48p-256g--a95 +++ b/hosts/taishan200-2280-2s48p-256g--a95 @@ -1,4 +1,3 @@ -arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G -- 2.23.0

1 0

[PATCH compass-ci] feat: scheduler: comsume exists queue
by Tong Qunfeng 27 Oct '20

27 Oct '20

[why] when there has no job for a testbox, scheduler will consume 4-5 times testbox*1 tbox_group*1 tbox*1 tbox_group/idle*(1|2) [how] query taskqueue the queue-name first. if the queue-name not exist, then there should be no job for this queue. submit idle job at background, let the consume not wait for submit idle job finished. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/sched.cr | 52 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 47 insertions(+), 5 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..8fbe9aa 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -409,19 +409,26 @@ class Sched tbox_group = JobHelper.match_tbox_group(testbox) tbox = tbox_group.partition("--")[0] - boxes = [testbox, tbox_group, tbox] + queue_list = query_consumable_keys(tbox) + + boxes = ["sched/" + testbox, + "sched/" + tbox_group, + "sched/" + tbox, + "sched/" + tbox_group + "/idle"] boxes.each do |box| + next if queue_list.select(box).size == 0 count.times do - job = prepare_job("sched/#{box}", testbox) + job = prepare_job(box, testbox) return job if job sleep(1) unless count == 1 end end - # when find no job at "sched/#{tbox_group}" - # try to get from "sched/#{tbox_group}/idle" - return get_idle_job(tbox_group, testbox) + # when find no job, auto submit idle job at background + spawn { auto_submit_idle_job(tbox_group) } + + return nil end private def prepare_job(queue_name, testbox) @@ -571,4 +578,39 @@ class Sched return %({"job_id": "#{job_id}", "job_state": "complete"}) end + + private def query_consumable_keys(shortest_queue_name) + keys = [] of String + search = "sched/" + shortest_queue_name + "*" + response = @task_queue.query_keys(search) + + return keys unless response[0] == 200 + + key_list = JSON.parse(response[1].to_json).as_a + + # add consumable keys + key_list.each do |key| + queue_name = consumable_key?("#{key}") + keys << queue_name if queue_name + end + + return keys + end + + private def consumable_key?(key_name) + if key_name =~ /(.*)\/(.*)$/ + case $2 + when "in_process" + return nil + when "ready" + return $1 + when "idle" + return key_name + else + return key_name + end + end + + return nil + end end -- 2.23.0

4 6

[PATCH compass-ci] fix: container: avoid bind to all ip
by Tong Qunfeng 27 Oct '20

27 Oct '20

[why] default docker "-p port:port" will bind to all ips. - "-p 0.0.0.0:port:port" will also bind to all ips. - if exists ipv6 ip, it will bind to ipv6 address too. - if bind to ipv6 address, there will be tcp6 socket in TIME_WAIT no need bind to all ips. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- container/scheduler/start | 9 +++++++-- container/taskqueue/start | 9 ++++++++- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/container/scheduler/start b/container/scheduler/start index 305916e..d61616a 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -47,14 +47,19 @@ FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' FLUENTD_SERVER_PORT = defaults['FLUENTD_SERVER_PORT'] || '24224' docker_rm "s001-alpine-#{SCHED_PORT}" +bind_list = [] +[SCHED_HOST, '172.17.0.1'].uniq.each do |ip| + bind_list << '-p' + bind_list << "#{ip}:#{SCHED_PORT}:#{SCHED_PORT}" +end + cmd = %W[ docker run --restart=always --name s001-alpine-#{SCHED_PORT} -d -u 1090:1090 - -p #{SCHED_PORT}:#{SCHED_PORT} -] + env + %W[ +] + bind_list + env + %W[ -e CCI_REPOS=#{CCI_REPOS} -e LKP_SRC=#{DEFAULT_LKP} -e CCI_SRC=#{DEFAULT_CCI} diff --git a/container/taskqueue/start b/container/taskqueue/start index 4760b7d..bdd5e7c 100755 --- a/container/taskqueue/start +++ b/container/taskqueue/start @@ -16,19 +16,26 @@ defaults = relevant_defaults(names) env = docker_env(defaults) service_port = defaults['TASKQUEUE_PORT'] || '3060' +service_host = defaults['TASKQUEUE_HOST'] || '172.17.0.1' service_name = 'taskqueue' service_image = "#{service_name}-alpine:v0.001-#{service_port}" docker_rm "#{service_name}-#{service_port}" +bind_list = [] +[service_host, '172.17.0.1'].uniq.each do |ip| + bind_list << '-p' + bind_list << "#{ip}:#{service_port}:#{service_port}" +end + cmd = %W[ docker run --name #{service_name}-#{service_port} --restart=always -d -u 1090:1090 - -p #{service_port}:#{service_port} ] +cmd += bind_list cmd += env cmd += %W[ -v /etc/localtime:/etc/localtime:ro -- 2.23.0

1 0

对于compass-ci的两个问题请教
by 邵薛华 27 Oct '20

27 Oct '20

您好，这里有两个问题麻烦请教一下您1.compass-ci之后的开发计划如何，对于compass-ci以后的功能延伸有哪些。 2.compass-ci在现在的开发程度来说，是否只能针对open Euler-20.03进行测试，能否对open Euler-20.09，或者我想通过compass-ci是否可以对基于openEuler延伸开发出来的其他.iso进行一次测试。

1 0

[PATCH compass-ci 5/5] container/send-mail: delete old useless files
by Luan Shengde 27 Oct '20

27 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-mail/conf/sources.list | 9 ----- container/send-mail/mail-post.rb | 48 --------------------------- 2 files changed, 57 deletions(-) delete mode 100644 container/send-mail/conf/sources.list delete mode 100755 container/send-mail/mail-post.rb diff --git a/container/send-mail/conf/sources.list b/container/send-mail/conf/sources.list deleted file mode 100644 index 8179e01..0000000 --- a/container/send-mail/conf/sources.list +++ /dev/null @@ -1,9 +0,0 @@ -deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib -deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib -deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib -deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib - -deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib -deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib -deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib -deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib diff --git a/container/send-mail/mail-post.rb b/container/send-mail/mail-post.rb deleted file mode 100755 index 4eecf6e..0000000 --- a/container/send-mail/mail-post.rb +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env ruby -# SPDX-License-Identifier: MulanPSL-2.0+ -# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -# frozen_string_literal: true - -require 'sinatra' -require 'json' -require 'yaml' -require 'open3' -require_relative 'send-mail.rb' - -set :bind, '0.0.0.0' -set :port, 11311 - -post '/send_mail_yaml' do - data = YAML.safe_load request.body.read - raise TypeError, data, 'request data type error' unless data.class.eql? Hash - - mail_info = { - 'references' => data['references'] || '', - 'from' => data['from'] || 'team(a)crystal.ci', - 'subject' => data['subject'], - 'to' => data['to'], - 'body' => data['body'] - } - check_send_mail(mail_info) -end - -post '/send_mail_text' do - data = Mail.read_from_string(request.body.read) - - mail_info = { - 'references' => data.references || '', - 'from' => data.from || 'team(a)crystal.ci', - 'subject' => data.subject, - 'to' => data.to, - 'body' => data.body.decoded - } - check_send_mail(mail_info) -end - -def check_send_mail(mail_info) - raise TypeError, data, 'empty subject.' if mail_info['subject'].empty? - raise TypeError, data, 'empty email address.' if mail_info['to'].empty? - raise TypeError, data, 'empty email content.' if mail_info['body'].empty? - - send_mail(mail_info) -end -- 2.23.0

1 0

[PATCH compass-ci 1/5] container/send-mail: send-mail.rb
by Luan Shengde 27 Oct '20

27 Oct '20

setup smtp config for email server invoke mail-post to hand the send mail request Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-mail/send-mail.rb | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/container/send-mail/send-mail.rb b/container/send-mail/send-mail.rb index f459e78..436736c 100755 --- a/container/send-mail/send-mail.rb +++ b/container/send-mail/send-mail.rb @@ -4,25 +4,14 @@ # frozen_string_literal: true require 'mail' +require 'sinatra' +require "#{ENV['CCI_SRC']}/lib/mail-post" -mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp +mail_server = %x(/sbin/ip route |awk '/default/ {print $3}').chomp -# setup smtp config smtp = { address: mail_server, enable_starttls_auto: false } Mail.defaults { delivery_method :smtp, smtp } - -# send mail -def send_mail(mail_info) - mail = Mail.new do - references mail_info['references'] - from mail_info['from'] - subject mail_info['subject'] - to mail_info['to'] - body mail_info['body'] - end - mail.deliver! -end -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add cci-makepkg.yaml
by Zhang Yale 27 Oct '20

27 Oct '20

[Why] Now we run cci-makepkg to make packages. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- jobs/cci-makepkg.yaml | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 jobs/cci-makepkg.yaml diff --git a/jobs/cci-makepkg.yaml b/jobs/cci-makepkg.yaml new file mode 100644 index 00000000..5d437969 --- /dev/null +++ b/jobs/cci-makepkg.yaml @@ -0,0 +1,5 @@ +suite: cci-makepkg +category: functional + +cci-makepkg: + benchmark: -- 2.23.0

1 0

[PATCH v2 lkp-tests] jobs: add cci-depends.yaml
by Zhang Yale 27 Oct '20

27 Oct '20

[Why] Now we run cci-depends to make pkg depends. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- jobs/cci-depends.yaml | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 jobs/cci-depends.yaml diff --git a/jobs/cci-depends.yaml b/jobs/cci-depends.yaml new file mode 100644 index 00000000..0867068d --- /dev/null +++ b/jobs/cci-depends.yaml @@ -0,0 +1,5 @@ +suite: cci-depends +category: functional + +cci-depends: + benchmark: -- 2.23.0

1 0

[PATCH compass-ci] /lib/es_query.rb: add range query
by Zhang Yuhang 27 Oct '20

27 Oct '20

[why] Some queries need to limit the range of fields. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- lib/es_query.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index aaaadc2..3cf9852 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -42,6 +42,8 @@ def build_mutli_field_subquery_body(items) if value.is_a?(Array) inner_query = build_multi_field_or_query_body(key, value) query_fields.push({ bool: { should: inner_query } }) + elsif key.to_s == 'range' + query_fields.push({ range: value }) else query_fields.push({ term: { key => value } }) end -- 2.23.0

3 6

[PATCH v4 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 27 Oct '20

27 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..c5f29d74 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n "$block_size" ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH v4 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 27 Oct '20

27 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..1e2239e5 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -2,6 +2,13 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m + disk: 2HDD fs: -- 2.23.0

1 0

[PATCH v7 compass-ci 5/6] container/send-internet-mail: start
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/start | 39 ++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100755 container/send-internet-mail/start diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start new file mode 100755 index 0000000..483f01a --- /dev/null +++ b/container/send-internet-mail/start @@ -0,0 +1,39 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'send-internet-mail' + +names = Set.new %w[ + ROBOT_EMAIL_ADDRESS + ROBOT_EMAIL_PASSWORD + SEND_MAIL_PORT +] + +defaults = relevant_defaults(names) + +defaults['CCI_SRC'] = '/c/compass-ci' + +env = docker_env(defaults) + +cmd = %W[ + docker run + --restart=always + --name=send-internet-mail + -d +] + env + %W[ + -p #{defaults['SEND_MAIL_PORT']}:#{defaults['SEND_MAIL_PORT']} + -v #{ENV['CCI_SRC']}:#{defaults['CCI_SRC']} + -v /etc/localtime:/etc/localtime:ro + -w #{defaults['CCI_SRC']}/container/send-internet-mail + --log-driver json-file + send-internet-mail +] + +cmd += ['ruby', 'send-internet-mail.rb'] + +system(*cmd) -- 2.23.0

2 1

[PATCH compass-ci] sparrow: ignore os-nfs container
by Liu Yinsi 26 Oct '20

26 Oct '20

os-nfs service no need to deploy. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 26778a4..b3e5374 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -30,7 +30,8 @@ do_one() mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container - [ "$container_name" == 'sub-fluentd' ] && exit + [ "$container_name" == 'sub-fluentd' ] || \ + [ "$container_name" == 'os-nfs' ] && exit if [ -z "$start_only" ]; then [ -x build ] && ./build [ -x install ] && ./install -- 2.23.0

1 0

[PATCH compass-ci] container: docker run delete fluentd configration
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] --log-driver has been configured in the global Docker. By default, each docker uses fluentd to collect logs and does not need to be configured separately. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/extract-stats/start | 7 ------- container/scheduler-https/start | 6 ------ container/scheduler/start | 6 ------ container/web-backend-nginx/start | 5 ----- container/web-backend/start | 5 ----- 5 files changed, 29 deletions(-) diff --git a/container/extract-stats/start b/container/extract-stats/start index e5ee777..74cf917 100755 --- a/container/extract-stats/start +++ b/container/extract-stats/start @@ -12,8 +12,6 @@ names = Set.new %w[ REDIS_PORT ES_HOST ES_PORT - FLUENTD_HOST - FLUENTD_PORT MAIL_HOST MAIL_PORT LAB @@ -24,8 +22,6 @@ env = docker_env(defaults) DEFAULT_LKP = '/c/lkp-tests' DEFAULT_CCI = '/c/compass-ci' -FLUENTD_HOST = defaults['FLUENTD_HOST'] || '172.17.0.1' -FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' SRV_HTTP_HOST = defaults['SRV_HTTP_HOST'] || Open3.capture3('curl ip.sb')[0].chomp SRV_HTTP_PORT = defaults['SRV_HTTP_PORT'] || '11300' docker_rm "extract-stats" @@ -44,11 +40,8 @@ cmd = %w[ -v /etc/localtime:/etc/localtime:ro -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} -v /srv/result:/result - --log-driver=fluentd - --log-opt fluentd-address=#{FLUENTD_HOST}:#{FLUENTD_PORT} --log-opt mode=non-blocking --log-opt max-buffer-size=4m - --log-opt tag=extract-stats extract-stats ] diff --git a/container/scheduler-https/start b/container/scheduler-https/start index 6b06769..38c4892 100755 --- a/container/scheduler-https/start +++ b/container/scheduler-https/start @@ -19,8 +19,6 @@ names = Set.new %w[ OS_HTTP_PORT INITRD_HTTP_HOST INITRD_HTTP_PORT - FLUENTD_HOST - FLUENTD_PORT TASKQUEUE_PORT TASKQUEUE_HOST ] @@ -34,8 +32,6 @@ SCHED_PORT = defaults['SCHED_PORT'] || '3000' SCHED_HOST = defaults['SCHED_HOST'] || '172.17.0.1' SCHED_HTTPS_PORT = defaults['SCHED_HTTPS_PORT'] || 11306 SCHED_HTTPS_HOST = defaults['SCHED_HTTPS_HOST'] || 'api.compass-ci.openeuler.org' -FLUENTD_HOST = defaults['FLUENTD_HOST'] || '172.17.0.1' -FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' docker_rm "s001-alpine-#{SCHED_HTTPS_PORT}" @@ -59,8 +55,6 @@ cmd = %W[ -v /srv/initrd:/srv/initrd:ro -v /srv/os:/srv/os:ro -v /etc/compass-ci/ca:/ca - --log-driver=fluentd - --log-opt fluentd-address=#{FLUENTD_HOST}:#{FLUENTD_PORT} --log-opt mode=non-blocking --log-opt max-buffer-size=4m --log-opt tag=scheduler-#{SCHED_HTTPS_PORT} diff --git a/container/scheduler/start b/container/scheduler/start index 305916e..e0d70a8 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -17,8 +17,6 @@ names = Set.new %w[ OS_HTTP_PORT INITRD_HTTP_HOST INITRD_HTTP_PORT - FLUENTD_HOST - FLUENTD_PORT TASKQUEUE_PORT TASKQUEUE_HOST GIT_SERVER @@ -42,8 +40,6 @@ DEFAULT_CCI = "#{CCI_REPOS}/compass-ci" LAB = defaults['lab'] || 'z9' SCHED_PORT = defaults['SCHED_PORT'] || '3000' SCHED_HOST = defaults['SCHED_HOST'] || '172.17.0.1' -FLUENTD_HOST = defaults['FLUENTD_HOST'] || '172.17.0.1' -FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' FLUENTD_SERVER_PORT = defaults['FLUENTD_SERVER_PORT'] || '24224' docker_rm "s001-alpine-#{SCHED_PORT}" @@ -71,8 +67,6 @@ cmd = %W[ -v /srv/result:/srv/result -v /srv/initrd:/srv/initrd:ro -v /srv/os:/srv/os:ro - --log-driver=fluentd - --log-opt fluentd-address=#{FLUENTD_HOST}:#{FLUENTD_PORT} --log-opt mode=non-blocking --log-opt max-buffer-size=4m --log-opt tag=scheduler-#{SCHED_PORT} diff --git a/container/web-backend-nginx/start b/container/web-backend-nginx/start index a28101d..4da4c8b 100755 --- a/container/web-backend-nginx/start +++ b/container/web-backend-nginx/start @@ -3,8 +3,6 @@ [[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci -[[ $FLUENTD_HOST ]] || FLUENTD_HOST=localhost -[[ $FLUENTD_PORT ]] || FLUENTD_PORT=24224 . $CCI_SRC/container/defconfig.sh docker_rm web-backend-nginx @@ -17,11 +15,8 @@ cmd=( -v /etc/ssl/certs:/opt/cert - --log-driver=fluentd - --log-opt fluentd-address=$FLUENTD_HOST:$FLUENTD_PORT --log-opt mode=non-blocking --log-opt max-buffer-size=4m - --log-opt tag=web-backend-nginx web-backend-nginx ) diff --git a/container/web-backend/start b/container/web-backend/start index 5fb493c..941fb11 100755 --- a/container/web-backend/start +++ b/container/web-backend/start @@ -5,8 +5,6 @@ [[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci [[ $LKP_SRC ]] || LKP_SRC=/c/lkp-tests [[ $UPSTREAM_REPOS_PATH ]] || UPSTREAM_REPOS_PATH=/c/upstream-repos -[[ $FLUENTD_HOST ]] || FLUENTD_HOST=localhost -[[ $FLUENTD_PORT ]] || FLUENTD_PORT=24224 . $CCI_SRC/container/defconfig.sh docker_rm web-backend @@ -27,11 +25,8 @@ cmd=( -v $LKP_SRC:/c/lkp-tests -v $UPSTREAM_REPOS_PATH:/c/upstream-repos - --log-driver=fluentd - --log-opt fluentd-address=$FLUENTD_HOST:$FLUENTD_PORT --log-opt mode=non-blocking --log-opt max-buffer-size=4m - --log-opt tag=web-backend debian:web-backend ) -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow: fix containers auto resume failed after reboot
by Liu Yinsi 26 Oct '20

26 Oct '20

call 4-docker/buildall to fix container auto resume depends issues. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 26 +++++++++++++++++--------- sparrow/7-systemd/cci-network | 9 +++++---- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..26778a4 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -3,6 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. CONTAINER_PATH="$CCI_SRC/container" +export start_only=$1 build_depends() { @@ -30,20 +31,27 @@ do_one() ( cd $container [ "$container_name" == 'sub-fluentd' ] && exit - [ -x build ] && ./build - [ -x install ] && ./install + if [ -z "$start_only" ]; then + [ -x build ] && ./build + [ -x install ] && ./install + fi [ -x first-run ] && ./first-run [ -x start ] && ./start ) lockfile-remove --lock-name "$container_name".lock } -tmpdir=$(mktemp -d) +buildall() +{ + tmpdir=$(mktemp -d) -for dir in $CONTAINER_PATH/*/ -do - build_depends $dir -done + for dir in $CONTAINER_PATH/*/ + do + build_depends $dir + done + + wait + rm -fr $tmpdir +} -wait -rm -fr $tmpdir +buildall diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 4aadcdc..0d21a2d 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -4,13 +4,14 @@ #this file used in our own systemd unit file: cci-network.service. #contents as follows will be automatically executed after system reboot. -[[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci +. $CCI_SRC/container/defconfig.sh + +# export ENV +. /etc/profile.d/compass.sh $CCI_SRC/sparrow/2-network/br0 $CCI_SRC/sparrow/2-network/iptables $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs -# --restart=always option is not absolutely reliable -containers=($(docker ps -a |grep -v NAMES |awk '{print $NF}')) -[ -n "$containers" ] && docker start "${containers[@]}" >/dev/null 2>&1 +$CCI_SRC/sparrow/4-docker/buildall start_only -- 2.23.0

1 0

[PATCH compass-ci] sparrow: fix containers auto resume failed after reboot
by Liu Yinsi 26 Oct '20

26 Oct '20

call 4-docker/buildall to fix container auto resume depends issues. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 26 +++++++++++++++++--------- sparrow/7-systemd/cci-network | 9 +++++---- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..26778a4 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -3,6 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. CONTAINER_PATH="$CCI_SRC/container" +export start_only=$1 build_depends() { @@ -30,20 +31,27 @@ do_one() ( cd $container [ "$container_name" == 'sub-fluentd' ] && exit - [ -x build ] && ./build - [ -x install ] && ./install + if [ -z "$start_only" ]; then + [ -x build ] && ./build + [ -x install ] && ./install + fi [ -x first-run ] && ./first-run [ -x start ] && ./start ) lockfile-remove --lock-name "$container_name".lock } -tmpdir=$(mktemp -d) +buildall() +{ + tmpdir=$(mktemp -d) -for dir in $CONTAINER_PATH/*/ -do - build_depends $dir -done + for dir in $CONTAINER_PATH/*/ + do + build_depends $dir + done + + wait + rm -fr $tmpdir +} -wait -rm -fr $tmpdir +buildall diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 4aadcdc..0d21a2d 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -4,13 +4,14 @@ #this file used in our own systemd unit file: cci-network.service. #contents as follows will be automatically executed after system reboot. -[[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci +. $CCI_SRC/container/defconfig.sh + +# export ENV +. /etc/profile.d/compass.sh $CCI_SRC/sparrow/2-network/br0 $CCI_SRC/sparrow/2-network/iptables $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs -# --restart=always option is not absolutely reliable -containers=($(docker ps -a |grep -v NAMES |awk '{print $NF}')) -[ -n "$containers" ] && docker start "${containers[@]}" >/dev/null 2>&1 +$CCI_SRC/sparrow/4-docker/buildall start_only -- 2.23.0

1 0

[PATCH lkp-tests] lib/monitor.rb: add stop reason
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] When the client disconnects from the ws, the reason is not printed. Like this: connect to ws://localhost:11310/filter ............. connection closed: Add stop reason, and it will be like: connect to ws://localhost:11310/filter .......... connection closed: timeout or connection closed: normal Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 9ce1d475..c20794e8 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -27,6 +27,7 @@ class Monitor load_default @result = [] @stop_query = {} + @reason = nil end def load_default @@ -87,16 +88,12 @@ class Monitor ssh root@#{ssh_host} -p #{ssh_port} -o StrictHostKeyChecking=no" end - def stop_em(web_socket) - web_socket.close - EM.stop - end - - def stop(data, web_socket) + def stop(data, web_socket, code = 1000, reason = 'normal') @stop_query.each do |key, value| return false unless data[key] == value end - stop_em(web_socket) + @reason = reason + web_socket.close(code, reason) end def run(timeout = nil) @@ -111,7 +108,8 @@ class Monitor if timeout && timeout != 0 EM.add_timer(timeout) do @exit_status_code = 1 - stop_em(ws) + @reason = 'timeout' + ws.close(1000, @reason) end end @@ -131,7 +129,9 @@ class Monitor end ws.on :close do |event| - puts "connection closed: #{event.reason}" + reason = event.reason || @reason + puts "connection closed: #{reason}" + EM.stop end end return @exit_status_code -- 2.23.0

2 1

[PATCH lkp-tests 1/2] fix(distro/depends): delete unnecessary depends
by Wang Yong 26 Oct '20

26 Oct '20

[Why] ruby-full contains ruby-dev $ apt depends ruby-full ruby-full Depends: ruby Depends: ruby-dev Depends: ri [How] delete this package from depend file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/cci-deploy | 1 - 1 file changed, 1 deletion(-) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index dac682505..c797c1eea 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -4,7 +4,6 @@ g++ make docker.io ruby-full -ruby-dev bridge-utils qemu dmidecode -- 2.23.0

2 2

[PATCH compass-ci] 0-package/common: change the value of fluentd port
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] Prior to this, fluentd used port 24224 to collection docker logs. But now, fluentd change to sub-fluentd, and uses 24225. master-fluentd uses port 24224 to get docker logs from sub-fluentd. So docker should forward logs to sub-fluentd and uses 24225 port. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/0-package/common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 31588f4..523438c 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -31,7 +31,7 @@ cat > /etc/docker/daemon.json <<EOF { "log-driver": "fluentd", "log-opts":{ - "fluentd-address": "localhost:24224", + "fluentd-address": "localhost:24225", "fluentd-async-connect": "true", "tag": "{{.Name}}" }, -- 2.23.0

1 0

[PATCH lkp-tests] fix(adaptation/*): update adaptation for several OS
by Wang Yong 26 Oct '20

26 Oct '20

[Why] some packages in other OS have different name or some Debian packages no need add mapping in red-hat related OS. Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/centos | 10 ++++++++-- distro/adaptation/fedora | 6 +++++- distro/adaptation/kylin | 10 ++++++++++ distro/adaptation/openeuler | 9 +++++++++ distro/adaptation/oracle | 4 +++- 5 files changed, 35 insertions(+), 4 deletions(-) diff --git a/distro/adaptation/centos b/distro/adaptation/centos index f92cee2ff..e7fabf36d 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -8,7 +8,6 @@ build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup clang-6.0: clang cmake: cmake cmake3 -dbench=4.0-2: dbench dnsutils: bind-utils dpkg-dev: dpkg-devel e2fslibs-dev: e2fsprogs-devel @@ -297,7 +296,6 @@ libtasn1-6: libtasn1 libtimedate-perl: perl-TimeDate libtirpc-common: libtirpc libtirpc3: libtirpc -network-manager: network-manager-applet libv4l-dev: libv4l-devel libvirt-clients: libvirt-client libvirt-daemon-system: libvirt-daemon @@ -328,3 +326,11 @@ dh-python: dh-strip-nondeterminism: dh-systemd: guile-2.0: guile +libpython2.7: python-libs +network-manager: NetworkManager +init-system-helpers: +javahelper: +junit4: junit +liboath-dev: liboath-devel +lsb-release: redhat-lsb +linux-base: diff --git a/distro/adaptation/fedora b/distro/adaptation/fedora index dd274d0f1..38b7e9dc4 100644 --- a/distro/adaptation/fedora +++ b/distro/adaptation/fedora @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel freeglut3: freeglut fsmark: fs_mark @@ -158,3 +157,8 @@ libslang2-dev: slang-devel python-dev: python3-devel systemtap-sdt-dev: systemtap-sdt-devel initramfs-tools: +init-system-helpers: +network-manager: NetworkManager +javahelper: +junit4: junit +linux-base: diff --git a/distro/adaptation/kylin b/distro/adaptation/kylin index c60a7f3ee..f143fde96 100644 --- a/distro/adaptation/kylin +++ b/distro/adaptation/kylin @@ -137,3 +137,13 @@ uuid-runtime: uuid xfslibs-dev: xfsprogs-devel xutils-dev: imake xorg-x11-util-macros zlib1g-dev: zlib-devel +libpython2.7: +network-manager: NetworkManager +wpasupplicant: wpa_supplicant +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel +linux-base: diff --git a/distro/adaptation/openeuler b/distro/adaptation/openeuler index ca8688b5a..e72f34fd2 100644 --- a/distro/adaptation/openeuler +++ b/distro/adaptation/openeuler @@ -284,3 +284,12 @@ guile-2.0: guile libmysqlclient-dev: mysql-devel libipc-run-perl: libklibc-dev: +libpython2.7: +network-manager: NetworkManager +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel rubygem-rdoc +linux-base: diff --git a/distro/adaptation/oracle b/distro/adaptation/oracle index f7fcfaa84..af8415ec3 100644 --- a/distro/adaptation/oracle +++ b/distro/adaptation/oracle @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-uek-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel f2fs-tools: freeglut3: freeglut @@ -133,3 +132,6 @@ xutils-dev: imake xz-utils: xz zlib1g-dev: zlib-devel initramfs-tools: +init-system-helpers: +javahelper: +linux-base: -- 2.23.0

1 0

[PATCH lkp-tests 2/2] fix(distro/depends): move i386 depends to x86_64
by Wang Yong 26 Oct '20

26 Oct '20

[Why] i386 depends are not suitable for aarch64 [How] move these packages to x86_64 file for further support: commit 5b5f5d2b2e1948555dd01c12dc31e77442a542b8 depends: auto load $program.$(arch) Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/kernel-selftests | 22 ---------------------- distro/depends/kernel-selftests.x86_64 | 23 +++++++++++++++++++++++ 2 files changed, 23 insertions(+), 22 deletions(-) create mode 100644 distro/depends/kernel-selftests.x86_64 diff --git a/distro/depends/kernel-selftests b/distro/depends/kernel-selftests index 3d08c85da..b02e37e22 100644 --- a/distro/depends/kernel-selftests +++ b/distro/depends/kernel-selftests @@ -2,21 +2,8 @@ gcc-8 gcc-8-multilib libc6-dev libc-bin -libc6-dev-i386 (x86_64) -libc6-dev-x32 libc6/unstable -lib32gcc-8-dev -libx32gcc-8-dev -libc6-i386 libc6-dev -libc6-x32 -libx32gcc1 -libx32gomp1 -libx32itm1 -libx32atomic1 -libx32asan5 -libx32ubsan1 -libx32quadmath0 rsync make libcap-ng-dev @@ -55,12 +42,3 @@ sendip ipvsadm libmnl-dev libkeyutils-dev -lib64gcc1 (i386) -lib64gomp1 (i386) -lib64itm1 (i386) -lib64atomic1 (i386) -lib64asan3 (i386) -libc6-amd64/unstable (i386) -libc6-dev-amd64/unstable (i386) -lib64gcc-8-dev (i386) -libx32gcc-8-dev (i386) diff --git a/distro/depends/kernel-selftests.x86_64 b/distro/depends/kernel-selftests.x86_64 new file mode 100644 index 000000000..9491ac3b5 --- /dev/null +++ b/distro/depends/kernel-selftests.x86_64 @@ -0,0 +1,23 @@ +libc6-dev-i386 (x86_64) +libc6-dev-x32 +libc6-i386 +libc6-x32 +libx32gcc1 +libx32gomp1 +libx32itm1 +libx32atomic1 +libx32asan5 +libx32ubsan1 +libx32quadmath0 +lib64gcc1 (i386) +lib64gomp1 (i386) +lib64itm1 (i386) +lib64atomic1 (i386) +lib64asan3 (i386) +libc6-amd64/unstable (i386) +libc6-dev-amd64/unstable (i386) +lib64gcc-8-dev (i386) +libx32gcc-8-dev (i386) +lib32gcc-8-dev +libx32gcc-8-dev + -- 2.23.0

1 0

[PATCH v7 compass-ci 6/6] container/send-internet-mail: send-mail.md
by Luan Shengde 26 Oct '20

26 Oct '20

documentation for send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/send-mail.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/send-mail.md diff --git a/doc/send-mail.md b/doc/send-mail.md new file mode 100644 index 0000000..63716d7 --- /dev/null +++ b/doc/send-mail.md @@ -0,0 +1,58 @@ + +# send mail + +## purpose + +The service is used to send mails with formatted mail data +- send intranet mail +- send internet mail + +## send mail port +- send intranet mail: 11311 +- send internet mail: 49000 + +## send mail host +- host the service running on + +## data format +- send_mail_yaml + - data format + data='{ + "subject": "email subject", + "to": "email_to_addr", + "body": "email message" + }' + or + data=" + subject: email subject + to: email_to_addr + body: email message + " + +- send_mail_text + - data format + data=" + To: email_to_addr + Subject: email_subject + + mail_msg_line1 + mail_msg_line2 + ... + " + +## usage: +- send_mail_yaml + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_yaml' -d "$data" +- send_mail_text + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "$data" + +## example + + data = " + To: test_email(a)163.com + Subject: test mail 10 + + test msg 1010 + " + + %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") -- 2.23.0

1 0

[PATCH v7 compass-ci 4/6] container/send-internet-mail: build
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

1 0

[PATCH v7 compass-ci 3/6] container/send-internet-mail: Dockerfile
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..d151bda --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v7 compass-ci 2/6] container/send-internet-mail: send-internet-mail.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp config for internet email server invoke mail-post to hand the send mail request Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- .../send-internet-mail/send-internet-mail.rb | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 container/send-internet-mail/send-internet-mail.rb diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb new file mode 100644 index 0000000..3b1d159 --- /dev/null +++ b/container/send-internet-mail/send-internet-mail.rb @@ -0,0 +1,21 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

1 0

[PATCH v7 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..5832bfb --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,55 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

1 0

[PATCH v2 compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 26 Oct '20

26 Oct '20

[Why] Two things here i do: 1) alter 'node_macs' to 'direct_macs'; 2) add one field to jobs, 'direct_ips'. Because lkp use those 2 fields to bind 'direct_ips' to 'direct_macs', after this, nodes can communicate with each other when running cluster job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 692180f..c32091f 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,16 @@ class Sched # collect all job ids job_ids = [] of String + net_id = "192.168.222" + ip0 = cluster_config["ip0"]? + if ip0 + ip0 = ip0.as_i + else + ip0 = 1 + end + # steps for each host - cluster_config.each do |host, config| + cluster_config["nodes"].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +227,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 26 Oct '20

26 Oct '20

[WHY] the path of *.cgz file generated by build-pkg script cannot be obtained dring test. So the path of cgz need to be changed. [HOW] below is example for openeuler: /srv/initrd/build-pkg/ => /srv/initrd/build-pkg/initramfs/openeuler/aarch64/20.03/ Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..eca09e64 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,6 +2,7 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url @@ -21,10 +22,14 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" @@ -75,7 +80,7 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 } -- 2.23.0

3 5

[PATCH v6 compass-ci 6/7] container/send-internet-mail: start
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/start | 42 ++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 container/send-internet-mail/start diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start new file mode 100755 index 0000000..6ea55d6 --- /dev/null +++ b/container/send-internet-mail/start @@ -0,0 +1,42 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'send-internet-mail' + +names = Set.new %w[ + ROBOT_EMAIL_ADDRESS_INTERNET + ROBOT_EMAIL_PASSWORD_INTERNET + SEND_MAIL_PORT_INTERNET +] + +defaults = relevant_defaults(names) + +ROBOT_EMAIL_ADDRESS = defaults['ROBOT_EMAIL_ADDRESS_INTERNET'] +ROBOT_EMAIL_PASSWORD = defaults['ROBOT_EMAIL_PASSWORD_INTERNET'] +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] + +cmd = %W[ + docker run + --restart=always + --name=send-internet-mail + -d + -e ROBOT_EMAIL_ADDRESS=#{ROBOT_EMAIL_ADDRESS} + -e ROBOT_EMAIL_PASSWORD=#{ROBOT_EMAIL_PASSWORD} + -e CCI_SRC=/c/compass-ci + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -p #{SEND_MAIL_PORT}:#{SEND_MAIL_PORT} + -v #{ENV['CCI_SRC']}:/c/compass-ci + -v /etc/localtime:/etc/localtime:ro + -w /c/compass-ci + --log-driver json-file + send-internet-mail +] + +cmd += ['ruby', "container/send-internet-mail/run.rb"] + +system(*cmd) -- 2.23.0

2 2

[PATCH v6 compass-ci 3/7] container/send-internet-mail: run.rb
by Luan Shengde 26 Oct '20

26 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 2

[PATCH v6 compass-ci 2/7] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

2 2

[PATCH v6 compass-ci 1/7] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..9fcf1fe --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,64 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = %x(/sbin/ip route | awk '/default/ {print $3}').chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

2 4

[PATCH lkp-tests] fio: fio test arg add iosched
by Wei Jihui 26 Oct '20

26 Oct '20

set iosched=my-deadline by test require Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- jobs/fio-basic-openeuler.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/jobs/fio-basic-openeuler.yaml b/jobs/fio-basic-openeuler.yaml index d2cc277b..55c98f97 100644 --- a/jobs/fio-basic-openeuler.yaml +++ b/jobs/fio-basic-openeuler.yaml @@ -7,6 +7,8 @@ disk: - 1HDD fs: - ext4 +iosched: +- mq-deadline nr_task: 24 fio-setup-basic: @@ -26,7 +28,6 @@ fio-setup-basic: bs: - 4k - 16k - - 32k - 64k - 128k - 256k -- 2.23.0

2 1

[PATCH v3 lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 26 Oct '20

26 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..dc4d157d 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -138,7 +138,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/"${os_arch}"-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +177,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/"${os_arch}"-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

1 0

git count 2020-10-23
by Wu Fengguang 26 Oct '20

26 Oct '20

Jihui is code star of the week, thanks Jihui! wfg@crystal ~/todo% bin/git-count 13 Wu Fengguang 12 Wei Jihui 10 Wu Zhende 6 Xiao Shenwei 6 Li Ping 5 Cao Xueliang 5 Li Yuanchao 4 Zhang Yu 4 Yu Chuan 4 Ren Wen 3 Bai Jing 3 Xu Xijian 3 Hu Xuejiao 3 Cui Yili 2 Zhang Yale 2 Wang Chenglong 1 Lu Weitao 1 Luan Shengde 1 Zhang Yuhang 1 Xie Zheng 0 chief 0 Du Kaitian 0 Liu Shaofei 0 Liu Yinsi 0 Ren Zhiwen 0 Sun Lijun 0 Sun Yukui 0 Shi Zhichao 0 Wang Yong 0 Lin Jiaxin 0 Liang Tao 0 Zhang Dewan 0 Lu Kaiyi 0 Zhou Shengrui

2 1

help star 2020-10-23
by Wu Fengguang 24 Oct '20

24 Oct '20

Xueliang is our help star, thanks to kind and helpful Xueliang! wfg@crystal ~/todo% bin/help-star people/*/credit/2020-10-23.md 25.9 Wu Fengguang 20.6 Cao Xueliang 17.0 Wei Jihui 12.8 Yu Chuan 10.1 Wu Zhende 9.8 Cui Yili 9.6 Liu Shaofei 9.5 Liu Yinsi 9.4 Bai Jing 8.5 Xu Xijian 8.3 Lu Weitao 8.1 Xiao Shenwei 7.8 Luan Shengde 7.7 Wang Yong 7.2 Zhang Yu 6.4 Zhang Yale 5.6 Wang Chenglong 5.5 Tong Qunfeng 4.8 Lin Jiaxin 4.5 Zhang Yuhang 4.5 Zhang Dewan 4.3 Zhou Shengrui 4.1 Li Yuanchao 4.1 Hu Xuejiao 3.1 Li Ping 2.9 Sun Yukui 2.8 Ren Wen 2.0 Du Kaitian 2.0 Zhang Meng 0.7 Xie Zheng 0.5 Shi Zhichao 0.0 Liang Tao 0.0 Lu Kaiyi

1 0

review star 2020-10-23
by Wu Fengguang 24 Oct '20

24 Oct '20

Jihui is our review star, thanks Jihui! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 211 107 264 63 Wu Fengguang 32 16 42 2 Xiao Shenwei 53 23 25 7 Wei Jihui 27 5 23 1 Yu Chuan 39 17 20 3 Wu Zhende 32 7 18 0 Zhang Yuhang 12 8 17 4 Lu Weitao 30 18 17 8 Cao Xueliang 30 10 16 2 Li Yuanchao 40 11 15 0 Wang Chenglong 8 8 14 2 Sun Yukui 33 5 14 9 Tong Qunfeng 13 12 12 3 Liu Shaofei 28 10 12 1 Xu Xijian 19 2 10 0 Liu Yinsi 4 4 10 5 Zhou Shengrui 22 6 10 0 Zhang Yu 68 3 7 3 Luan Shengde 4 4 6 1 Wang Yong 21 4 4 1 Zhang Yale 17 2 4 1 Li Ping 7 2 4 0 Xie Zheng 3 3 3 1 Lin Jiaxin 11 1 1 1 Cui Yili 27 1 1 0 Ren Wen 16 3 1 0 Lu Kaiyi 7 0 0 0 Hu Xuejiao 12 6 0 1 Bai Jing

1 0

[PATCH v6 compass-ci 7/7] container/send-internet-mail: send-mail.md
by Luan Shengde 23 Oct '20

23 Oct '20

documentation for send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/send-mail.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/send-mail.md diff --git a/doc/send-mail.md b/doc/send-mail.md new file mode 100644 index 0000000..86cd889 --- /dev/null +++ b/doc/send-mail.md @@ -0,0 +1,58 @@ + +# send mail + +## purpose + +The send mail is used to send mails with formatted mail data +- send intranet mail +- send internet mail + +## send mail port +- send intranet: 11311 +- send internet: 49000 + +## send mail host +- the host service running on + +## data format +- send_mail_yaml + - data format + data='{ + "subject": "email subject", + "to": "email_to_addr", + "body": "email message" + }' + or + data=" + subject: email subject + to: email_to_addr + body: email message + " + +- send_mail_text + - data format + data=" + To: email_to_addr + Subject: email_subject + + mail_msg_line1 + mail_msg_line2 + ... + " + +## usage: +- send_mail_yaml + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_yaml' -d "$data" +- send_mail_text + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "$data" + +## example + + data = " + To: test_email(a)163.com + Subject: test mail 10 + + test msg 1010 + " + + %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") -- 2.23.0

1 0

[PATCH v6 compass-ci 5/7] container/send-internet-mail: build
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

1 0

[PATCH v6 compass-ci 4/7] container/send-internet-mail: Dockerfile
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..d151bda --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v1 compass-ci] add queue field into job.yaml
by Xiao Shenwei 23 Oct '20

23 Oct '20

[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/job.cr | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..43417ea 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end private def append_init_field @@ -199,6 +200,23 @@ class Job self["result_service"] = "raw_upload" end + private def set_queue + if @hash.has_key?("queue") + queue_prefix = @hash["queue"].to_s.split(".")[0] + if (queue_prefix != tbox_group.to_s) + raise "an incorrect queue-prefix is specified: #{queue_prefix}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0

1 0

[PATCH lkp-tests] jobs: fix makepkg-deps.yaml
by Zhang Yale 23 Oct '20

23 Oct '20

[Why] Now we run cci-depends to make pkg depends. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- jobs/makepkg-deps.yaml | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/jobs/makepkg-deps.yaml b/jobs/makepkg-deps.yaml index cad8466a..3e8bc113 100644 --- a/jobs/makepkg-deps.yaml +++ b/jobs/makepkg-deps.yaml @@ -1,11 +1,5 @@ -suite: makepkg-deps -testcase: makepkg-deps +suite: cci-depends +category: functional -testbox: lkp-hsw-4ex1 -branch: linus/master -commit: RELEASE - -rootfs: -- debian-x86_64.cgz - -makepkg-deps: +cci-depends: + benchmark: -- 2.23.0

1 0

[PATCH v5 compass-ci 6/6] container/send-internet-mail: start
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/start | 74 ++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100755 container/send-internet-mail/start diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start new file mode 100755 index 0000000..1ce17bf --- /dev/null +++ b/container/send-internet-mail/start @@ -0,0 +1,74 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'send-internet-mail' + +names = Set.new %w[ + ROBOT_EMAIL_ADDRESS_INTERNET + ROBOT_EMAIL_PASSWORD_INTERNET + SEND_MAIL_PORT_INTERNET +] + +defaults = relevant_defaults(names) + +ROBOT_EMAIL_ADDRESS = defaults['ROBOT_EMAIL_ADDRESS_INTERNET'] +ROBOT_EMAIL_PASSWORD = defaults['ROBOT_EMAIL_PASSWORD_INTERNET'] +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] + +cmd = %W[ + docker run + --restart=always + --name=send-internet-mail + -d + -e ROBOT_EMAIL_ADDRESS=#{ROBOT_EMAIL_ADDRESS} + -e ROBOT_EMAIL_PASSWORD=#{ROBOT_EMAIL_PASSWORD} + -e CCI_SRC=/c/compass-ci + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -p #{SEND_MAIL_PORT}:#{SEND_MAIL_PORT} + -v #{ENV['CCI_SRC']}:/c/compass-ci + -v /etc/localtime:/etc/localtime:ro + -w /c/compass-ci + --log-driver json-file + send-internet-mail +] +cmd += ['ruby', "container/send-internet-mail/run.rb"] + +system(*cmd) + +puts <<EOF +port: +--- +data='{ +"subject": "email subject", +"to": "email_to_addr", +"body": "email message" +}' + +or + +data=" +subject: email subject +to: email_to_addr +body: email message" + +Usage: + curl -XPOST 'send-mail-server:49000/send_mail_yaml' -d "$data" + +--- +data=" +To: email_to_addr +Subject: email_subject + +mail_msg_line1 +mail_msg_line2 +... +" + +Usage: + curl -XPOST 'send-mail-server:49000/send_mail_text' -d "$data" +EOF -- 2.23.0

4 6

[PATCH v3 lkp-tests] tests: add get_cpu function
by Zhang Yu 23 Oct '20

23 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..efa38075 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,20 +1,23 @@ #!/bin/bash -get_node() +get_mem() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + echo $1: $(lscpu | awk -F': *' '/'$2':/{print $2}') } +get_cpu arch 'Architecture' +get_cpu nr_node 'NUMA\snode$s$' +get_cpu nr_cpu '^CPU$s$' +get_cpu model_name 'Model\sname' + get_nr_disk() { nr_hdd_disks=$(lsblk -d -o name,ROTA | grep -v loop | awk '{print $2}' | grep 1 | wc -l) @@ -101,11 +104,10 @@ get_ipmi_ip() main() { - get_node + get_mem get_nr_disk get_partitions get_mac - get_model_name get_ipmi_ip } -- 2.23.0

2 2

[PATCH v2 compass-ci] container/debian: solve build error
by Cui Yili 23 Oct '20

23 Oct '20

[why] Executing "container/debian/build", error when apt-get installs the package: E: Could not configure 'libc6:arm64'. E: Could not perform immediate configuration on 'libnss-nis:arm64'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2) [how] Increase the parameter "-o APT::Immediate-Configure=false" when apt-get installs the package. reference: https://www.debian.org/releases/wheezy/armhf/release-notes/ch-upgrading.en.… Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/debian/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/debian/Dockerfile b/container/debian/Dockerfile index 1d310da..21b6a22 100644 --- a/container/debian/Dockerfile +++ b/container/debian/Dockerfile @@ -12,6 +12,6 @@ COPY root / RUN sed -i "s/^exit [0-9]*/exit 0/" /usr/sbin/policy-rc.d RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ + apt-get install -o APT::Immediate-Configure=false -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ mkdir -p /run/sshd -- 2.23.0

1 0

[PATCH v4 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..c5f29d74 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n "$block_size" ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH v4 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..1e2239e5 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -2,6 +2,13 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m + disk: 2HDD fs: -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: adapt to x86_64 architectures
by Liu Yinsi 23 Oct '20

23 Oct '20

lockfile-create download link compatible with x86_64 and aarch64 architectures. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index bd1f728..da13831 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -24,7 +24,8 @@ pkgs=( yum install -y "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(arch) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

2 1

[PATCH v5 compass-ci 5/6] container/send-internet-mail: build
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

2 1

[PATCH v2 lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 23 Oct '20

23 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..424615be 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -3,6 +3,7 @@ # - test_memory_size # - mode # - test +# - os_arch ## lmbench is a suite of simple, portable, ANSI/C microbenchmarks for ## UNIX/POSIX. In general, it measures two key features: latency and @@ -138,7 +139,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/$os_arch-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +178,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/$os_arch-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

2 1

[PATCH compass-ci] container/debian: solve build error
by Cui Yili 23 Oct '20

23 Oct '20

Executing "container/debian/build", error when apt-get installs the package: E: Could not configure 'libc6:arm64'. E: Could not perform immediate configuration on 'libnss-nis:arm64'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2) Increase the parameter "-o APT::Immediate-Configure=false" when apt-get installs the package. reference: https://www.debian.org/releases/wheezy/armhf/release-notes/ch-upgrading.en.… Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/debian/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/debian/Dockerfile b/container/debian/Dockerfile index 1d310da..21b6a22 100644 --- a/container/debian/Dockerfile +++ b/container/debian/Dockerfile @@ -12,6 +12,6 @@ COPY root / RUN sed -i "s/^exit [0-9]*/exit 0/" /usr/sbin/policy-rc.d RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ + apt-get install -o APT::Immediate-Configure=false -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ mkdir -p /run/sshd -- 2.23.0

2 1

[PATCH v3 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexpected problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

2 1

[PATCH v3 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

2 1

[PATCH compass-ci] sparrow/0-package/os: skip when intall failed
by Xu Xijian 23 Oct '20

23 Oct '20

When install several packages, if one of them is installed failed, should go ahead to install next packages. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- sparrow/0-package/os/centos | 2 +- sparrow/0-package/os/debian | 2 +- sparrow/0-package/os/openEuler | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..4243f99 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -22,4 +22,4 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..16ef0a9 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -20,4 +20,4 @@ pkgs=( busybox ) -apt-get install -y "${pkgs[@]}" +apt-get install -y --fix-missing "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..ba298c8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -21,7 +21,7 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -- 2.23.0

2 1

[PATCH v3 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..992d4a14 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n $block_size ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

Re: [PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Zhang Yu 23 Oct '20

23 Oct '20

> log_echo "$cmd &" > $cmd > $TMP/iozone-$i & > i=$((i+1)) >+ iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest You should add a judge, if $block_size isn't in yaml file, this maybe error. Thanks, Zhangyu

1 0

[PATCH v5 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 23 Oct '20

23 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 68 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..e62640a --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,68 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.class.eql? Hash + + mail_info = { + 'references' => data['references'], + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'references' => data.references, + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + check_send_mail(mail_info) +end + +def check_send_mail(mail_info) + mail_info['from'] = ENV['ROBOT_EMAIL_ADDRESS'] + + raise 'no subject.' if mail_info['subject'].nil? + raise 'no email address.' if mail_info['to'].nil? + raise 'no email content.' if mail_info['body'].nil? + + send_mail(mail_info) +end + +def send_mail(mail_info) + mail = Mail.new do + references mail_info['references'] + from mail_info['from'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

3 6

[PATCH v2 lkp-tests] tests: add "arch" field with physical machine
by Zhang Yu 23 Oct '20

23 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..c0e17f26 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,18 +1,20 @@ #!/bin/bash -get_node() +get_mem() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + lscpu=$(lscpu) + echo "nr_node: $(echo "$lscpu" | grep "NUMA node(s)" | awk '{print $3}')" + echo "nr_cpu: $(echo "$lscpu" | grep "^CPU(s)" | awk '{print $2}')" + echo "arch: $(echo "$lscpu" | grep "Architecture" | awk '{print $2}')" + echo "model_name: $(echo "$lscpu" | grep "Model name" | awk '{print $3}')" } get_nr_disk() @@ -101,11 +103,11 @@ get_ipmi_ip() main() { - get_node + get_cpu + get_mem get_nr_disk get_partitions get_mac - get_model_name get_ipmi_ip } -- 2.23.0

5 4

[PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/iozone b/tests/iozone index 3095ce80..0a813621 100755 --- a/tests/iozone +++ b/tests/iozone @@ -11,6 +11,7 @@ do log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) + iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest done log_eval wait -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

1 0

[PATCH v5 compass-ci 2/6] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 23 Oct '20

23 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

3 6

[PATCH lkp-tests] chore: install iproute2
by Liu Yinsi 23 Oct '20

23 Oct '20

command ip used in sparrow/2-network/br0. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/cci-deploy | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index dac68250..01085f59 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -9,3 +9,4 @@ bridge-utils qemu dmidecode lockfile-progs +iproute2 -- 2.23.0

3 5

[PATCH compass-ci] sparrow/0-package: adapt to x86_64 architectures
by Liu Yinsi 23 Oct '20

23 Oct '20

lockfile-create download link compatible with x86_64 and aarch64 architectures. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index bd1f728..e408d77 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -24,7 +24,8 @@ pkgs=( yum install -y "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(uname -p) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

3 3

[PATCH v2 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexcepted problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 3/6] container/send-internet-mail: run.rb
by Luan Shengde 23 Oct '20

23 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 1

[PATCH lkp-tests] api-avx2neon: delete unnecessary fields
by Li Yuanchao 23 Oct '20

23 Oct '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 4/6] container/send-internet-mail: Dockerfile
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..598064b --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail && \ + apt-get install -y ruby-json && \ + apt-get install -y ruby-sinatra -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 23 Oct '20

23 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. Ips can bind to on macs after setting 'direct_mac' and 'direct_ips' for cluster jobsI [HowTo] 1) Need a cluster config file in $LKP_SRC/cluster, for example: ```yaml assumption: <assumption01> switch: switch01 ip0: 3 --- vm-2p8g--renwen: roles: [ server ] macs: [ "ec:f4:bb:cb:7b:92" ] vm-2p4g--renwen: roles: [ client ] macs: [ "ec:f4:bb:cb:54:90" ] ``` nr_node=2 cs1 switch=sw1 ip0=0 ($ip_prefix.0-.1) cs2 switch=sw1 ip0=2 ($ip_prefix.2-.3) "switch": only need if 2+ clusters share the same switch "ip0": only need if shared switch, and != 0 "assumption": the info about this cluster config 2) Scheduler get config from config file and set 'direct_macs' and 'direct_ips' for jobs. 3) When running a cluster job, bind direct_ips to direct_ macs in test environment. 4) Test client connect with daemon client to test job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 070a16b..8922874 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse_all(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,13 @@ class Sched # collect all job ids job_ids = [] of String + basic_config = cluster_config[0] + net_id = basic_config["net_id"]? || "192.168.222" + ip0 = basic_config["ip0"].as_i + # switch = basic_config["switch"].to_s # useless now + # steps for each host - cluster_config.each do |host, config| + cluster_config[-1].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +224,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

2 2

[PATCH v4 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 23 Oct '20

23 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 ++++ tests/iso2rootfs | 76 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 74 insertions(+), 8 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..e8a027b0dad2 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,12 +53,18 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t for yaml_t in "${yaml_vars[@]}" do - [ -n "$(eval echo "\$${yaml_t}")" ] || die "var$${yaml_t}$ is none" + [ -n "$(eval echo "\$${yaml_t}")" ] || die "cannot fine value of var: ${yaml_t}." done } @@ -133,7 +139,7 @@ mount_rootfs() install_pkgs "cifs-utils" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac @@ -164,7 +170,7 @@ check_sha256sum_update() if [ -n "$SHA256SUM_CACHE" ] then - [ -n "$SHA256SUM_NET" ] || die "get net sha256sum failed !!!" + [ -n "$SHA256SUM_NET" ] || die "cannot get sha256sum of dailybuild iso." if [ "$SHA256SUM_CACHE" == "$SHA256SUM_NET" ] then @@ -185,7 +191,9 @@ check_os() { log_info "starting check os ..." echo "${ALLOW_OS[@]}" | grep -wq "${os}" || - die "current host $${os}$ is not in $${ALLOW_OS[@]}$, exit !!!" + die "current os is not in allow os. + current os: ${os}. + allow os: $${ALLOW_OS[@]}$" } get_cgz() @@ -210,12 +218,12 @@ install_pkgs() local pkg_installer command -v yum && pkg_installer="yum" - [[ -n "$pkg_installer" ]] || die "can not find pkg installer !!!" + [[ -n "$pkg_installer" ]] || die "cannot find pkg installer." local pt for pt do - "$pkg_installer" install -y "$pt" || die "install pkg$$pt$ failed !!!" + "$pkg_installer" install -y "$pt" || die "cannot install pkg: $pt." done } @@ -364,7 +372,7 @@ test_rootfs() root_path="cifs:${ROOTFS_SERVER_PATH}/$(basename "${ROOTFS_DES_DIR}"),${ROOTFS_MOUNT_PARAM}" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac get_qemu_efi_fd @@ -404,6 +412,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "cannot find value of var: LKP_SRC." + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "cannot find test yaml in LKP_SRC/jobs. + test yaml: ${test_yaml}." + + generate_submit_append_str + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +473,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +492,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

2 1

[PATCH compass-ci] job.cr: initrd modules and headers with initramfs
by Xu Xijian 23 Oct '20

23 Oct '20

When run job with initramfs, we should initrd modules.cgz and headers.cgz too. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..75d75e5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -303,10 +303,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-default.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-default.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 1

[PATCH compass-ci] sparrow: install iproute
by Liu Yinsi 23 Oct '20

23 Oct '20

sparrow/2-network/br0 needs ip command. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..adfdfd5 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -20,6 +20,7 @@ pkgs=( liblockfile lockfile-progs busybox + iproute ) yum install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..48bb4c5 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -18,6 +18,7 @@ pkgs=( lockfile-progs bridge-utils busybox + iproute2 ) apt-get install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..bd1f728 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -19,6 +19,7 @@ pkgs=( redis wget busybox + iproute ) yum install -y "${pkgs[@]}" -- 2.23.0

1 0

[PATCH lkp-tests] lib/monitor.rb: only one way to ssh
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] Now, all the scenarios use "sshr" to login testbox. So delete the direct way to ssh. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index fcf7e220..9ce1d475 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -75,18 +75,9 @@ class Monitor return unless data['log'] data = JSON.parse(data['log']) - if job['uuid'] - # means Internet user - # using the redirection service to log in - return unless data['ssh_port'] + return unless data['ssh_port'] - ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) - else - # intranet user: direct login internal IP - return unless data['ip'] - - ssh_connect(data['ip'], 22, web_socket) - end + ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) end def ssh_connect(ssh_host, ssh_port, web_socket) -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: add 2 fields into job: direct_macs and direct_ips
by Ren Wen 22 Oct '20

22 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. After adding 2 fileds: 'direct_macs' and 'direct_ips', can bind ips to macs. [HowTo] When you test a cluster job, need to config cluster file correctly. Below is the template: ```yaml <hostname>: roles: [ <"server">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] <hostname>: roles: [ <"client">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] ``` For 'ips': 1) should be the intranet IP 2) all ips in cluster file should be in the same network segment 3) subnet mask is 24. [InFuture] Set default ips when there's no ips in cluster file or only one host has ips. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..a26091b 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -219,7 +219,8 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + job["direct_macs"] = config["macs"].as_a.join(" ") + job["direct_ips"] = config["ips"].as_a.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

4 4

[PATCH lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 22 Oct '20

22 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..f09c49e2 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -138,7 +138,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/aarch64-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +177,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/aarch64-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

3 3

[PATCH compass-ci] sbin: complier => compiler
by Cao Xueliang 22 Oct '20

22 Oct '20

[why] The container crystal-compiler build a image and named alpine:crystal-compiler, the sbin crystal-format use the image. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sbin/crystal-format | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/crystal-format b/sbin/crystal-format index 14f82b2..99f7a6a 100755 --- a/sbin/crystal-format +++ b/sbin/crystal-format @@ -15,7 +15,7 @@ cmd=( --rm -v $v_dir:$v_dir -w $PWD - alpine:crystal-complier + alpine:crystal-compiler crystal tool format "$@" ) -- 2.23.0

3 2

[PATCH compass-ci] auto_submit: testbox 2p8g => 2p16g for build-pkg
by Li Yuanchao 22 Oct '20

22 Oct '20

[why] 2p8g testbox only 1.3G left for build-pkg job, it's not enough. and there is error info write error: No space left on device Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index 1096014..edfbfb1 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -4,4 +4,4 @@ compass-ci/compass-ci: - testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=vm-2p8g os=centos os_version=7.6 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml archlinux: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

1 0

[PATCH v2 lkp-tests] lib: fix btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 22 Oct '20

22 Oct '20

When job.yaml contains "fs: **", and /srv/initrd/deps/**/**/fs.cgz(btrfs-progs), it will execute mkfs.btrfs -f $rootfs_partition. but job.yaml don't container "fs: **" to appear warning: mkfs.btrfs command not found Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/bootstrap.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..16789c5c 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,6 +456,7 @@ mount_rootfs() ROOTFS_DIR=/opt/rootfs mkdir -p $ROOTFS_DIR mount $rootfs_partition $ROOTFS_DIR || { + command -v mkfs.btrfs >/dev/null || return mkfs.btrfs -f $rootfs_partition mount $rootfs_partition $ROOTFS_DIR } -- 2.23.0

3 2

[PATCH lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 22 Oct '20

22 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..f0c2878c 100755 --- a/tests/iozone +++ b/tests/iozone @@ -7,7 +7,7 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

2 1

[PATCH lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 22 Oct '20

22 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..d17137e8 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,13 +1,17 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m -disk: 2HDD +disk: 1HDD fs: -- xfs - ext4 -- btrfs iosched: - kyber -- 2.23.0

2 1

[PATCH lab-z9] hosts: add common configs
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-512g | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 hosts/taishan200-2280-2s48p-512g diff --git a/hosts/taishan200-2280-2s48p-512g b/hosts/taishan200-2280-2s48p-512g new file mode 100644 index 0000000..5a4126a --- /dev/null +++ b/hosts/taishan200-2280-2s48p-512g @@ -0,0 +1,4 @@ +nr_cpu: 96 +nr_node: 4 +memory: 512G +arch: aarch64 -- 2.23.0

1 0

[PATCH compass-ci] git_mirror: put fork_stat to es
by Li Yuanchao 22 Oct '20

22 Oct '20

[why] We want to know whether git-mirror works well, so save last 10 times fetch_time and new_refs_time, and put them to es Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 593d1eb..2b78523 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -98,8 +98,10 @@ class MirrorMain @fork_stat[stat_key] = { queued: false, priority: 0, - fetch_time: nil, - new_refs_time: nil + fetch_time: [], + offset_fetch: 0, + new_refs_time: [], + offset_new_refs: 0 } end @@ -152,10 +154,9 @@ class MirrorMain feedback_info = @feedback_queue.pop(true) git_repo = feedback_info[:git_repo] - @fork_stat[git_repo][:queued] = false + update_fork_stat(git_repo, feedback_info[:possible_new_refs]) return unless feedback_info[:possible_new_refs] - @fork_stat[git_repo][:priority] += 1 return reload_fork_info if git_repo == 'upstream-repos/upstream-repos' new_refs = check_new_refs(git_repo) @@ -196,8 +197,8 @@ class MirrorMain @git_info[git_repo] = YAML.safe_load(File.open(repodir)) @git_info[git_repo]['git_repo'] = git_repo @git_info[git_repo].merge!(@defaults[project]) if @defaults[project] - es_repo_update(git_repo, @git_info[git_repo]) fork_stat_init(git_repo) + es_repo_update(git_repo) @priority_queue.push git_repo, @priority @priority += 1 end @@ -255,11 +256,35 @@ class MirrorMain end end - def es_repo_update(git_repo, repo_info) + def es_repo_update(git_repo) + repo_info = @git_info[git_repo].merge(@fork_stat[git_repo]) + repo_info.delete(:cur_refs) if repo_info.key?(:cur_refs) body = { "doc": repo_info, "doc_as_upsert": true } @es_client.update(index: 'repo', type: '_doc', id: git_repo, body: body) end + + def update_stat_fetch(git_repo) + @fork_stat[git_repo][:queued] = false + offset_fetch = @fork_stat[git_repo][:offset_fetch] + offset_fetch = 0 if offset_fetch >= 10 + @fork_stat[git_repo][:fetch_time][offset_fetch] = Time.now.to_s + @fork_stat[git_repo][:offset_fetch] = offset_fetch + 1 + end + + def update_stat_new_refs(git_repo) + @fork_stat[git_repo][:priority] += 1 + offset_new_refs = @fork_stat[git_repo][:offset_new_refs] + offset_new_refs = 0 if offset_new_refs >= 10 + @fork_stat[git_repo][:new_refs_time][offset_new_refs] = Time.now.to_s + @fork_stat[git_repo][:offset_new_refs] = offset_new_refs + 1 + end + + def update_fork_stat(git_repo, possible_new_refs) + update_stat_fetch(git_repo) + update_stat_new_refs(git_repo) if possible_new_refs + es_repo_update(git_repo) + end end -- 2.23.0

1 0

[PATCH v2 compass-ci] /src/lib/web_backend.rb: support multi-fields search in "/get_jobs"
by Zhang Yuhang 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 46 ++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 48cfadb..c3f6bef 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -248,23 +248,32 @@ def es_search(must, size, from) return es_query(query)['hits']['hits'], total end -def get_job(result) - job = {} - ALL_FIELDS.each do |f| - job[f] = result[f] +def get_jobs_result(result) + jobs = [] + result.each do |r| + job = {} + ALL_FIELDS.each do |f| + job[f] = r['_source'][f] + end + jobs << job end - job + jobs end -def search_job(git_repo, page_size, page_num) +def search_job(condition_fields, page_size, page_num) must = [] - must << { regexp: { upstream_repo: ".*#{git_repo}.*" } } if git_repo - jobs = [] - result, total = es_search(must, page_size, page_num * page_size) - result.each do |r| - jobs << get_job(r['_source']) + FIELDS.each do |field| + value = space_to_nil(condition_fields[field]) + next unless value + + must << if field.to_s == 'upstream_repo' + { regexp: { field => ".*#{value}.*" } } + else + { term: { field => value } } + end end - return jobs, total + result, total = es_search(must, page_size, page_num * page_size) + return get_jobs_result(result), total end def get_banner(git_repo, branches) @@ -291,17 +300,14 @@ def get_optimize_jobs_braches(jobs) end def get_jobs_body(params) - git_repo = space_to_nil(params[:upstream_repo]) - page_size = get_positive_number(params[:page_size], 20) - page_num = get_positive_number(params[:page_num], 1) - 1 - jobs, total = search_job(git_repo, page_size, page_num) + page_size = get_positive_number(params.delete(:page_size), 20) + page_num = get_positive_number(params.delete(:page_num), 1) - 1 + jobs, total = search_job(params, page_size, page_num) jobs, branches = get_optimize_jobs_braches(jobs) { total: total, - filter: { - upstream_repo: git_repo - }, - banner: get_banner(git_repo, branches), + filter: params, + banner: get_banner(params[:upstream_repo], branches), jobs: jobs, fields: FIELDS }.to_json -- 2.23.0

1 0

[PATCH v4 lkp-tests] sshd: send email to users for login services
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 99 +++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 74 insertions(+), 25 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..82db61a6 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key -# email +# my_email +# sshr_port +# sshr_port_len +# sshr_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -11,40 +21,79 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { - deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") - data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh - -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: - - Login: - ssh root@$PUB_IP - Due time: - $deadline - - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch + data="To: $my_email +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_username: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: + + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch Regards -compass-ci +Compass-Ci " } +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Applying $testbox failed + +Dear $my_username: + Sorry to inform you that your application failed. + You may need to wait a moment, or check whether your pub_key exists. + +Regards +Compass-Ci +" +} + +compose_email() +{ + deadline=$(date -d "$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$pub_key" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

1 0

[PATCH v3 lkp-tests] sshd: send emails to login services for users
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 98 ++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 74 insertions(+), 24 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..1ebc9d00 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key -# email +# my_email +# sshr_port +# sshr_port_len +# sshr_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -11,40 +21,80 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") - data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh - -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: - - Login: - ssh root@$PUB_IP - Due time: - $deadline - - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch + data="To: $my_email +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_username: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: + + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch + +Regards +Compass-Ci +" +} + +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Applying $testbox failed + +Dear $my_username: + Sorry to inform you that your application failed. + You may need to wait a moment, or check whether your pub_key exists. Regards -compass-ci +Compass-Ci " } +compose_email() +{ + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$pub_key" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

2 2

[PATCH compass-ci] lib/sched.cr: add result_root output logs
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] The result_root directory of the job stores the running results. It plays an important role in locating problems. So puts this info to docker logs, submit -m can show result_root. [Output Example] {"job_id"=>"z9.1", "message"=>"", "job_state"=>"submit", "result_root"=>"/srv/result/iperf/vm-2p8g/2020-10-22/z9.1"} {"job_id"=>"z9.1", "result_root"=>"/srv/result/iperf/vm-2p8g/2020-10-22/z9.1", "job_state"=>"set result root"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..070a16b 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -227,6 +227,7 @@ class Sched "job_id" => job_id, "message" => message.to_s, "job_state" => "submit", + "result_root" => "/srv#{job.result_root}" } return job_messages if response["error"]? end @@ -278,6 +279,7 @@ class Sched "job_id" => job_id, "message" => message.to_s, "job_state" => "submit", + "result_root" => "/srv#{job.result_root}" }] end @@ -441,6 +443,7 @@ class Sched if job job.update({"testbox" => testbox}) job.set_result_root + puts %({"job_id": "#{job_id}", "result_root": "/srv#{job.result_root}", "job_state": "set result root"}) @redis.set_job(job) end return job -- 2.23.0

1 0

[PATCH lkp-tests] fixed a bug that modified shellrc in root-path when make install
by Xie Zheng 22 Oct '20

22 Oct '20

--- Makefile | 4 ++-- sbin/set-env.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 37387809..907dba9e 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ ifeq ($(TARGET_DIR_BIN), ) TARGET_DIR_BIN := /usr/local/bin endif -ifneq ($(shell whoami), "root") +ifneq ($(shell whoami), root) TARGET_DIR_BIN := $${HOME}/bin endif @@ -14,8 +14,8 @@ subsystem: install: mkdir -p $(TARGET_DIR_BIN) ln -sf $(shell pwd)/bin/lkp $(TARGET_DIR_BIN)/lkp - bash sbin/install-dependencies.sh bash sbin/set-env.sh + sudo bash sbin/install-dependencies.sh .PHONY: doc doc: diff --git a/sbin/set-env.sh b/sbin/set-env.sh index b9046519..a8b161fa 100644 --- a/sbin/set-env.sh +++ b/sbin/set-env.sh @@ -5,7 +5,7 @@ write_shellrc() { echo "export LKP_SRC=$PWD" >> $(shell_profile) - echo "export PATH=\$PATH:$PWD/sbin:$PWD/bin" >> $(shell_profile) + echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $(shell_profile) } write_host() -- 2.23.0

1 0

[PATCH lab-z9] hosts: add "arch" field to host files
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/dc-1g | 1 + hosts/dc-2g | 1 + hosts/dc-4g | 1 + hosts/dc-8g | 1 + hosts/taishan200-2280-2s48p-256g | 1 + hosts/taishan200-2280-2s48p-256g--a101 | 1 + hosts/taishan200-2280-2s48p-256g--a102 | 1 + hosts/taishan200-2280-2s48p-256g--a11 | 1 + hosts/taishan200-2280-2s48p-256g--a13 | 1 + hosts/taishan200-2280-2s48p-256g--a14 | 1 + hosts/taishan200-2280-2s48p-256g--a15 | 1 + hosts/taishan200-2280-2s48p-256g--a17 | 1 + hosts/taishan200-2280-2s48p-256g--a18 | 1 + hosts/taishan200-2280-2s48p-256g--a19 | 1 + hosts/taishan200-2280-2s48p-256g--a21 | 1 + hosts/taishan200-2280-2s48p-256g--a22 | 1 + hosts/taishan200-2280-2s48p-256g--a24 | 1 + hosts/taishan200-2280-2s48p-256g--a25 | 1 + hosts/taishan200-2280-2s48p-256g--a26 | 1 + hosts/taishan200-2280-2s48p-256g--a27 | 1 + hosts/taishan200-2280-2s48p-256g--a28 | 1 + hosts/taishan200-2280-2s48p-256g--a29 | 1 + hosts/taishan200-2280-2s48p-256g--a30 | 1 + hosts/taishan200-2280-2s48p-256g--a31 | 1 + hosts/taishan200-2280-2s48p-256g--a32 | 1 + hosts/taishan200-2280-2s48p-256g--a33 | 1 + hosts/taishan200-2280-2s48p-256g--a34 | 1 + hosts/taishan200-2280-2s48p-256g--a52 | 1 + hosts/taishan200-2280-2s48p-256g--a53 | 1 + hosts/taishan200-2280-2s48p-256g--a60 | 1 + hosts/taishan200-2280-2s48p-256g--a61 | 1 + hosts/taishan200-2280-2s48p-256g--a62 | 1 + hosts/taishan200-2280-2s48p-256g--a66 | 1 + hosts/taishan200-2280-2s48p-256g--a67 | 1 + hosts/taishan200-2280-2s48p-256g--a68 | 1 + hosts/taishan200-2280-2s48p-256g--a69 | 1 + hosts/taishan200-2280-2s48p-256g--a70 | 1 + hosts/taishan200-2280-2s48p-256g--a71 | 1 + hosts/taishan200-2280-2s48p-256g--a95 | 1 + hosts/taishan200-2280-2s64p-256g | 1 + hosts/taishan200-2280-2s64p-256g--a10 | 1 + hosts/taishan200-2280-2s64p-256g--a3 | 1 + hosts/taishan200-2280-2s64p-256g--a36 | 1 + hosts/taishan200-2280-2s64p-256g--a4 | 1 + hosts/taishan200-2280-2s64p-256g--a40 | 1 + hosts/taishan200-2280-2s64p-256g--a41 | 1 + hosts/taishan200-2280-2s64p-256g--a42 | 1 + hosts/taishan200-2280-2s64p-256g--a8 | 1 + hosts/taishan200-2280-2s64p-256g--a9 | 1 + hosts/vm-1p1g | 1 + hosts/vm-1p1g-pxe | 1 + hosts/vm-2p1g | 1 + hosts/vm-2p1g-pxe | 1 + hosts/vm-2p4g | 1 + hosts/vm-2p4g-pxe | 1 + hosts/vm-2p8g | 1 + hosts/vm-2p8g-pxe | 1 + hosts/vm-hi1620-1p1g | 1 + hosts/vm-hi1620-2p1g | 1 + hosts/vm-hi1620-2p4g | 1 + hosts/vm-hi1620-2p8g | 1 + 61 files changed, 61 insertions(+) diff --git a/hosts/dc-1g b/hosts/dc-1g index ec1e59b..afeba49 100644 --- a/hosts/dc-1g +++ b/hosts/dc-1g @@ -1 +1,2 @@ +arch: aarch64 memory: 1g diff --git a/hosts/dc-2g b/hosts/dc-2g index b9509a2..6956411 100644 --- a/hosts/dc-2g +++ b/hosts/dc-2g @@ -1 +1,2 @@ +arch: aarch64 memory: 2g diff --git a/hosts/dc-4g b/hosts/dc-4g index 52e0636..1407332 100644 --- a/hosts/dc-4g +++ b/hosts/dc-4g @@ -1 +1,2 @@ +arch: aarch64 memory: 4g diff --git a/hosts/dc-8g b/hosts/dc-8g index 333d625..ec61c1d 100644 --- a/hosts/dc-8g +++ b/hosts/dc-8g @@ -1 +1,2 @@ +arch: aarch64 memory: 8g diff --git a/hosts/taishan200-2280-2s48p-256g b/hosts/taishan200-2280-2s48p-256g index abba938..642e8e1 100644 --- a/hosts/taishan200-2280-2s48p-256g +++ b/hosts/taishan200-2280-2s48p-256g @@ -1,3 +1,4 @@ +arch: aarch64 nr_cpu: 96 nr_node: 4 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a101 b/hosts/taishan200-2280-2s48p-256g--a101 index 1226913..f8a0429 100644 --- a/hosts/taishan200-2280-2s48p-256g--a101 +++ b/hosts/taishan200-2280-2s48p-256g--a101 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-256g--a102 index 267a962..e1d3f10 100644 --- a/hosts/taishan200-2280-2s48p-256g--a102 +++ b/hosts/taishan200-2280-2s48p-256g--a102 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G diff --git a/hosts/taishan200-2280-2s48p-256g--a11 b/hosts/taishan200-2280-2s48p-256g--a11 index a281e24..619dc06 100644 --- a/hosts/taishan200-2280-2s48p-256g--a11 +++ b/hosts/taishan200-2280-2s48p-256g--a11 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 index 991694c..468c175 100644 --- a/hosts/taishan200-2280-2s48p-256g--a13 +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 index d6b2970..24ea9dc 100644 --- a/hosts/taishan200-2280-2s48p-256g--a14 +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 index d66b003..9c28258 100644 --- a/hosts/taishan200-2280-2s48p-256g--a15 +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 index 7485faa..16b065d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a17 +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 index 8673d64..9d1ec01 100644 --- a/hosts/taishan200-2280-2s48p-256g--a18 +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 index 8ffac96..217eaa9 100644 --- a/hosts/taishan200-2280-2s48p-256g--a19 +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 index b1fae8b..738738d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a21 +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 index 2d46733..bb29d94 100644 --- a/hosts/taishan200-2280-2s48p-256g--a22 +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index d666366..60ea1d2 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index 8e43697..916513e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 index 15cc4e0..4a5a0f3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a26 +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 index e6b3135..129589c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a27 +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 index f27d5e0..f8a77f8 100644 --- a/hosts/taishan200-2280-2s48p-256g--a28 +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 index 74c614f..37814cc 100644 --- a/hosts/taishan200-2280-2s48p-256g--a29 +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 index 568316f..f722edb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a30 +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 index 4f49685..7d6c67c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a31 +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 index ac0279e..fb17e38 100644 --- a/hosts/taishan200-2280-2s48p-256g--a32 +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 index 20b8323..a3a6b03 100644 --- a/hosts/taishan200-2280-2s48p-256g--a33 +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 index b657141..5a15949 100644 --- a/hosts/taishan200-2280-2s48p-256g--a34 +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 index 339b2a8..2bd7096 100644 --- a/hosts/taishan200-2280-2s48p-256g--a52 +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 index e1dca3a..bf0894b 100644 --- a/hosts/taishan200-2280-2s48p-256g--a53 +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 index 9a92a85..2255f2f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a60 +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 index bef5d8a..73079cb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a61 +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 index b41eaf7..abaac6e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a62 +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index c9fb714..fce93d1 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index a5ab220..6b32609 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index afd0101..4bdd8f4 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index 8ac90cc..3d3be80 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 463febf..89c3c77 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 index 501ea4b..3121212 100644 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ b/hosts/taishan200-2280-2s48p-256g--a71 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a95 b/hosts/taishan200-2280-2s48p-256g--a95 index 3fd8124..213d10f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a95 +++ b/hosts/taishan200-2280-2s48p-256g--a95 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G diff --git a/hosts/taishan200-2280-2s64p-256g b/hosts/taishan200-2280-2s64p-256g index ab798a3..16349d9 100644 --- a/hosts/taishan200-2280-2s64p-256g +++ b/hosts/taishan200-2280-2s64p-256g @@ -1,3 +1,4 @@ +arch: aarch64 nr_cpu: 128 nr_node: 4 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index f99a406..cdb32f5 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a3 b/hosts/taishan200-2280-2s64p-256g--a3 index 744978e..8f34f3f 100644 --- a/hosts/taishan200-2280-2s64p-256g--a3 +++ b/hosts/taishan200-2280-2s64p-256g--a3 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a36 b/hosts/taishan200-2280-2s64p-256g--a36 index 51ba3a4..9757144 100644 --- a/hosts/taishan200-2280-2s64p-256g--a36 +++ b/hosts/taishan200-2280-2s64p-256g--a36 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a4 b/hosts/taishan200-2280-2s64p-256g--a4 index 0f73c52..bcfaea7 100644 --- a/hosts/taishan200-2280-2s64p-256g--a4 +++ b/hosts/taishan200-2280-2s64p-256g--a4 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 index 6825c38..d9a9937 100644 --- a/hosts/taishan200-2280-2s64p-256g--a40 +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 index 5937093..91531fd 100644 --- a/hosts/taishan200-2280-2s64p-256g--a41 +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 index 1f5d3f0..ed1f188 100644 --- a/hosts/taishan200-2280-2s64p-256g--a42 +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a8 b/hosts/taishan200-2280-2s64p-256g--a8 index 64f2750..e515306 100644 --- a/hosts/taishan200-2280-2s64p-256g--a8 +++ b/hosts/taishan200-2280-2s64p-256g--a8 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 index b21e4d3..ce3aa0a 100644 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ b/hosts/taishan200-2280-2s64p-256g--a9 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/vm-1p1g b/hosts/vm-1p1g index c64cc36..3af3646 100644 --- a/hosts/vm-1p1g +++ b/hosts/vm-1p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-1p1g-pxe b/hosts/vm-1p1g-pxe index 233f639..4571c23 100644 --- a/hosts/vm-1p1g-pxe +++ b/hosts/vm-1p1g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p1g b/hosts/vm-2p1g index c781033..0293ce9 100644 --- a/hosts/vm-2p1g +++ b/hosts/vm-2p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p1g-pxe b/hosts/vm-2p1g-pxe index 6129d20..7f76598 100644 --- a/hosts/vm-2p1g-pxe +++ b/hosts/vm-2p1g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p4g b/hosts/vm-2p4g index 7e9570d..219c4c9 100644 --- a/hosts/vm-2p4g +++ b/hosts/vm-2p4g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p4g-pxe b/hosts/vm-2p4g-pxe index 8ba4e58..5f0de24 100644 --- a/hosts/vm-2p4g-pxe +++ b/hosts/vm-2p4g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p8g b/hosts/vm-2p8g index e5d5f84..6249322 100644 --- a/hosts/vm-2p8g +++ b/hosts/vm-2p8g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p8g-pxe b/hosts/vm-2p8g-pxe index ae83d6f..9b09402 100644 --- a/hosts/vm-2p8g-pxe +++ b/hosts/vm-2p8g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-hi1620-1p1g b/hosts/vm-hi1620-1p1g index bf2a7b3..2226bdc 100644 --- a/hosts/vm-hi1620-1p1g +++ b/hosts/vm-hi1620-1p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 1 diff --git a/hosts/vm-hi1620-2p1g b/hosts/vm-hi1620-2p1g index 1c4786a..e2952eb 100644 --- a/hosts/vm-hi1620-2p1g +++ b/hosts/vm-hi1620-2p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p4g b/hosts/vm-hi1620-2p4g index f2a5b5e..83d8133 100644 --- a/hosts/vm-hi1620-2p4g +++ b/hosts/vm-hi1620-2p4g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p8g b/hosts/vm-hi1620-2p8g index 1780524..a4a0577 100644 --- a/hosts/vm-hi1620-2p8g +++ b/hosts/vm-hi1620-2p8g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 -- 2.23.0

1 0

[PATCH compass-ci] sched: no need limit access_key update
by Cao Xueliang 22 Oct '20

22 Oct '20

[why] When update result_root the access_key need update too. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index c6fb0fd..8f5dd36 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -191,7 +191,7 @@ class Job end private def set_access_key - self["access_key"] = "#{Random::Secure.hex(10)}-#{id}" unless @hash["access_key"]? + self["access_key"] = "#{Random::Secure.hex(10)}-#{id}" self["access_key_file"] = File.join("/srv/", "#{result_root}", ".#{access_key}") end -- 2.23.0

1 0

[PATCH v2 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 22 Oct '20

22 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 +++++ tests/iso2rootfs | 60 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..9a40f7b07892 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,6 +53,12 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t @@ -404,6 +410,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "no LKP_SRC !!!" + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "test yaml$${test_yaml}$ exist in {LKP_SRC}/jobs !!!" + + generate_submit_append_str + [ -z "${SUBMIT_APPEND_STR}" ] || SUBMIT_APPEND_STR="-a ${SUBMIT_APPEND_STR}" + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +471,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +490,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

3 5

[PATCH compass-ci] code-spec: grep-friendly fixed substring
by Wu Fengguang 22 Oct '20

22 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/code-spec.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index 37c86b6..6f2dddd 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -19,6 +19,13 @@ code注意事项 - 如果注释仅仅是代码的简单重复, 不提供额外信息量, 不要写. +- puts/printf字符串应当容易搜索(grep)。特别注意包含足够长的固定内容substring. +例如： + die "test yaml$${test_yaml}$ not exist in ${LKP_SRC}/jobs !!!" +==> + die "cannot find test yaml ($test_yaml) in $LKP_SRC/jobs" + ^^^^^^^^^^^^^^^^^^^^^ grep-friendly fixed substring + 重构友好编码 ============ -- 2.23.0

1 0

[PATCH v2 lkp-tests] sshd: send emails to login services for users
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index cd57be4f..1ebc9d00 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,16 +1,16 @@ #!/bin/sh # pub_key # my_email -# my_uuid -# ssh_port_len -# ssh_port_base +# sshr_port +# sshr_port_len +# sshr_port_base . $LKP_SRC/lib/http.sh sshr_ip=$LKP_SERVER -sshr_port=5050 -[ -n "$ssh_port_len" ] || ssh_port_len=2000 -[ -n "$ssh_port_base" ] || ssh_port_base=50000 +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -23,6 +23,7 @@ run_ssh() data_success() { + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") data="To: $my_email Subject: [NOTIFY Compass-ci] $testbox ready to use @@ -43,7 +44,7 @@ Dear $my_username: $os $os_version $os_arch Regards -Compass-ci +Compass-Ci " } @@ -54,10 +55,10 @@ Subject: [NOTIFY Compass-ci] Applying $testbox failed Dear $my_username: Sorry to inform you that your application failed. - You may need to wait for a while and then try again. + You may need to wait a moment, or check whether your pub_key exists. Regards -Compass-ci +Compass-Ci " } @@ -73,11 +74,11 @@ compose_email() set_port() { - [ -n "$my_uuid" ] || return + [ -n "$pub_key" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%"$ssh_port_len"+"$ssh_port_base")) + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return @@ -87,7 +88,7 @@ set_port() run_email() { - [ -n "$my_email" ] && [ -n "$my_uuid" ] || return 0 + [ -n "$my_email" ] || return 0 my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" -- 2.23.0

2 2