- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 1/2] container/web_backend: add a interface /get_testboxes
by Li Yuanchao 02 Dec '20

02 Dec '20

It would return a testbox list to web request Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/web-backend/web-backend | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/container/web-backend/web-backend b/container/web-backend/web-backend index d68e9f2..11ba604 100755 --- a/container/web-backend/web-backend +++ b/container/web-backend/web-backend @@ -103,3 +103,10 @@ post '/compare_template' do request.body.rewind # in case someone already read it compare_template(request.body.read) end + +# GET /get_testboxes +# Response: +# - { total: total, testboxes[ 'testbox1', 'testbox2', ... ] } +get '/get_testboxes' do + query_testboxes +end -- 2.23.0

1 0

[PATCH v4 compass-ci] scheduler: fix the values diplayed in job.yaml
by Bai Jing 02 Dec '20

02 Dec '20

[original values] initrds_uri: 'initrd http://172.168.131.113:8800/initrd/osimage/openeuler/aarch64/20.03/20201022… initrd http://172.168.131.113:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig… initrd http://172.17.0.1:8000/os/openeuler/aarch64/20.03/boot/modules-4.19.90-2003… [now values] - initrd http://172.168.131.113:8800/initrd/osimage/openeuler/aarch64/20.03/20201022… - initrd http://172.168.131.113:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig… - initrd http://172.17.0.1:8000/os/openeuler/aarch64/20.03/boot/modules-4.19.90-2003… Signed-off-by: Bai Jing <799286817(a)qq.com> --- src/lib/job.cr | 8 ++++---- src/scheduler/find_job_boot.cr | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 2a811cb..a4345e0 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -413,7 +413,7 @@ class Job end private def get_initrds - temp_initrds = [] of String + temp_initrds = [] of String if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) @@ -437,13 +437,13 @@ class Job end private def set_initrds_uri - uris = "" + initrds_uri_values = [] of JSON::Any get_initrds().each do |initrd| - uris += "initrd #{initrd}\n" + initrds_uri_values << JSON::Any.new("initrd #{initrd}") end - self["initrds_uri"] = uris + @hash["initrds_uri"] = JSON::Any.new(initrds_uri_values) end private def set_user_lkp_src diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 9805464..3b68b59 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -134,7 +134,7 @@ class Sched private def get_boot_ipxe(job : Job) response = "#!ipxe\n\n" - response += job.initrds_uri + response += Array(String).from_json(job.initrds_uri).join("\n") + "\n" response += job.kernel_uri response += job.kernel_params response += "\nboot\n" -- 2.23.0

1 0

[PATCH v3 compass-ci] scheduler: fix the values diplayed in job.yaml
by Bai Jing 02 Dec '20

02 Dec '20

[original values] initrds_uri: 'initrd http://172.168.131.113:8800/initrd/osimage/openeuler/aarch64/20.03/20201022… initrd http://172.168.131.113:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig… initrd http://172.17.0.1:8000/os/openeuler/aarch64/20.03/boot/modules-4.19.90-2003… [now values] - initrd http://172.168.131.113:8800/initrd/osimage/openeuler/aarch64/20.03/20201022… - initrd http://172.168.131.113:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig… - initrd http://172.17.0.1:8000/os/openeuler/aarch64/20.03/boot/modules-4.19.90-2003… Signed-off-by: Bai Jing <799286817(a)qq.com> --- src/lib/job.cr | 6 ++++-- src/scheduler/find_job_boot.cr | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 2a811cb..ae3ba6e 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -413,7 +413,7 @@ class Job end private def get_initrds - temp_initrds = [] of String + temp_initrds = [] of String if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) @@ -438,12 +438,14 @@ class Job private def set_initrds_uri uris = "" + initrds_uri_values = [] of JSON::Any get_initrds().each do |initrd| uris += "initrd #{initrd}\n" + initrds_uri_values << JSON::Any.new("initrd #{initrd}") end - self["initrds_uri"] = uris + @hash["initrds_uri"] = JSON::Any.new(initrds_uri_values) end private def set_user_lkp_src diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 9805464..3b68b59 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -134,7 +134,7 @@ class Sched private def get_boot_ipxe(job : Job) response = "#!ipxe\n\n" - response += job.initrds_uri + response += Array(String).from_json(job.initrds_uri).join("\n") + "\n" response += job.kernel_uri response += job.kernel_params response += "\nboot\n" -- 2.23.0

2 1

[PATCH v3 lkp-tests] spec/submit: add job yaml for changing multi-args
by Bai Jing 02 Dec '20

02 Dec '20

[test cmd] rspec $LKP_SRC/spec/submit_spec.rb [output] /lkp-tests/spec/submit/matrix/unixbench-spec.yaml => /lkp-tests/spec/submit/matrix/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml /lkp-tests/spec/submit/matrix/unixbench-spec.yaml => /lkp-tests/spec/submit/matrix/unixbench-spec/unixbench-spec-3-300-shell1.yaml Signed-off-by: Bai Jing <799286817(a)qq.com> --- spec/submit/matrix/unixbench-spec.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/submit/matrix/unixbench-spec.yaml b/spec/submit/matrix/unixbench-spec.yaml index e17322b4..851f0473 100644 --- a/spec/submit/matrix/unixbench-spec.yaml +++ b/spec/submit/matrix/unixbench-spec.yaml @@ -2,8 +2,8 @@ suite: unixbench runtime: 300s -nr_task test: - - 1 whetstone-double - - 3 shell1 +nr_task|test: + - 1|whetstone-double + - 3|shell1 unixbench: -- 2.23.0

1 0

[PATCH v1 compass-ci] sched: optimize regular matching in format_string function
by Cao Xueliang 02 Dec '20

02 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/pp_params.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/scheduler/pp_params.cr b/src/scheduler/pp_params.cr index da8d8f3..03bf0b8 100644 --- a/src/scheduler/pp_params.cr +++ b/src/scheduler/pp_params.cr @@ -50,10 +50,8 @@ class Job private def format_string(original_str) temp = [] of String - original_str.each_char do |char| - if "#{char}" =~ /-|\.|\/|\w/ - temp << "#{char}".gsub("\/", "-") - end + original_str.gsub('/', '-').each_char do |char| + temp << "#{char}" if "#{char}" =~ /\w|-|\./ end return temp.join() -- 2.23.0

1 0

[PATCH compass-ci] sched: improve format string function
by Cao Xueliang 02 Dec '20

02 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/pp_params.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/scheduler/pp_params.cr b/src/scheduler/pp_params.cr index da8d8f3..03bf0b8 100644 --- a/src/scheduler/pp_params.cr +++ b/src/scheduler/pp_params.cr @@ -50,10 +50,8 @@ class Job private def format_string(original_str) temp = [] of String - original_str.each_char do |char| - if "#{char}" =~ /-|\.|\/|\w/ - temp << "#{char}".gsub("\/", "-") - end + original_str.gsub('/', '-').each_char do |char| + temp << "#{char}" if "#{char}" =~ /\w|-|\./ end return temp.join() -- 2.23.0

2 4

[PATCH compass-ci] How to add a machine to compass-ci
by Zhang Dewan 02 Dec '20

02 Dec '20

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- ...00\350\210\254\350\277\207\347\250\213.md" | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 "doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" diff --git "a/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" "b/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" new file mode 100644 index 0000000..3fa402b --- /dev/null +++ "b/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" @@ -0,0 +1,49 @@ + +这个过程可以使用“注册--发现”的模型来理解。 + +## 一台机器成为测试机: +- 需要完成注册 +- 需要能够执行调度器（scheduler）给它的任务 + +## 注册 +### 1. 注册到哪里 +测试机需要注册到z9上面的redis-db. + +x.x.x.x:xxxx> `hgetall "sched/mac2host"` # 登录redis, 这个命令可以查看已经注册的测试机信息 + +### 2. 怎么注册 +使用scheduler服务接口注册，模板： +`curl -X PUT "http://$server_ip:$port/set_host_mac?hostname=$hostname&mac=$mac` + +可以看到，这个调用需要两个参数：mac和hostname + +#### 2.1 接口参数说明 +##### 2.1.1 mac +当前我们的机器都是局域网内的资源，所以没有比使用mac注册更直接、方便的了。 + +###### 2.1.1.1 如何获取mac地址，这里用TaiShan服务器举例： +- 登录bmc，打开信息--系统信息--网络--网卡--第一张网卡--port1，即可查看。 +- 登录bios，进入Advanced--Network Device List，即可查看。 +- 使用一台和机器的bmc同网段的机器，执行以下命令即可查看： +`ipmitool -I lanplus -H $cur_ip -U $iBMC_user -P $iBMC_passwd raw 0x30 0x90 0x01 0x03 0x0${nic_idx} 0x00|awk '{print $5,$6,$7,$8,$9,$10}' |sed 's/\s/\-/g'` +- 给服务器安装os，登录上去，执行命令，即可查看： +`ip link | grep "^2:" -A 1| grep link |awk '{print $2}` + +##### 2.1.2 hostname +如同每一张网卡的mac是规则且唯一的，hostname作为资源的标识也是如此。 + +###### 2.1.2.1 如何定义hostname，这里还是以Taishan服务器为例： +` taishan200-2280-${cpuNum}s${coreNum}p-${memSize:-256}g--${aN}` +> taishan200-2280 服务器型号 +> cpuNum cpu个数 +> coreNum cpu核数 +> memSize 内存大小 +> aN 该类型机器编号 + +## 发现 +### 1. 怎么发现测试机并给它分配任务 +测试机器需要将启动方式设置pxe启动，然后在启动环节向scheduler发起请求，获取启动参数和任务信息。 + +### 2. 设置PXE启动 +- 华为服务器默认pxe启动，无需配置。 +- 其他设备，根据产品类型，按文档操作即可。 -- 2.23.0

2 2

[PATCH lab-z9] hosts: add host file of x86_64
by Zhang Yu 02 Dec '20

02 Dec '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/2288hv5-2s44p-384g | 4 ++++ hosts/2288hv5-2s44p-384g--b6 | 12 ++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 hosts/2288hv5-2s44p-384g create mode 100644 hosts/2288hv5-2s44p-384g--b6 diff --git a/hosts/2288hv5-2s44p-384g b/hosts/2288hv5-2s44p-384g new file mode 100644 index 0000000..ffb5d45 --- /dev/null +++ b/hosts/2288hv5-2s44p-384g @@ -0,0 +1,4 @@ +arch: x86_64 +nr_cpu: 88 +nr_node: 2 +memory: 384G diff --git a/hosts/2288hv5-2s44p-384g--b6 b/hosts/2288hv5-2s44p-384g--b6 new file mode 100644 index 0000000..2262524 --- /dev/null +++ b/hosts/2288hv5-2s44p-384g--b6 @@ -0,0 +1,12 @@ +arch: x86_64 +nr_node: 2 +nr_cpu: 88 +memory: 384G +nr_ssd_partitions: 1 +ssd_partitions: + - /dev/disk/by-id/ata-MTFDDAK480TDC-1AT1ZABYY_20032653CE5E +mac_addr: + - 6c:eb:b6:50:c0:31 + - 6c:eb:b6:50:c0:32 +model_name: Intel(R) Xeon(R) Gold 6266C CPU @ 3.00GHz +ipmi_ip: 9.3.3.11 -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/manual: update apply-account.md
by Luan Shengde 02 Dec '20

02 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 362ead2..c1b5206 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -16,6 +16,7 @@ Email template: --- - please replace the 3 parts in {{ }} with your information. +- the email name should be in english, as: "David Rientjes" in the template. - my oss commit: a git commit URL that has your contribution. ## step 2: receive an email @@ -36,3 +37,11 @@ Which contains account information for you: ## step 4: try it out Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + +## notice: + +In case that you received a failed email with message as below: + + The repo url for your commit is not in our upstream-repo list. + +Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: update apply-account.md
by Luan Shengde 02 Dec '20

02 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 362ead2..4b18428 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -16,6 +16,7 @@ Email template: --- - please replace the 3 parts in {{ }} with your information. +- the email name should be in english, as: "David Rientjes" in the template. - my oss commit: a git commit URL that has your contribution. ## step 2: receive an email @@ -36,3 +37,11 @@ Which contains account information for you: ## step 4: try it out Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + +## notice: + +In case of you receive failed email with message: + + The repo url for your commit is not in our upstream-repo list. + +Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… -- 2.23.0

3 4