January 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v6 compass-ci] lib/sched_client.rb: add some common methods for provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

[why] we have some providers: ipxe, container, libvirt all of them need register testbox info and consume job from scheduler [how] each method corresponds to a interface, so ensuring the independence of them. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/sched_client.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/lib/sched_client.rb b/lib/sched_client.rb index fec4b14..93f9ccd 100644 --- a/lib/sched_client.rb +++ b/lib/sched_client.rb @@ -16,4 +16,37 @@ class SchedClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/submit_job") resource.post(job_json) end + + # scheduler API /boot.:boot_type/:parameter/:value + # boot_type parameter + # 1 ipxe mac + # 2 container hostname + # 3 libvirt mac + def consume_job(boot_type, parameter, value) + RestClient.get "http://#{@host}:#{@port}/boot.#{boot_type}/#{parameter}/#{value}" + end + + def register_mac2host(hostname, mac) + RestClient.put( + "http://#{@host}:#{@port}/set_host_mac?hostname=#{hostname}&mac=#{mac}", {} + ) + end + + def register_host2queues(hostname, queues) + RestClient.put( + "http://#{@host}:#{@port}/set_host2queues?host=#{hostname}&queues=#{queues}", {} + ) + end + + def delete_mac2host(mac) + RestClient.put( + "http://#{@host}:#{@port}/del_host_mac?mac=#{mac}", {} + ) + end + + def delete_host2queues(hostname) + RestClient.put( + "http://#{@host}:#{@port}/del_host2queues?host=#{hostname}", {} + ) + end end -- 2.23.0

1 0

[PATCH lab-z9] add hosts-file of taishan200-2280-2s64p-256g--a5 which include NMVe SSD
by Zhang Dewan 26 Jan '21

26 Jan '21

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- hosts/taishan200-2280-2s64p-256g--a5 | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a5 diff --git a/hosts/taishan200-2280-2s64p-256g--a5 b/hosts/taishan200-2280-2s64p-256g--a5 new file mode 100644 index 0000000..fc9a594 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a5 @@ -0,0 +1,26 @@ +memory: 256G +nr_hdd_partitions: 1 +nr_ssd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399981306c5 +ssd_partitions: + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43957 + - /dev/disk/by-id/nvme-HWE36P43012M000N_032BEJ10GC000328 +mac_addr: + - 84:46:fe:26:d3:47 + - 84:46:fe:26:d3:48 + - 84:46:fe:26:d3:49 + - 84:46:fe:26:d3:4a + - ac:b3:b5:1e:b4:ab + - ac:b3:b5:1e:b4:ac + - ac:b3:b5:1e:b4:ad + - ac:b3:b5:1e:b4:ae + - ac:b3:b5:1e:ba:9b + - ac:b3:b5:1e:ba:9c + - ac:b3:b5:1e:ba:9d + - ac:b3:b5:1e:ba:9e +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.1.5 -- 2.23.0

2 2

[PATCH v5 compass-ci] lib/sched_client.rb: add some common methods for provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

[why] we have some providers: ipxe, container, libvirt all of them need register testbox info and consume job from scheduler Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/sched_client.rb | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/lib/sched_client.rb b/lib/sched_client.rb index fec4b14..a81ffbb 100644 --- a/lib/sched_client.rb +++ b/lib/sched_client.rb @@ -16,4 +16,41 @@ class SchedClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/submit_job") resource.post(job_json) end + + # scheduler API /boot.:boot_type/:parameter/:value + # boot_type parameter + # 1 ipxe mac + # 2 container hostname + # 3 libvirt mac + def consume_job(boot_type, parameter, value) + RestClient.get "http://#{@host}:#{@port}/boot.#{boot_type}/#{parameter}/#{value}" + end + + def register_mac2host(hostname, mac) + @hostname = hostname + @mac = mac + RestClient.put( + "http://#{@host}:#{@port}/set_host_mac?hostname=#{@hostname}&mac=#{@mac}", {} + ) + end + + def register_host2queues(hostname, queues) + @hostname = hostname + @queues = queues + RestClient.put( + "http://#{@host}:#{@port}/set_host2queues?host=#{@hostname}&queues=#{@queues}", {} + ) + end + + def delete_mac2host + RestClient.put( + "http://#{@host}:#{@port}/del_host_mac?mac=#{@mac}", {} + ) + end + + def delete_host2queues + RestClient.put( + "http://#{@host}:#{@port}/del_host2queues?host=#{@hostname}", {} + ) + end end -- 2.23.0

2 2

[PATCH v2 compass-ci] container: expose/report error when check modules exist
by Liu Yinsi 26 Jan '21

26 Jan '21

[why] sparrow/2-network/nfs: modprobe nfs modprobe nfsd sparrow/2-network/cifs: modprobe cifs when locally deploy compass-ci, the above codes have modprobe nfs, nfsd, cifs, no need to do again when execute sparrow/4-docker/buildall(deploy containers), exit and report error message to function callers is better. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 3 ++- container/os-cifs/start | 3 ++- container/os-nfs/start | 6 ++++-- container/result-cifs/start | 3 ++- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 77558a1..c70b318 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm initrd-cifs diff --git a/container/os-cifs/start b/container/os-cifs/start index 5e08238..6df167f 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm os-cifs diff --git a/container/os-nfs/start b/container/os-nfs/start index 8b8ff5f..3b0ebb5 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -5,10 +5,12 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^nfs\s" || { - sudo modprobe nfs + echo "[ERROR] nfs module is not loaded in the Docker host's kernel (try: sudo modprobe nfs)" + exit } lsmod | grep -q "^nfsd\s" || { - sudo modprobe nfsd + echo "[ERROR] nfsd module is not loaded in the Docker host's kernel (try: sudo modprobe nfsd)" + exit } # Several nfs servers running will cause conflict, so rm an old before running a new nfs server docker. diff --git a/container/result-cifs/start b/container/result-cifs/start index 2d6acba..3992cdd 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm result-cifs -- 2.23.0

2 4

[PATCH compass-ci] sparrow/3-code: set params
by Liu Yinsi 26 Jan '21

26 Jan '21

before: deploy compass-ci client and server in one machine. after: not only deploy compass-ci client and server in one machine, but also on different servers. [why] for user locally deploy compass-ci, sometimes need to deploy client in other machine, so add $1. it's interrelated with the follow patch: [PATCH lkp-tests] setup/simplify-ci: delete default value Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 2ba9662..00a2e0f 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -4,6 +4,9 @@ server_name=$(hostname | cut -f1 -d.) server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + +[ -n "$1" ] && server_ip=$1 + : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH v2 lkp-tests] lkp-bootstrap.service: fix killmode error
by Wang Chenglong 26 Jan '21

26 Jan '21

[Error] Welcome to openEuler 20.09! /usr/lib/systemd/system/lkp-bootstrap.service:11: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the ser vice. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. [How] config the lkp-bootstrap.service: KillMode=control-group Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service index 842c4aef8..a9eb5152a 100644 --- a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service +++ b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service @@ -8,7 +8,7 @@ Type=forking Environment=HOME=/root ExecStart=/etc/init.d/lkp-bootstrap IgnoreSIGPIPE=no -KillMode=none +KillMode=control-group TasksMax=infinity [Install] -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: add introduction
by Wu Fengguang 26 Jan '21

26 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index a0b2c82..b5430e7 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -1,5 +1,15 @@ # apply account +You'll need an account to submit jobs. The account is mainly for tracking your resource consumptions. +We have limited pool of testboxes, so there have to be some usage control. + +We offer free accounts to OSS (Open Source Software) contributors. +You can prove yourself as OSS contributor by providing "my oss commit" in the below email. + +We offer free accounts to our collaborators too. If you are one of our +collaborators, you should know the collaboration channel to apply account. +In this case you just need provide name, email, ssh pubkey and purpose. + ## step 1: send email to apply account Email template: @@ -16,8 +26,9 @@ Email template: --- - please replace the 3 parts in {{ }} with your information. -- the email name should be in english, as: "David Rientjes" in the template. -- my oss commit: a git commit URL that has your contribution. +- the email name should be in English or Chinese Pinyin, as: "David Rientjes" in the template. +- my oss commit: a git commit URL that has your contribution. It should be in a + well known git repo that's registered in [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos) ## step 2: receive an email @@ -29,7 +40,7 @@ Which contains account information for you: ## step 3: one-time setup -**Follow the email** to setup your local environment: +**Follow instructions in the email** to setup your local environment: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml @@ -38,10 +49,3 @@ Which contains account information for you: Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -## notice: - -In case that you received a failed email with message as below: - - The repo url for your commit is not in our upstream-repo list. - -Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… -- 2.23.0

1 0

[PATCH compass-ci] uniform coding style
by Li Ping 26 Jan '21

26 Jan '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 12c1f3e..7760c93 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -204,7 +204,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, suite_list, latest_jobs=nil, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suite_list, latest_jobs = nil, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix -- 2.23.0

1 0

[PATCH v3 compass-ci 5/5] upload-libvirt-xml/nginx.conf: nginx.conf in upload-libvirt-xml container
by cuiyili 26 Jan '21

26 Jan '21

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/upload-libvirt-xml/nginx.conf | 43 +++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 container/upload-libvirt-xml/nginx.conf diff --git a/container/upload-libvirt-xml/nginx.conf b/container/upload-libvirt-xml/nginx.conf new file mode 100644 index 0000000..aef8fb8 --- /dev/null +++ b/container/upload-libvirt-xml/nginx.conf @@ -0,0 +1,43 @@ +# For more information on configuration, see: +# * Official English Documentation: http://nginx.org/en/docs/ + +user lkp; +worker_processes auto; +pid /run/nginx.pid; + +events {} + +http { + server { + listen 3081; + server_name "upload-libvirt-xml"; + server_tokens off; + client_max_body_size 50m; + access_log /tmp/access.log; + error_log /tmp/error.log; + + location / { + allow all; + root /srv/cci/libvirt-xml/; + + autoindex on; + create_full_put_path on; + dav_methods PUT MKCOL; + dav_access user:rw group:rw all:rw; + content_by_lua_block { + ngx.req.read_body() + local filename = string.gsub(ngx.var.request_uri, "/", "") + if string.match(filename, "([^/]+).xml$") then + add_path = string.match(filename, "([^/]+).xml$") + local data = ngx.req.get_body_data() + local file = io.open("/srv/cci/libvirt-xml/"..filename, "w") + file:write(data) + file:close() + file_md5 = io.popen("cd /srv/cci/libvirt-xml/ && file_md5=$(md5sum "..filename.." | awk '{print $1}') && mkdir -p "..add_path.." && mv "..filename.." "..add_path.."/${file_md5}.xml && echo ${file_md5}"):read() + ngx.say(add_path.."/"..tostring(file_md5)..".xml") + end + } + } + + } +} -- 2.23.0

3 2

[PATCH v2 compass-ci] container: fix maintainer email address format
by Liu Yinsi 26 Jan '21

26 Jan '21

1. maintainer email address format use: xxx <xxx(a)xxx.com> 2. Update maintainers, shellcheck for Bai Jing <799286817(a)qq.com> web-backend-nginx and web-backend for Lu Weitao <luweitaobe(a)163.com> FYI, Baijing and Weitao, please help to maintain the containers. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/assist-result/Dockerfile | 2 +- container/assistant/Dockerfile | 2 +- container/send-internet-mail/Dockerfile | 2 +- container/send-mail/Dockerfile | 2 +- container/shellcheck/Dockerfile | 2 +- container/web-backend-nginx/Dockerfile | 2 +- container/web-backend/Dockerfile | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/container/assist-result/Dockerfile b/container/assist-result/Dockerfile index 60a2fbf..ce124b9 100644 --- a/container/assist-result/Dockerfile +++ b/container/assist-result/Dockerfile @@ -3,7 +3,7 @@ FROM alpine:3.11 -MAINTAINER caoxl78320(a)163.com +MAINTAINER Cao Xueliang <caoxl78320(a)163.com> RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ apk update && \ diff --git a/container/assistant/Dockerfile b/container/assistant/Dockerfile index f1ad7a2..04d2da0 100644 --- a/container/assistant/Dockerfile +++ b/container/assistant/Dockerfile @@ -3,7 +3,7 @@ FROM debian -MAINTAINER caoxl78320(a)163.com +MAINTAINER Cao Xueliang <caoxl78320(a)163.com> ENV DEBIAN_FRONTEND noninteractive diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile index 03e3327..29f1bc6 100644 --- a/container/send-internet-mail/Dockerfile +++ b/container/send-internet-mail/Dockerfile @@ -2,7 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM debian -MAINTAINER shdluan(a)163.com +MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \ diff --git a/container/send-mail/Dockerfile b/container/send-mail/Dockerfile index d151bda..b334b07 100644 --- a/container/send-mail/Dockerfile +++ b/container/send-mail/Dockerfile @@ -2,7 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM debian -MAINTAINER shdluan(a)163.com +MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \ diff --git a/container/shellcheck/Dockerfile b/container/shellcheck/Dockerfile index b78beb7..0bfe7c0 100644 --- a/container/shellcheck/Dockerfile +++ b/container/shellcheck/Dockerfile @@ -3,7 +3,7 @@ FROM debian -MAINTAINER 18509160991(a)163.com +MAINTAINER Bai Jing <799286817(a)qq.com> ENV DEBIAN_FRONTEND noninteractive diff --git a/container/web-backend-nginx/Dockerfile b/container/web-backend-nginx/Dockerfile index a3882ac..bbf7af2 100644 --- a/container/web-backend-nginx/Dockerfile +++ b/container/web-backend-nginx/Dockerfile @@ -3,6 +3,6 @@ FROM nginx:alpine -MAINTAINER Zhang Yuhang <o0OiIl(a)hotmail.com> +MAINTAINER Lu Weitao <luweitaobe(a)163.com> COPY nginx.conf /etc/nginx/ diff --git a/container/web-backend/Dockerfile b/container/web-backend/Dockerfile index 2ac5148..a26c167 100644 --- a/container/web-backend/Dockerfile +++ b/container/web-backend/Dockerfile @@ -2,7 +2,7 @@ FROM debian -MAINTAINER Zhang Yuhang <o0OiIl(a)hotmail.com> +MAINTAINER Lu Weitao <luweitaobe(a)163.com> ENV DEBIAN_FRONTEND noninteractive -- 2.23.0

1 0