- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] lib/job.rb: optimize load config
by Wei Jihui 03 Feb '21

03 Feb '21

[why] current load config order can not load physical machine [how] optimize load config for: 1. load /etc/compass-ci/defaults/*.yaml load ~/.config/compass-ci/defaults/*.yaml --> get lab 2. load ~/.config/compass-ci/include/lab/$lab.yaml -->get my_token 3. now user get testbox, load load hosts config 4. load other file: like lkp-tests/include/* Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 477a8a12..9174cadc 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -367,26 +367,15 @@ class Job return @job.include?('tbox_group') end - def get_lab_hosts_file - lab_hosts_file_name = @job['testbox'] - lab_hosts_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{lab_hosts_file_name}" - - if File.file?(lab_hosts_file) - lab_hosts_file - else - nil - end - end - def get_hosts_file - lab_hosts_file = get_lab_hosts_file - return lab_hosts_file if lab_hosts_file + testbox_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{@job['testbox']}" + return testbox_file if File.file?(testbox_file) - hosts_file_name = @job['tbox_group'].split(/\.|--/)[0] - hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" - return hosts_file if File.file?(hosts_file) + tbox_group = @job['tbox_group'].split(/\.|--/)[0] + tbox_group_file = "#{LKP_SRC}/hosts/#{tbox_group}" + return tbox_group_file if File.file?(tbox_group_file) - raise ArgumentError, "hosts file not exist: #{hosts_file_name}, maybe need check testbox field" + raise ArgumentError, "hosts file not exist: #{tbox_group}, maybe need check testbox field" end def include_files @@ -438,6 +427,9 @@ class Job lab_yaml = File.join(self_config_path, 'include/lab', "#{@defaults['lab']}.yaml") load_one_defaults(lab_yaml, @job) + + merge_defaults + load_hosts_config end def load_defaults(first_time = true) @@ -749,7 +741,6 @@ class Job each_job_init job = deepcopy(@job) load_defaults - load_hosts_config each_job_init each_job(&block) @jobs.each do |hash| @@ -757,7 +748,6 @@ class Job @job = deepcopy(job) @job.merge!(hash) load_defaults - load_hosts_config each_job_init each_job(&block) end -- 2.23.0

1 0

[PATCH v3 compass-ci] delimiter/utils.rb: fix get the wrong job id when submit job failed
by Cao Xueliang 03 Feb '21

03 Feb '21

If we submit job failed, we get the response "xxx job id=0, error: xxx". We should match the response at first and return nil, otherwise we will get the wrong job id. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41b65de..3120ca6 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -69,6 +69,7 @@ module Utils save_job_to_yaml(job, PROCESS_JOB_YAML) response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" + return nil if response =~ /job id=0/ return $1 if response =~ /job id=(.*)/ return nil -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

2 2

[PATCH v2 lkp-tests] installer/oracle: add shebang
by Wang Yong 03 Feb '21

03 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/oracle | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/installer/oracle b/distro/installer/oracle index b09abc945..11df716d6 100755 --- a/distro/installer/oracle +++ b/distro/installer/oracle @@ -1,3 +1,5 @@ +#!/bin/sh + if ! rpm -q epel-release >/dev/null; then yum install -y epel-release yum makecache fast -- 2.23.0

1 0

[PATCH lkp-tests] installer/oracle: add shebang
by Wang Yong 03 Feb '21

03 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/oracle | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/installer/oracle b/distro/installer/oracle index b09abc945..0831a5b6c 100755 --- a/distro/installer/oracle +++ b/distro/installer/oracle @@ -1,3 +1,5 @@ +#!/bin/bash + if ! rpm -q epel-release >/dev/null; then yum install -y epel-release yum makecache fast -- 2.23.0

2 2

[PATCH compass-ci] dnsmasq: open the dns function
by Yu Chuan 03 Feb '21

03 Feb '21

[Why] Generally, dnsmasq is used for a internal network lab, and as the number of machines in the lab increases, we need an internal dns server to perform dns relay and forward function. [How] 1. open dns function: - set 'port=53'; 2. set the upstream dns server of dnsmasq: - add param: 'resolv-conf'; - add resolv.conf and mount it to container. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/conf/etc/resolv.conf | 1 + container/dnsmasq/dnsmasq.d/dnsmasq.conf | 7 ++++++- container/dnsmasq/start | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 container/dnsmasq/conf/etc/resolv.conf diff --git a/container/dnsmasq/conf/etc/resolv.conf b/container/dnsmasq/conf/etc/resolv.conf new file mode 100644 index 000000000000..bad8aa25de25 --- /dev/null +++ b/container/dnsmasq/conf/etc/resolv.conf @@ -0,0 +1 @@ +nameserver 114.114.114.114 diff --git a/container/dnsmasq/dnsmasq.d/dnsmasq.conf b/container/dnsmasq/dnsmasq.d/dnsmasq.conf index 6141faf0436b..877216d8abdb 100644 --- a/container/dnsmasq/dnsmasq.d/dnsmasq.conf +++ b/container/dnsmasq/dnsmasq.d/dnsmasq.conf @@ -1,4 +1,9 @@ -port=0 +# open dns function +port=53 + +# setup the upstream dns server of dnsmasq +resolv-file=/etc/resolv.conf + interface=br0 enable-tftp diff --git a/container/dnsmasq/start b/container/dnsmasq/start index ff4247046eb1..a75ff209e64f 100755 --- a/container/dnsmasq/start +++ b/container/dnsmasq/start @@ -27,6 +27,7 @@ cmd=( --publish 67:67/udp --publish 69:69/udp -v ${WORKSPACE}/dnsmasq.d:/etc/dnsmasq.d + -v ${WORKSPACE}/conf/etc/resolv.conf:/etc/resolv.conf:ro # specify the upstream dns server of dnsmasq -v ${DNSMASQ_CONF_DIR}/lease_dir:/var/lib/misc # persist the dhcp lease file -v /etc/localtime:/etc/localtime:ro -v /tftpboot:/tftpboot:ro -- 2.23.0

1 0

[PATCH v2 compass-ci] delimiter/utils.rb: deal submit job failed
by Cao Xueliang 03 Feb '21

03 Feb '21

If we submit job failed, we get the response "xxx job id=0, error: xxx", we should match the response at first and return nil. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41b65de..3120ca6 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -69,6 +69,7 @@ module Utils save_job_to_yaml(job, PROCESS_JOB_YAML) response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" + return nil if response =~ /job id=0/ return $1 if response =~ /job id=(.*)/ return nil -- 2.23.0

1 0

[PATCH v3 compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty before git clone. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..507c4fe 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ -d "openresty" ] && rm -rf openresty + git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

1 0

[PATCH v5 compass-ci 4/4] container/submit: README file
by Luan Shengde 03 Feb '21

03 Feb '21

help file for the container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 container/submit/README.md diff --git a/container/submit/README.md b/container/submit/README.md new file mode 100644 index 0000000..6772961 --- /dev/null +++ b/container/submit/README.md @@ -0,0 +1,26 @@ +# Install lkp-tests and its depends in container. +# With docker container, will suit most of the linux OSes. +# Avoid much repetitive testing work to adapt various OSes. + +Prepare work: + apply account + config default yaml + generate ssh keys + +Installation: + git clone https://gitee.com/wu_fengguang/compass-ci.git + + cd compass-ci/container/submit + ./build + + Add soft link for executable file 'submit' to $PATH: + + ln -s $PWD/submit /usr/bin/submit + +Usage: + You can directly use the command 'submit' to submit jobs. + It's the same as you install the lkp-tests at your local host. + + Example: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v5 compass-ci 3/4] container/submit: executable file submit
by Luan Shengde 03 Feb '21

03 Feb '21

executable file for container: submit file 'submit' is used as the command installed in the container. it can be run as the real command submit itself in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..4e7e24f --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M%S) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /root/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0