- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib/git_mirror.rb: don't count new ref if it is not new
by Li Yuanchao 09 Mar '21

09 Mar '21

When repos are added, new_refs_count will increase. But many repos are not updated that day, that cause new_refs_count increase much more than real new refs number. Now only count those really new. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 1fa8c9e..f711747 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -203,7 +203,7 @@ class MirrorMain feedback_info[:new_refs] = new_refs send_message(feedback_info) - new_refs_log(git_repo, new_refs[:heads].length) + new_refs_log(git_repo, new_refs[:heads].length) if last_commit_new?(git_repo) end def do_push(fork_key) @@ -336,7 +336,7 @@ class MirrorMain def update_fork_stat(git_repo, possible_new_refs) update_stat_fetch(git_repo) - update_stat_new_refs(git_repo) if possible_new_refs + update_stat_new_refs(git_repo) if possible_new_refs && last_commit_new?(git_repo) es_repo_update(git_repo) end end @@ -530,4 +530,11 @@ class MirrorMain nr_new_branch: nr_new_branch }.to_json) end + + def last_commit_new?(git_repo) + inactive_time = %x(git -C /srv/git/#{@git_info[git_repo]['belong']}/#{git_repo}.git log --pretty=format:"%cr" -1) + return false if inactive_time =~ /(day|week|month|year)/ + + return true + end end -- 2.23.0

1 0

[PATCH compass-ci] lib/lifecycle: persistent restart queue
by Wu Zhende 09 Mar '21

09 Mar '21

After the service restarted, the queue information is not lost. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/lifecycle.cr | 2 +- src/lib/mq.cr | 21 +++++++++++++++------ 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index 16830d0..3dab6e8 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -311,7 +311,7 @@ class Lifecycle return if Time.local < deadline mq_queue = get_machine_reboot_queue(testbox) - @mq.pushlish_confirm(mq_queue, machine.to_json) + @mq.pushlish_confirm(mq_queue, machine.to_json, durable: true) machine["state"] = "rebooting_queue" machine["time"] = Time.local.to_s("%Y-%m-%dT%H:%M:%S+0800") diff --git a/src/lib/mq.cr b/src/lib/mq.cr index 011d732..87c67e9 100644 --- a/src/lib/mq.cr +++ b/src/lib/mq.cr @@ -3,6 +3,7 @@ require "singleton" require "amqp-client" +require "amq-protocol" class MQClient getter ch : AMQP::Client::Channel @@ -19,14 +20,22 @@ class MQClient Singleton::Of(self).instance end - def pushlish_confirm(queue, msg) - q = @ch.queue(queue) - q.publish_confirm msg + def pushlish_confirm(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + q = @ch.queue(queue, passive, durable, exclusive, auto_delete) + if durable + q.publish_confirm(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) + else + q.publish_confirm msg + end end - def pushlish(queue, msg) - q = @ch.queue(queue) - q.publish msg + def pushlish(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + q = @ch.queue(queue, passive, durable, exclusive, auto_delete) + if durable + q.publish(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) + else + q.publish(msg) + end end def get(queue) -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/2] jobs/iso2rootfs.yaml: change some field to secrets field.
by Li Ping 09 Mar '21

09 Mar '21

[why] the related fields when submit test.yaml should be add to secrets field. submit_name: submit_email: submit_token: and the required fields when submit test.yaml not have much to do with the iso2rootfs be changed to level-1 parameter. Signed-off-by: Li Ping <1477412247(a)qq.com> --- jobs/iso2rootfs.yaml | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 55c4345fa..5df2cdf61 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -51,23 +51,24 @@ iso2rootfs: # http://1.1.1.1//dailybuilds/1970-01-01-00-00-00/openEuler-20.03-LTS-aarch64… dailybuild_iso_url_file: - ################# - # submit test yaml related fields - ################# +################# +# submit test yaml related fields +################# - # three required fields when submit test yaml - submit_name: +# three required fields when submit test yaml +secrets: + submit_name: submit_email: submit_token: - # submit target tests yaml - ## 1. You can add as many jobs as you like. - ## 2. The following three fields is required for every test job. - test1_yaml: - test1_os_mount: - test1_testbox: - ## the following three fields is required when your job.yaml and - ## script for this test are from the internet. - test1_git_url: - test1_git_yaml: - test1_git_script: +# submit target tests yaml +## 1. You can add as many jobs as you like. +## 2. The following three fields is required for every test job. +test1_yaml: +test1_os_mount: +test1_testbox: +## the following three fields is required when your job.yaml and +## script for this test are from the internet. +test1_git_url: +test1_git_yaml: +test1_git_script: -- 2.23.0

2 2

[PATCH v2 lkp-tests 2/2] tests/rpmbuild-pkg: delete os_mount field in sync_dest dir.
by Li Ping 09 Mar '21

09 Mar '21

[why] rpm has no much to do with os_mount Signed-off-by: Li Ping <1477412247(a)qq.com> --- tests/rpmbuild-pkg | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 38de16e66..607bb772d 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -6,8 +6,7 @@ [ -n "$rpm_repo" ] || die "rpm_repo is empty" package_name=${rpm_repo##*/} -[[ "$os_mount" = "cifs" ]] && os_mount="nfs" -sync_dest="/initrd/rpmbuild-pkg/${os_mount}/${os}/${os_arch}/${os_version}/${package_name}" +sync_dest="/initrd/rpmbuild-pkg/${os}/${os_arch}/${os_version}/${package_name}" init_workspace() { @@ -25,7 +24,7 @@ download_rpm_repo() { local filelist=$(curl -sS -H 'Content-Type: Application/json' -XPOST "$GIT_SERVER"':8100/git_command' \ -d '{"git_repo": "'${rpm_repo}'", "git_command": ["git-ls-files", "."]}') - for pkgfile in "${filelist[@]}" + for pkgfile in ${filelist[@]} do local dir="SOURCES" echo $pkgfile | egrep "\.spec$" && dir="SPECS" -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] sbin/submit: optimize submit -r
by Li Ping 09 Mar '21

09 Mar '21

[why] submit -m -r iperf.yaml submit /home/liping/lkp-tests/jobs/iperf.yaml, got job id=z9.1183160 submit /home/liping/lkp-tests/jobs/iperf.yaml, got job id=z9.1183161 query=>{"job_id":["z9.1183160","z9.1183161"]} connect to ws://172.168.131.2:11310/filter {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:36.649+0800","job_id":"z9.1183161","message":"","job_state":"submit","result_root":"/srv/result/iperf/2021-03-05/vm-2p8g/openeuler-20.03-aarch64/udp-300/z9.1183161","status_code":200,"method":"POST","resource":"/submit_job","elapsed":"24.68ms"} {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:39.699+0800","job_id":"z9.1183160","result_root":"/srv/result/iperf/2021-03-05/vm-2p8g/openeuler-20.03-aarch64/tcp-300/z9.1183160","job_state":"set result root","status_code":200,"method":"GET","resource":"/boot.ipxe/mac/0a-9f-90-36-ae-6b","elapsed":"8.75ms"} {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:39.735+0800","job_id":"z9.1183160","job_state":"boot","status_code":200,"method":"GET","resource":"/boot.ipxe/mac/0a-9f-90-36-ae-6b","elapsed":"44.08ms"} ==> only get one result [how] get result_root directly from the msg not from fluentd data Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/monitor.rb | 36 ++++++++++-------------------------- sbin/monitor | 8 ++++++++ sbin/submit | 4 +++- 3 files changed, 21 insertions(+), 27 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 78582be6a..24191fa8d 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -11,7 +11,7 @@ require 'eventmachine' require 'json' class Monitor - attr_accessor :monitor_url, :query, :overrides, :action, :job, :result, :stop_query + attr_accessor :monitor_url, :query, :overrides, :action, :job, :result, :result_roots, :stop_query def initialize(monitor_url = '', query = {}, action = {}) @monitor_url = monitor_url @@ -25,9 +25,9 @@ class Monitor @exit_status_code = 0 @defaults = {} @result = [] + @result_roots = [] @stop_query = {} @reason = nil - @result_root = nil end def load_default @@ -95,29 +95,13 @@ class Monitor exec cmd end - def set_result_root(data) - return unless @action['mirror_result'] - return unless data['log'] - - data = JSON.parse(data['log']) - return unless data['result_root'] - - @result_root = data['result_root'] - end - - def lftp_mirror - @result_root.delete_prefix!('/srv') - srv_http_host = job['SRV_HTTP_HOST'] || 'api.compass-ci.openeuler.org' - srv_http_port = job['SRV_HTTP_PORT'] || '11300' - url = "http://#{srv_http_host}:#{srv_http_port}#{@result_root}" - system "lftp -c mirror #{url} >/dev/null 2>&1" - end - - def mirror_result(data) - if @result_root - lftp_mirror - else - set_result_root(data) + def mirror_result + @result_roots.each do |res| + res.to_s.delete_prefix!('/srv') + srv_http_host = job['SRV_HTTP_HOST'] || 'api.compass-ci.openeuler.org' + srv_http_port = job['SRV_HTTP_PORT'] || '11300' + url = "http://#{srv_http_host}:#{srv_http_port}#{res}" + system "lftp -c mirror #{url} >/dev/null 2>&1" end end @@ -163,7 +147,7 @@ class Monitor output(data) connect(data, ws) - mirror_result(data) + mirror_result stop(data, ws) if @action['stop'] end diff --git a/sbin/monitor b/sbin/monitor index def47bf6a..a2e677fc0 100755 --- a/sbin/monitor +++ b/sbin/monitor @@ -12,6 +12,7 @@ require 'yaml' opt_set_key_value = {} opt_action = {'output' => true} opt_job = {} +opt_results = [] opt_monitor_url = nil opt_timeout = 0 opt_stop_query = {} @@ -35,6 +36,12 @@ options = OptionParser.new do |opts| opt_action[action] = true end end + + opts.on('-r result_roots', '--results results', 'set monitor result_roots') do |result_roots| + result_roots.split(',').each do |res| + opt_results << res + end + end opts.separator ' action: output, output log' opts.separator ' action: connect, connect to the host when get ip' opts.separator ' action: stop, exit monitor client when get log' @@ -79,6 +86,7 @@ monitor = Monitor.new monitor.monitor_url = opt_monitor_url if opt_monitor_url monitor.overrides = opt_set_key_value monitor.action = opt_action +monitor.result_roots = opt_results monitor.job = opt_job monitor.stop_query = opt_stop_query diff --git a/sbin/submit b/sbin/submit index e5e41828c..c2af7254d 100755 --- a/sbin/submit +++ b/sbin/submit @@ -19,6 +19,7 @@ opt_monitor = false opt_monitor_query = {} opt_my_queue = false actions = ['output', 'stop'] +result_roots = [] options = OptionParser.new do |opts| opts.banner = 'Usage: submit [options] job1.yaml job2.yaml ...' @@ -199,6 +200,7 @@ ARGV.each do |jobfile| messages = scheduler_client.submit_job(job_json) JSON.parse(messages).each do |msg| if msg['message'].empty? + result_roots << msg['result_root'] job_ids << msg['job_id'].to_s puts("submit #{jobfile}, got job id=#{msg['job_id']}") else @@ -212,6 +214,6 @@ end if opt_monitor job_hash_list[0].delete('define_files') opt_monitor_query.merge!({'job_id' => job_ids}) - cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_hash_list[0].to_json}' -a #{actions.join(',')} -s 'job_state: extract_finished'" + cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_hash_list[0].to_json}' -a #{actions.join(',')} -r #{result_roots.join(',')} -s 'job_state: extract_finished'" exec cmd end -- 2.23.0

1 0

Re: [PATCH lkp-tests 1/2] sbin/makepkg: fix whether to change domain of submodules
by Wu Fengguang 09 Mar '21

09 Mar '21

On Wed, Feb 10, 2021 at 02:36:57PM +0800, Liu Shaofei wrote: >the param "--norevisedomain" indicates that all of submodules will not >be changed domian name, and downloaded from remote end. spell: domian => domain >Signed-off-by: Liu Shaofei <370072077(a)qq.com> >--- > sbin/makepkg | 19 ++++++++++++++----- > 1 file changed, 14 insertions(+), 5 deletions(-) > >diff --git a/sbin/makepkg b/sbin/makepkg >index 7a92ea71b..41e19a513 100755 >--- a/sbin/makepkg >+++ b/sbin/makepkg >@@ -89,6 +89,7 @@ SIGNPKG='' > SPLITPKG=0 > SOURCEONLY=0 > VERIFYSOURCE=0 >+NOREVISEDOMAIN=0 You are adding an option to makepkg. To keep compatibility, if the option is not given, it should do nothing. So I suggest change it to REWRITEGITSERVER=0 And use option name rewritegitserver Thanks, Fengguang > # Forces the pkgver of the current PKGBUILD. Used by the fakeroot call > # when dealing with svn/cvs/etc PKGBUILDs. >@@ -587,6 +588,16 @@ download_git() { > fi > } > >+# modify submodules address to local server address. >+# (( NOREVISEDOMAIN )) is true, not replace submodules domain, and download it from remote end. >+modify_gitmodules() { >+ (( NOREVISEDOMAIN )) && return >+ >+ if [[ -f ".gitmodules" ]] && [[ -n ${GIT_SERVER} ]]; then >+ git config --system url."git://${GIT_SERVER}/".insteadOf https:// >+ fi >+} >+ > extract_git() { > local netfile=$1 > >@@ -623,10 +634,7 @@ extract_git() { > > cd_safe "${dir##*/}" > >- # modify submodules address to local server address. >- if [[ -f ".gitmodules" ]] && [[ -n ${LKP_SERVER} ]]; then >- echo -e "[url \"git://${LKP_SERVER}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig >- fi >+ modify_gitmodules > > local ref=origin/HEAD > if [[ -n $fragment ]]; then >@@ -3462,7 +3470,7 @@ OPT_LONG=('allsource' 'check' 'clean' 'cleanbuild' 'config:' 'force' 'geninteg' > 'help' 'holdver' 'ignorearch' 'install' 'key:' 'log' 'noarchive' 'nobuild' > 'nocolor' 'nocheck' 'nodeps' 'noextract' 'noprepare' 'nosign' 'pkg:' 'repackage' > 'rmdeps' 'sign' 'skipchecksums' 'skipinteg' 'skippgpcheck' 'source' 'syncdeps' >- 'verifysource' 'version') >+ 'verifysource' 'version' 'norevisedomain') > > # Pacman Options > OPT_LONG+=('asdeps' 'noconfirm' 'needed' 'noprogressbar') >@@ -3514,6 +3522,7 @@ while true; do > -s|--syncdeps) DEP_BIN=1 ;; > -S|--source) SOURCEONLY=1 ;; > --verifysource) VERIFYSOURCE=1 ;; >+ --norevisedomain) NOREVISEDOMAIN=1 ;; > > -h|--help) usage; exit 0 ;; # E_OK > -V|--version) version; exit 0 ;; # E_OK >-- >2.23.0 >

2 1

[PATCH compass-ci] sparrow/install-server: use "start-only" instead of "run-only"
by Liu Yinsi 09 Mar '21

09 Mar '21

[why] when locally deploy compass-ci server, no need to check docker images exist, just start container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/install-server b/sparrow/install-server index c532956..f10040a 100755 --- a/sparrow/install-server +++ b/sparrow/install-server @@ -16,5 +16,5 @@ cd $CCI_SRC/sparrow || exit 2-network/cifs 3-code/dev-env . /etc/profile.d/compass.sh -4-docker/buildall run-only +4-docker/buildall start-only 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH compass-ci] sparrow/README.md: delete unnecessary steps
by Liu Yinsi 09 Mar '21

09 Mar '21

1. some steps already finish in sparrow/install-tiny, so delete unnecessary steps. 2. submit iperf.yaml, the default os_mount is container, so change the contents printed by the terminal of the command 3. add FAQ Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/README.md | 64 ++++++++++++++++++++--------------------------- 1 file changed, 27 insertions(+), 37 deletions(-) diff --git a/sparrow/README.md b/sparrow/README.md index 703e8f9..d7b8e81 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -29,26 +29,28 @@ 1. 登录 openEuler 系统 -2. 配置git账号 +2. 设置文件权限 ```bash - git config --global user.name "xxx" - git config --global user.email "xxx(a)xxx.com" - ``` - -3. 设置文件权限并关闭SELINUX - ```bash - umask 002 && setenforce 0 + umask 002 ``` >**说明：** - >setenforce 0 只是暂时禁用SELINUX，需要修改/etc/selinux/config中的SELINUX=enforcing改为SELINUX=permissive或disabled才能长期有效 - >umask 002 只是暂时设置umask值，需要修改/etc/bashrc中的umask值为002才能长期有效 + >umask 002 只是暂时设置umask值，需要修改/etc/bashrc中的umask值为002才能长期有效 + >执行下个步骤前请确保当前umask值为002，否则因文件权限问题将导致部分服务无法正常运行。 -4. 创建工作目录并克隆 compass-ci 项目代码 +3. 创建工作目录并克隆 compass-ci 项目代码 ```bash mkdir /c/ && ln -s /c/compass-ci /c/cci git clone https://gitee.com/wu_fengguang/compass-ci.git ``` + +4. 编辑setup.yaml配置用户名和邮箱 + ```bash + vi /c/compass-ci/sparrow/setup.yaml + ``` + + >**说明：** + >只需填写my_name, my_email, 且my_name, my_email冒号后面必须有1个空格。 5. 执行一键部署脚本 install-tiny ```bash @@ -57,38 +59,18 @@ #### 提交测试任务前的准备 -1. 生成lkp-aarch64.cgz压缩包 - ```bash - cd /c/compass-ci/container/lkp-initrd && ./run - ``` -2. 验证账号 +1. 测试环境是否可以提交job测试 ```bash - cd /c/compass-ci/sbin && ./build-my-info.rb - ``` - -3. 测试环境是否可以提交job测试 - ```bash - submit iperf.yaml testbox=vm-2p8g + submit iperf.yaml ``` 执行上述命令正常情况下会提示信息如下: ``` - submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory - submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory - ``` - compass-ci搭建完毕，执行步骤4下载所需要的rootfs文件就可以开始进行测试了。 - -4. 下载rootfs文件（根据所需要的rootfs在[该目录](http://124.160.11.58:11300/os/)下获取对应版本的cgz文件） - ```bash - mkdir -p /srv/os/openeuler/aarch64/20.03 - cd /srv/os/openeuler/aarch64/20.03 - wget http://api.compass-ci.openeuler.org:11300/os/openeuler/aarch64/20.03.cgz - ``` - -5. 解压rootfs cgz 文件 - ```bash - gzip -dc 20.03.cgz | cpio -idv + submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 + submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 + submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 ``` + compass-ci搭建完毕，下面就可以开始进行测试了。 #### 提交测试任务到本地compass-ci 本文以测试用例iperf.yaml为例 @@ -117,3 +99,11 @@ >**说明：** >[登陆测试环境调测任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + +# FAQ + +* 选择下载os rootfs + + 启动qemu测试机需要[os rootfs文件](http://api.compass-ci.openeuler.org:11304/os/)， + 一键部署默认下载os rootfs为openeuler aarch64 20.03，当部署完毕后，可在该目录下查看: /srv/os/openeuler/aarch64/20.03 + 如需要其他os rootfs，可使用该脚本下载: /c/compass-ci/sbin/download-rootfs -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: add chinese document
by Luan Shengde 09 Mar '21

09 Mar '21

Add chinese document for submit container. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/build-lkp-tests-container.zh.md | 63 ++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 doc/manual/build-lkp-tests-container.zh.md diff --git a/doc/manual/build-lkp-tests-container.zh.md b/doc/manual/build-lkp-tests-container.zh.md new file mode 100644 index 0000000..10a1611 --- /dev/null +++ b/doc/manual/build-lkp-tests-container.zh.md @@ -0,0 +1,63 @@ +# 前言 + +为了适应在各个版本的Linux操作系统上运行，我们提供了lkp-tests的容器化解决方案。 +你无需在您的本地服务上安装lkp-tests，也将避免缺乏依赖包而导致的安装失败。 + +# 前置准备 + + 请在您的本地服务器上完成下面步骤： + + - 安装docker + - 申请账号并配置默认的yaml文件 + - 生成本地的公私钥 + +# 构建容器 + +## 1. 下载资源 + + 本地下载lkp-tests和compass-ci + + 下载资源： + + git clone https://gitee.com/wu_fengguang/compass-ci.git + git clone https://gitee.com/wu_fengguang/lkp-tests.git + +## 2. 添加环境变量 + + 配置环境变量： + + echo "export LKP_SRC=$PWD/lkp-tests" >> ~/.${SHELL##*/}rc + echo "export CCI_SRC=$PWD/compass-ci" >> ~/.${SHELL##*/}rc + source ~/.${SHELL##*/}rc + +## 3. 构建镜像 + + 镜像构建： + + cd compass-ci/container/submit + ./build + +## 4. 添加可执行文件 + + 创建可执行文件： + + ln -s $CCI_SRC/container/submit/submit /usr/bin/submit + +# 开始第一个任务 + + 说明： + + 和在你的本地服务器上安装lkp-tests一样，您可以直接使用‘submit’命令提交任务。 + 每一个任务都会生成一个一次性的容器来执行。 + 可执行文件submit会将您下载的lkp-tests挂载到容器内的lkp-tests路径上，这样您在本地编辑lkp-tests文件，在容器内也会生效。 + + + + 提交任务命令示例： + + + submit -c -m testbox=vm-2p8g borrow-1h.yaml + + submit命令： + + 了解更多submit命令的使用方法，请参考：[submit user manual](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/su… -- 2.23.0

1 0

[PATCH compass-ci] scheduler/pp_params.cr: fix get pp_params
by Wei Jihui 09 Mar '21

09 Mar '21

[error] if pp params only have one value and this value's length is bigger than 40, pp_params will be empty. [how] add all values to pp_params. if one value's length + pp_params's length bigger then 40, then cut value. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- src/scheduler/pp_params.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/scheduler/pp_params.cr b/src/scheduler/pp_params.cr index 03bf0b8..dc24938 100644 --- a/src/scheduler/pp_params.cr +++ b/src/scheduler/pp_params.cr @@ -37,7 +37,8 @@ class Job value = format_string(hash[key].to_s) next if 0 == value.size - if 40 > value.size + items_size + values.size + if items_size < 40 + value = value[0...(40-items_size)] if items_size + value.size > 40 values << value items_size += value.size else -- 2.23.0

2 1

[PATCH v2 compass-ci] /src/extract-stats: rescue when the task-queue disconnect
by Lu Weitao 09 Mar '21

09 Mar '21

[Why] the connection between stats_worker and task-queue may be interrupted, and the old connection will be disable, so stats_worker need create a new connection with task-queue Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/extract-stats/stats_worker.cr | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index 836d1f8..dbb3fe2 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -27,8 +27,11 @@ class StatsWorker end rescue e STDERR.puts e.message + error_message = e.message + # incase of many error message when task-queue, ES does not work sleep(10) + @tq = TaskQueueAPI.new if error_message && error_message.includes?("3060': Connection refused") end end end -- 2.23.0

1 0

[PATCH compass-ci] container/rabbitmq: support data persistence to hard disks
by Wu Zhende 09 Mar '21

09 Mar '21

If data persistence is not performed, all data in the queue will be cleared when the service is restarted. However, some queues need to be restarted to restore data. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/rabbitmq/start | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/container/rabbitmq/start b/container/rabbitmq/start index 6b875f3..cdf5dee 100755 --- a/container/rabbitmq/start +++ b/container/rabbitmq/start @@ -4,16 +4,21 @@ . $CCI_SRC/container/defconfig.sh -docker_rm rabbitmq +docker_name=rabbitmq + +docker_rm $docker_name cmd=( docker run --restart=always -d - --name rabbitmq + --name $docker_name + --hostname $docker_name + -u 1090 -p 5672:5672 -p 15672:15672 -v /etc/localtime:/etc/localtime:ro + -v /srv/rabbitmq:/var/lib/rabbitmq rabbitmq:3-management ) -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: judge local before handle 90lkp
by Xu Xijian 09 Mar '21

09 Mar '21

[why] 90lkp is only used for local mount to replace 90overlay-root, don't use 90lkp with other mount, the errlog occured with cifs. [errlog] Starting dracut pre-pivot and cleanup hook... [ 7.812554] dracut-pre-pivot[626]: cp: cannot create directory '/sysroot/lkp': Read-only file system [ 7.816968] dracut-pre-pivot[626]: cp: cannot create symbolic link '/sysroot/etc/X11/X': Read-only file system Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh b/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh index 8be927c..44f994e 100755 --- a/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh +++ b/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh @@ -3,6 +3,10 @@ . /lib/dracut-lib.sh +if ! getargbool 0 local; then + return +fi + # transfer LKP dirs [ -d /lkp ] || return 0 -- 2.23.0

1 0

[PATCH compass-ci] lib/git_mirror.rb: github download workaround
by Li Yuanchao 09 Mar '21

09 Mar '21

There is a problem to download from github by http or https. fatal: unable to access 'https://github.com/liyc-github/AvxToNeon/': Failed to connect to github.com port 443: Connection timed out When it occurs, change url to "git://xxx". Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 53c9e59..1fa8c9e 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -42,16 +42,20 @@ class GitMirror def git_clone(url, mirror_dir) url = get_url(Array(url)[0]) 10.times do - %x(git clone --mirror --depth 1 #{url} #{mirror_dir} 2>&1) + stderr = %x(git clone --mirror --depth 1 #{url} #{mirror_dir} 2>&1) return true if File.directory?(mirror_dir) && File.exist?("#{mirror_dir}/config") + + if stderr.include?('Failed to connect to github.com port 443') + url = "git://#{url.split('://')[1]}" + end end return false end def git_fetch(mirror_dir) if File.exist?("#{mirror_dir}/shallow") + FileUtils.rm("#{mirror_dir}/shallow.lock") if File.exist?("#{mirror_dir}/shallow.lock") %x(git -C #{mirror_dir} fetch --unshallow 2>&1) - return false end fetch_info = %x(git -C #{mirror_dir} fetch 2>&1) @@ -59,6 +63,11 @@ class GitMirror if fetch_info.include?(ERR_MESSAGE) && Dir.empty?(mirror_dir) FileUtils.rmdir(mirror_dir) end + + if fetch_info.include?('Failed to connect to github.com port 443') + url = %x(git -C #{mirror_dir} ls-remote --get-url origin).chomp + %x(git -C #{mirror_dir} remote set-url origin git://#{url.split('://')[1]}) + end return fetch_info.include? '->' end @@ -66,7 +75,7 @@ class GitMirror url = get_url(Array(url)[0]) git_url = %x(git -C #{mirror_dir} ls-remote --get-url origin).chomp - return true if url == git_url + return true if git_url == url || git_url.include?(url.split('://')[1]) return false end -- 2.23.0

2 1

[PATCH lkp-tests] sbin/submit: fix input docker_image value by cmdline
by Wei Jihui 09 Mar '21

09 Mar '21

[error] input: submit docker_image=centos:7 job.yaml output: docker_image: centos [why] submit split centos:7 to [centos, 7] [how] -s "docker_image: centos:7" ==> docker_image: centos:7 docker_image=centos:7 ==> docker_image: centos:7 Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- sbin/submit | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sbin/submit b/sbin/submit index e5e41828..feb65d00 100755 --- a/sbin/submit +++ b/sbin/submit @@ -28,7 +28,7 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on("-s 'KEY: VALUE'", "--set 'KEY: VALUE'", 'add YAML hash to job') do |key_value| - k, v = key_value.sub(' ', '').split(':') + k, v = key_value.sub(' ', '').split(':', 2) opt_set_key_value[k] = v end @@ -59,7 +59,7 @@ options = OptionParser.new do |opts| opts.on('-m', '--monitor', "monitor job status: use -m 'KEY: VALUE' to add rule") do opt_monitor = true - k, v = ARGV[0].sub(' ', '').split(':') if ARGV[0] + k, v = ARGV[0].sub(' ', '').split(':', 2) if ARGV[0] if (k && !k.empty?) && (v && !v.empty?) opt_monitor_query[k] = v ARGV.shift @@ -78,7 +78,7 @@ options.parse!(ARGV) seen_job_yaml = false ARGV.delete_if do |arg| if arg.index '=' - k, v = arg.sub(' ', '').sub(/=/, ':').split(':') + k, v = arg.sub(' ', '').split('=', 2) if seen_job_yaml opt_set_key_value[k] = v else -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow/install-tiny: call 5-build/register-account
by Liu Yinsi 09 Mar '21

09 Mar '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-tiny | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/install-tiny b/sparrow/install-tiny index 9e94e39..e48bd25 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -21,4 +21,5 @@ cd $CCI_SRC/sparrow || exit 4-docker/buildall 5-build/ipxe 5-build/os-ready +5-build/register-account 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow/install-tiny: call 0-package/read-config
by Liu Yinsi 09 Mar '21

09 Mar '21

for users locally deploy compass-ci, call 0-package/read-config, export my_name, my_email to register account. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-tiny | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/install-tiny b/sparrow/install-tiny index 5f026e5..9e94e39 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -7,6 +7,7 @@ [[ $CCI_SRC ]] || export CCI_SRC=$(cd $(dirname $(realpath $0)); git rev-parse --show-toplevel) cd $CCI_SRC/sparrow || exit +. 0-package/read-config 0-package/install 1-storage/tiny 1-storage/permission -- 2.23.0

1 0

[PATCH v3 lkp-tests] setup/simplify-ci: call $CCI_SRC/sparrow/install-client
by Liu Yinsi 09 Mar '21

09 Mar '21

[why] for simplicy-ci job test, if there is $SCHED_HOST, and export server_ip=$SCHED_HOST, when call $CCI_SRC/sparrow/install-client, config sparrow/setup.yaml and execute sparrow/0-package/read-config(export server_ip) is unnecessary. for locally deploy compass-ci client, must config sparrow/setup.yaml, and execute sparrow/0-package/read-config(export server_ip), if server_ip not exists, will call log_error() and exit. [how] in $LKP_SRC/setup/simplify-ci, just call $CCI_SRC/sparrow/install-client script, make code more reusablely. refer patch: Subject: [PATCH v5 compass-ci 1/2] sparrow/install-client: add deploy compass-ci client script Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- setup/simplify-ci | 61 ++++------------------------------------------- 1 file changed, 5 insertions(+), 56 deletions(-) diff --git a/setup/simplify-ci b/setup/simplify-ci index 5aede9cdb..83e20b5f8 100755 --- a/setup/simplify-ci +++ b/setup/simplify-ci @@ -2,8 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${SCHED_HOST:=172.17.0.1} -: ${SCHED_PORT:=3000} +export server_ip=$SCHED_HOST git_ci() { @@ -17,61 +16,11 @@ git_ci() git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 } -dev_env() +deploy() { - export sched_host=$SCHED_HOST - export sched_port=$SCHED_PORT - 3-code/dev-env + cd /c/compass-ci/sparrow && ./install-client } -install_env() -{ - cd /c/compass-ci/sparrow || return - 0-package/install - 1-storage/tiny - 5-build/ipxe & - 1-storage/permission - 2-network/br0 - 2-network/iptables - 3-code/git - dev_env - . /etc/profile.d/compass.sh -} - -boot_ipxe() -{ - sed -i "s%172.17.0.1%$SCHED_HOST%g" /tftpboot/boot.ipxe - sed -i "s%3000%$SCHED_PORT%g" /tftpboot/boot.ipxe -} - -run_service() -{ - ( - cd $CCI_SRC/container/dnsmasq || return - ./build - ./start - boot_ipxe - )& - ( - cd $CCI_SRC/container/qemu-efi || return - ./build - ./install - )& - ( - cd $CCI_SRC/container/fluentd-base || return - ./build - cd $CCI_SRC/container/sub-fluentd || return - ./build - ./start - )& -} - -main() -{ - git_ci || return 1 - install_env - run_service -} +git_ci || exit 1 +deploy -main -wait -- 2.23.0

1 0

[PATCH lkp-tests] tests/rpmbuild-pkg: delete os_mount field in sync_dest dir.
by Li Ping 09 Mar '21

09 Mar '21

[why] rpm has no much to do with os_mount Signed-off-by: Li Ping <1477412247(a)qq.com> --- tests/rpmbuild-pkg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 38de16e66..b641e97d1 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -6,8 +6,7 @@ [ -n "$rpm_repo" ] || die "rpm_repo is empty" package_name=${rpm_repo##*/} -[[ "$os_mount" = "cifs" ]] && os_mount="nfs" -sync_dest="/initrd/rpmbuild-pkg/${os_mount}/${os}/${os_arch}/${os_version}/${package_name}" +sync_dest="/initrd/rpmbuild-pkg/${os}/${os_arch}/${os_version}/${package_name}" init_workspace() { -- 2.23.0

1 0

[PATCH v2 compass-ci] src/lib/job.cr: optimize the get_pkg_common_dir function
by Li Ping 09 Mar '21

09 Mar '21

[why] 1. rpm has no much to do with os_mount. add a judgment 2. optimize the code style Signed-off-by: Li Ping <1477412247(a)qq.com> --- src/lib/job.cr | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0b4bee4..fe1b7c1 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -258,23 +258,28 @@ class Job end def get_pkg_common_dir - pkg_style = nil + pkg_style = Hash(String, JSON::Any).new {|h, k| h[k] = JSON::Any.new(nil)} ["cci-makepkg", "cci-depends", "build-pkg", "rpmbuild-pkg"].each do |item| - pkg_style = @hash[item]? - break if pkg_style - end - return nil unless pkg_style + next unless @hash[item]? - pkg_style = JSON.parse("{}") if pkg_style == nil + pkg_style.merge!((a)hash[item].as_h) + break + end + return nil if pkg_style.empty? - tmp_os = pkg_style["os"]? == nil ? "#{os}" : pkg_style["os"] - tmp_os_arch = pkg_style["os_arch"]? == nil ? "#{os_arch}" : pkg_style["os_arch"] - tmp_os_version = pkg_style["os_version"]? == nil ? "#{os_version}" : pkg_style["os_version"] + tmp_os = pkg_style["os"].as_s? || "#{os}" + tmp_os_arch = pkg_style["os_arch"].as_s? || "#{os_arch}" + tmp_os_version = pkg_style["os_version"].as_s? || "#{os_version}" - tmp_os_mount = pkg_style["os_mount"]? == nil ? "#{os_mount}" : pkg_style["os_mount"] - mount_type = tmp_os_mount == "cifs" ? "nfs" : tmp_os_mount.dup + mount_type = pkg_style["os_mount"].as_s? || "#{os_mount}" + # same usage for client + mount_type = "nfs" if mount_type == "cifs" - common_dir = "#{mount_type}/#{tmp_os}/#{tmp_os_arch}/#{tmp_os_version}" + if @hash["rpmbuild-pkg"]? + common_dir = "#{tmp_os}/#{tmp_os_arch}/#{tmp_os_version}" + else + common_dir = "#{mount_type}/#{tmp_os}/#{tmp_os_arch}/#{tmp_os_version}" + end return common_dir end -- 2.23.0

1 0

[PATCH v3 lkp-tests 2/2] sbin/submit: optimize submit -r
by Li Ping 09 Mar '21

09 Mar '21

[why] submit -m -r iperf.yaml submit /home/liping/lkp-tests/jobs/iperf.yaml, got job id=z9.1183160 submit /home/liping/lkp-tests/jobs/iperf.yaml, got job id=z9.1183161 query=>{"job_id":["z9.1183160","z9.1183161"]} connect to ws://172.168.131.2:11310/filter {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:36.649+0800","job_id":"z9.1183161","message":"","job_state":"submit","result_root":"/srv/result/iperf/2021-03-05/vm-2p8g/openeuler-20.03-aarch64/udp-300/z9.1183161","status_code":200,"method":"POST","resource":"/submit_job","elapsed":"24.68ms"} {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:39.699+0800","job_id":"z9.1183160","result_root":"/srv/result/iperf/2021-03-05/vm-2p8g/openeuler-20.03-aarch64/tcp-300/z9.1183160","job_state":"set result root","status_code":200,"method":"GET","resource":"/boot.ipxe/mac/0a-9f-90-36-ae-6b","elapsed":"8.75ms"} {"level_num":1,"level":"INFO","time":"2021-03-05T11:31:39.735+0800","job_id":"z9.1183160","job_state":"boot","status_code":200,"method":"GET","resource":"/boot.ipxe/mac/0a-9f-90-36-ae-6b","elapsed":"44.08ms"} ==> only get one result [how] get result_root directly from the msg not from fluentd data Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/monitor.rb | 36 ++++++++++-------------------------- sbin/monitor | 8 ++++++++ sbin/submit | 4 +++- 3 files changed, 21 insertions(+), 27 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 78582be6a..24191fa8d 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -11,7 +11,7 @@ require 'eventmachine' require 'json' class Monitor - attr_accessor :monitor_url, :query, :overrides, :action, :job, :result, :stop_query + attr_accessor :monitor_url, :query, :overrides, :action, :job, :result, :result_roots, :stop_query def initialize(monitor_url = '', query = {}, action = {}) @monitor_url = monitor_url @@ -25,9 +25,9 @@ class Monitor @exit_status_code = 0 @defaults = {} @result = [] + @result_roots = [] @stop_query = {} @reason = nil - @result_root = nil end def load_default @@ -95,29 +95,13 @@ class Monitor exec cmd end - def set_result_root(data) - return unless @action['mirror_result'] - return unless data['log'] - - data = JSON.parse(data['log']) - return unless data['result_root'] - - @result_root = data['result_root'] - end - - def lftp_mirror - @result_root.delete_prefix!('/srv') - srv_http_host = job['SRV_HTTP_HOST'] || 'api.compass-ci.openeuler.org' - srv_http_port = job['SRV_HTTP_PORT'] || '11300' - url = "http://#{srv_http_host}:#{srv_http_port}#{@result_root}" - system "lftp -c mirror #{url} >/dev/null 2>&1" - end - - def mirror_result(data) - if @result_root - lftp_mirror - else - set_result_root(data) + def mirror_result + @result_roots.each do |res| + res.to_s.delete_prefix!('/srv') + srv_http_host = job['SRV_HTTP_HOST'] || 'api.compass-ci.openeuler.org' + srv_http_port = job['SRV_HTTP_PORT'] || '11300' + url = "http://#{srv_http_host}:#{srv_http_port}#{res}" + system "lftp -c mirror #{url} >/dev/null 2>&1" end end @@ -163,7 +147,7 @@ class Monitor output(data) connect(data, ws) - mirror_result(data) + mirror_result stop(data, ws) if @action['stop'] end diff --git a/sbin/monitor b/sbin/monitor index def47bf6a..a2e677fc0 100755 --- a/sbin/monitor +++ b/sbin/monitor @@ -12,6 +12,7 @@ require 'yaml' opt_set_key_value = {} opt_action = {'output' => true} opt_job = {} +opt_results = [] opt_monitor_url = nil opt_timeout = 0 opt_stop_query = {} @@ -35,6 +36,12 @@ options = OptionParser.new do |opts| opt_action[action] = true end end + + opts.on('-r result_roots', '--results results', 'set monitor result_roots') do |result_roots| + result_roots.split(',').each do |res| + opt_results << res + end + end opts.separator ' action: output, output log' opts.separator ' action: connect, connect to the host when get ip' opts.separator ' action: stop, exit monitor client when get log' @@ -79,6 +86,7 @@ monitor = Monitor.new monitor.monitor_url = opt_monitor_url if opt_monitor_url monitor.overrides = opt_set_key_value monitor.action = opt_action +monitor.result_roots = opt_results monitor.job = opt_job monitor.stop_query = opt_stop_query diff --git a/sbin/submit b/sbin/submit index a6f0cd387..44fd0a7db 100755 --- a/sbin/submit +++ b/sbin/submit @@ -19,6 +19,7 @@ opt_monitor = false opt_monitor_query = {} opt_my_queue = false actions = ['output', 'stop'] +result_roots = [] options = OptionParser.new do |opts| opts.banner = 'Usage: submit [options] job1.yaml job2.yaml ...' @@ -197,6 +198,7 @@ ARGV.each do |jobfile| messages = scheduler_client.submit_job(job_json) JSON.parse(messages).each do |msg| if msg['message'].empty? + result_roots << msg['result_root'] job_ids << msg['job_id'].to_s puts("submit #{jobfile}, got job id=#{msg['job_id']}") else @@ -210,6 +212,6 @@ end if opt_monitor job_hash_list[0].delete('define_files') opt_monitor_query.merge!({'job_id' => job_ids}) - cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_hash_list[0].to_json}' -a #{actions.join(',')} -s 'job_state: extract_finished'" + cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_hash_list[0].to_json}' -a #{actions.join(',')} -r #{result_roots.join(',')} -s 'job_state: extract_finished'" exec cmd end -- 2.23.0

1 0

[PATCH compass-ci] sbin/es-jobs-mapping.sh: add my_email field
by Lu Weitao 09 Mar '21

09 Mar '21

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/es-jobs-mapping.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sbin/es-jobs-mapping.sh b/sbin/es-jobs-mapping.sh index 9c9f7b6..edc6159 100755 --- a/sbin/es-jobs-mapping.sh +++ b/sbin/es-jobs-mapping.sh @@ -140,6 +140,9 @@ else "upstream_commit": { "type": "keyword" }, + "my_email": { + "type": "keyword" + }, "enqueue_time": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss" -- 2.23.0

1 0

[PATCH v6 compass-ci] doc/manual: add document for submit container
by Luan Shengde 09 Mar '21

09 Mar '21

Add English document for submit container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/build-lkp-tests-container.en.md | 63 ++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 doc/manual/build-lkp-tests-container.en.md diff --git a/doc/manual/build-lkp-tests-container.en.md b/doc/manual/build-lkp-tests-container.en.md new file mode 100644 index 0000000..53f6887 --- /dev/null +++ b/doc/manual/build-lkp-tests-container.en.md @@ -0,0 +1,63 @@ +# Preface + +We provide a docker container to suit various of Linux OS(es). +In this case you do not need to install the lkp-tests to your local server. +Also you can avoid installation failures from undesired dependency package(s). + +# Prepare + + Do on your local server: + + - install docker + - apply account and config default yaml + - generate ssh key(s) + +# build container + +## 1. download resource + + Download lkp-tests and compass-ci to your local server. + + Command(s): + + git clone https://gitee.com/wu_fengguang/compass-ci.git + git clone https://gitee.com/wu_fengguang/lkp-tests.git + +## 2. add environment variable(s) + + Command(s): + + echo "export LKP_SRC=$PWD/lkp-tests" >> ~/.${SHELL##*/}rc + echo "export CCI_SRC=$PWD/compass-ci" >> ~/.${SHELL##*/}rc + source ~/.${SHELL##*/}rc + +## 3. build docker image + + Command(s): + + cd compass-ci/container/submit + ./build + +## 4. add executable file + + Command(s): + + ln -s $CCI_SRC/container/submit/submit /usr/bin/submit + +# try it + + instruction: + + You can directly use the command 'submit' to submit jobs. + It is the same as you install the lkp-tests on your own server. + It will start a disposable container to submit your job. + The container will attach the directory lkp-tests to the container itself. + You can edit the job yaml(s) in lkp-tests/jobs and it will take effect when you submit jobs. + + Example: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml + + About submit: + + For detailed usage for command submit, please reference to: [submit user manual](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/su… -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/mail-robot: fix nil error
by Luan Shengde 09 Mar '21

09 Mar '21

[why] When parsing user_info from the mail content, user_info may be nil if it's empty or a empty line. It may throw 'has no include? method for nil' error when use user_info.include? Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/parse-apply-account-email.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index 272f1a1..e6c35f7 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -63,8 +63,10 @@ class ParseApplyAccountEmail users.delete('') users.each do |user| user_info = YAML.safe_load(user) + next if user_info.nil? + next unless user_info.include?('my_email') - users_info << user_info if user_info.include?('my_email') + users_info << user_info end return users_info -- 2.23.0

1 0

[PATCH compass-ci 1/2] container/mail-robot: fix def invoke error
by Luan Shengde 09 Mar '21

09 Mar '21

[why] the function name like: send_mail(error_message, account_info, assign_account_vm) but the invoking method is like: send_mail(e.message) Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb index f3f61c3..17705ec 100755 --- a/container/mail-robot/lib/apply-account.rb +++ b/container/mail-robot/lib/apply-account.rb @@ -100,7 +100,7 @@ class ApplyAccount puts e.message puts e.backtrace - send_mail(e.message) + send_mail(e.message, '', '') end def forward_users -- 2.23.0

1 0