- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 1/4] lib/git_bisect: use "--first-parent" option when start bisect
by Cao Xueliang 19 Jan '21

19 Jan '21

1. The alpine 3.13 support to use the latest git which has the "--first-parent" option. 2. This option is useful in avoiding false positives when a merged branch contained broken or non-buildable commits, but the merge itself was OK. Ref: https://git-scm.com/docs/git-bisect Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/delimiter/Dockerfile | 2 +- lib/git_bisect.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/container/delimiter/Dockerfile b/container/delimiter/Dockerfile index ab7519d..c4e5074 100644 --- a/container/delimiter/Dockerfile +++ b/container/delimiter/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:3.13 RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ apk update && \ diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 2fee60c..e577afd 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -82,7 +82,7 @@ class GitBisect puts "bad_commit: #{@bad_commit}" puts "good_commit: #{@good_commit}" - result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` + result = `git -C #{@work_dir} bisect start --first-parent #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: optimize borrow-machine.zh.md
by Luan Shengde 19 Jan '21

19 Jan '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/borrow-machine.zh.md | 288 +++++++------------------------- 1 file changed, 59 insertions(+), 229 deletions(-) diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 916aea7..e4c15f2 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -1,243 +1,73 @@ -# 前提条件 - -请确认您已按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… -- send apply account email. -- receive email from compass-ci-robot(a)qq.com. -- local environment configuration. - - -# 申请测试机（虚拟机） - -1. 生成本地RSA公私钥对 - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. 根据需求选择yaml - - 每位用户`/home/${USER}`目录下面都存放了一个 lkp-tests 的文件夹。 - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. 提交yaml并连接测试机（虚拟机） - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170593 - query=>{"job_id":["z9.170593"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170593","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593"} - {"job_id": "z9.170593", "result_root": "/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593", "job_state": "set result root"} - {"job_id": "z9.170593", "job_state": "boot"} - {"job_id": "z9.170593", "job_state": "download"} - {"time":"2020-11-26 14:45:06","mac":"0a-1f-0d-3c-91-5c","ip":"172.18.156.13","job_id":"z9.170593","state":"running","testbox":"vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - {"job_state":"running","job_id":"z9.170593"} - {"job_id": "z9.170593", "state": "set ssh port", "ssh_port": "51840", "tbox_name": "vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:51840' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Thu Nov 26 06:44:18 CST 2020 - - System load: 0.83 - Processes: 107 - Memory used: 6.1% - Swap used: 0.0% - Usage On: 89% - IP address: 172.18.156.13 - Users online: 1 - - - - root@vm-2p8g ~# - ``` - - 更多关于`submit命令如何使用`、`testbox都有什么可选项`、`如何borrow指定的操作系统`，请参见文章末尾FAQ。 - -4. 使用完毕退还测试机（虚拟机） - - ```shell - root@vm-2p8g ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` - - -# 申请测试机（物理机） - - -1. 生成本地RSA公私钥对 - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. 根据需求选择yaml - - 每位用户`/home/${USER}`目录下面都存放了一个 lkp-tests 的文件夹。 - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. 提交yaml并连接测试机（物理机） - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=taishan200-2280-2s64p-256g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170594 - query=>{"job_id":["z9.170594"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170594","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594"} - {"job_id": "z9.170594", "result_root": "/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594", "job_state": "set result root"} - {"job_id": "z9.170594", "job_state": "boot"} - {"job_id": "z9.170594", "job_state": "download"} - {"time":"2020-11-26 14:51:56","mac":"84-46-fe-26-d3-47","ip":"172.168.178.48","job_id":"z9.170594","state":"running","testbox":"taishan200-2280-2s64p-256g--a5"} - {"job_state":"running","job_id":"z9.170594"} - {"job_id": "z9.170594", "state": "set ssh port", "ssh_port": "50420", "tbox_name": "taishan200-2280-2s64p-256g--a5"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:50420' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 +# 前置准备 +- 申请account +- 配置默认yaml文件 - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 +如果未完成以上步骤，请参考[apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.md)进行操作。 - System information as of time: Thu Nov 26 14:51:59 CST 2020 +# 申请测试机 - System load: 1.31 - Processes: 1020 - Memory used: 5.1% - Swap used: 0.0% - Usage On: 3% - IP address: 172.168.178.48 - Users online: 1 - - - - root@taishan200-2280-2s64p-256g--a5 ~# - ``` - - 更多关于`submit命令如何使用`、`testbox都有什么可选项`、`如何borrow指定的操作系统`，请参见文章末尾FAQ。 - -4. 使用完毕退还测试机（物理机） - - ```shell - root@taishan200-2280-2s64p-256g--a5 ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` +## 1. 生成本地RSA公/私钥对 + 使用下面命令查看是否已存在公/私钥： + + ls -a ~/.ssh + + 如果没有现成的公/私钥对，请使用下面命令进行生成： + + ssh-keygen -r rsa + +## 2. 选择job yaml + + 在 ~/lkp-tests/jobs 目录下未您提供了众多job yaml模板。 + + 使用以下命令可筛选出借机器的模板： + + cd ~/lkp-tests/jobs + ls -a borrow* + + - 检查您选择的模板中引用的公钥地址和文件名和你本地的公钥一致。 + +## 3. 提交job + + 使用以下命令提交job： + + submit -c -m testbox=vm-2p8g borrow-1h.yaml + + - 使用该命令，您可以实时查看job状态，测试机正常运行后直接登入。 + +## 4. 退还测试机 + + 到期自动退还： + + 借用期限到达后，测试接将自动重启归还。 + + 手动退还： + + 执行‘reboot’命令进行归还。 + + - 所有测试机在执行‘reboot’命令后都会被归还。 + - 机器归还后，如果您还需要继续借用，请重新提交job进行申请。 # FAQ +## 自定义借用时长 + + 在借用机器的模板中，找到’runtime‘字段，根据需求修改借用时长。 + +## submit命令指导 + + [submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + +## 可选的testbox -* 如何自行修改申请时长 - - ```shell - hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml - suite: borrow - testcase: borrow - - runtime: 1h - ssh_pub_key: <%= - begin - File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp - rescue - nil - end - %> - sshd: - # sleep at the bottom - sleep: - hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml - sleep: 1h - # 使用vim来修改你的runtime字段的值 - hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml - # 修改完毕后重新submit即可 - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - ``` - -* Submit命令指导 - - 参考文档：[submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… - -* testbox有什么可选项 - - testbox可选项请参考：https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts + 可选的testbox模板，请参考：https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts >![](./../public_sys-resources/icon-note.gif) **说明：** > - > 虚拟机的testbox : vm-xxx - > 物理机的testbox : taishan200-2280-xxx + > - 容器: dc-xxx + > - 虚拟机: vm-xxx + > - 物理机: taishan200-2280-xxx @@ -245,6 +75,6 @@ > - 物理机的testbox若选择以`--axx`结尾的，则表示指定到了具体的某一个物理机。若此物理机任务队列中已经有任务在排队，则需要等待队列中前面的任务执行完毕后，才会轮到你提交的borrow任务。 > - 物理机的testbox若不选择以`-axx`结尾的，表示不指定具体的某一个物理机。则此时集群中的空闲物理机会即时被分配执行你的borrow任务。 -* 如何 borrow 指定的操作系统 +## 如何 borrow 指定的操作系统关于支持的`os`, `os_arch`, `os_version`，参见：[os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/os-os_verison-os_arch.md) -- 2.23.0

1 0

[PATCH lkp-tests] distro/adaptation: add pgbench dependency
by Hu Xuejiao 19 Jan '21

19 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- distro/adaptation/centos | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/adaptation/centos b/distro/adaptation/centos index 2cfe3877..8fbae666 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -335,3 +335,4 @@ liboath-dev: liboath-devel lsb-release: redhat-lsb linux-base: openssh-client: openssh-clients +postgresql-server: postgresql -- 2.23.0

1 0

[PATCH compass-ci] scheduler: respond an '{}' when 'vt' is non-existent or empty
by Ren Wen 19 Jan '21

19 Jan '21

In job.yaml: case1: no 'vt' -> {} case2: vt: -> {} case3: vt: domain: <domain.xml> -> {"domain":"<domain.xml>"} Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 152d549..f16dc30 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -174,6 +174,9 @@ class Sched _kernel_params = job["kernel_params"]? _kernel_params = _kernel_params.as_a.map(&.to_s).join(" ") if _kernel_params + _vt = job["vt"]? + _vt = ({} of String => String) unless (_vt && _vt != nil) + return { "job_id" => job.id, "kernel_uri" => job.kernel_uri, @@ -181,7 +184,7 @@ class Sched "kernel_params" => _kernel_params, "result_root" => job.result_root, "LKP_SERVER" => job["LKP_SERVER"]?, - "vt" => job["vt"]?, + "vt" => _vt, "RESULT_WEBDAV_PORT" => job["RESULT_WEBDAV_PORT"]? || "3080", }.to_json end -- 2.23.0

1 0

[PATCH v2 compass-ci 3/3] delimiter: use submit script in lkp-tests to submit job
by Cao Xueliang 19 Jan '21

19 Jan '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/constants.rb | 1 + src/delimiter/utils.rb | 17 +++++++++++------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index 23b5053..457e33d 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -13,3 +13,4 @@ DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" # The files which are in this dir can be uploaded by lkp-tests TMP_RESULT_ROOT = ENV['TMP_RESULT_ROOT'] || '/tmp/lkp/result' +PROCESS_JOB_YAML = "/tmp/process.yaml" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index aaad394..2c6549c 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -61,15 +61,17 @@ module Utils return monitor.run end + def save_job_to_yaml(job, yaml_file) + File.open(yaml_file, 'w') { |f| YAML.dump(job, f) } + end + def submit_job(job) - sched = SchedClient.new - response = sched.submit_job(job.to_json) + save_job_to_yaml(job, PROCESS_JOB_YAML) + response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" - res_arr = JSON.parse(response) - return nil if res_arr.empty? || !res_arr[0]['message'].empty? + return $1 if response =~ /job id=(.*)/ - # just consider build-pkg job - return res_arr[0]['job_id'] + return nil end # submit the bad job @@ -84,6 +86,8 @@ module Utils extract_finished = monitor_run_stop(query) return nil unless extract_finished.zero? + raise "the job is incredible for bisect: #{new_job_id}" unless credible?(job['bad_job_id'], new_job_id, error_id) + stats = query_stats(new_job_id, 10) raise "es cant query #{new_job_id} stats field!" unless stats @@ -138,6 +142,7 @@ module Utils job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id + job.delete('tboxgroup') job.delete('subqueue') job.delete('queue') job.delete('id') -- 2.23.0

1 0

[PATCH v2 compass-ci 2/3] src/stats_worker: select error_id which has the compile file
by Cao Xueliang 19 Jan '21

19 Jan '21

We select the error_id which has the compile file for more credible git bisect. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/extract-stats/stats_worker.cr | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..01bae21 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -82,10 +82,10 @@ class StatsWorker } ) - new_error_ids = check_new_error_ids(error_ids, job_id) - unless new_error_ids.empty? + error_id = select_error_id(check_new_error_ids(error_ids, job_id)) + if error_id STDOUT.puts "send a delimiter task: job_id is #{job_id}" - @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => new_error_ids.sample, + @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => error_id, "job_id" => job_id, "lab" => LAB}.to_json)) end @@ -93,6 +93,14 @@ class StatsWorker system "echo '#{msg}'" end + def select_error_id(new_error_ids : Array) + new_error_ids.each do |error_id| + return error_id if /(cpp|c|h):(warning|error)/i =~ error_id + end + + return nil + end + def check_new_error_ids(error_ids : Array, job_id : String) new_error_ids = [] of String error_ids.each do |error_id| -- 2.23.0

1 0

[PATCH v2 compass-ci 1/3] lib/git_bisect: use "--first-parent" option when start bisect
by Cao Xueliang 19 Jan '21

19 Jan '21

1. The alpine 3.13 support to use the latest git which has the "--first-parent" option. 2. This option is useful in avoiding false positives when a merged branch contained broken or non-buildable commits, but the merge itself was OK. Ref: https://git-scm.com/docs/git-bisect Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/delimiter/Dockerfile | 6 +++--- lib/git_bisect.rb | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/container/delimiter/Dockerfile b/container/delimiter/Dockerfile index ab7519d..cc57320 100644 --- a/container/delimiter/Dockerfile +++ b/container/delimiter/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:3.13 RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ apk update && \ @@ -9,5 +9,5 @@ RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweiclou 'g++' 'gcc' 'pcre' 'libevent' 'make' 'git' 'bash' 'grep' 'coreutils' 'curl' 'util-linux' RUN umask 002 && \ - echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ - gem install rest-client activesupport git json yaml threadpool elasticsearch faye-websocket terminal-table mail io-console + gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ + gem install rest-client activesupport git json yaml elasticsearch faye-websocket terminal-table mail io-console diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 2fee60c..e577afd 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -82,7 +82,7 @@ class GitBisect puts "bad_commit: #{@bad_commit}" puts "good_commit: #{@good_commit}" - result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` + result = `git -C #{@work_dir} bisect start --first-parent #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: more response of boot.libvirt
by Ren Wen 19 Jan '21

19 Jan '21

To download from libvirt-xml path, adding such response: - SRV_HTTP_HOST - SRV_HTTP_PORT Signed-off-by: Ren Wen <15991987063(a)163.com> --- container/scheduler/start | 3 +++ src/scheduler/constants.cr | 3 +++ src/scheduler/find_job_boot.cr | 2 ++ 3 files changed, 8 insertions(+) diff --git a/container/scheduler/start b/container/scheduler/start index 70a3496..c32a7a8 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -25,6 +25,8 @@ names = Set.new %w[ SSHR_PORT SSHR_PORT_BASE SSHR_PORT_LEN + SRV_HTTP_HOST + SRV_HTTP_PORT lab ] @@ -35,6 +37,7 @@ defaults['SSHR_PORT_LEN'] ||= 2000 defaults['SCHED_PORT'] ||= '3000' defaults['SCHED_HOST'] ||= '172.17.0.1' defaults['MASTER_FLUENTD_PORT'] ||= '24224' +defaults['SRV_HTTP_PORT'] ||= '11300' env = docker_env(defaults) CCI_REPOS = ENV['CCI_REPOS'] || '/c' diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index 1146b21..668da6d 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -23,6 +23,9 @@ INITRD_HTTP_PORT = (ENV.has_key?("INITRD_HTTP_PORT") ? ENV["INITRD_HTTP_PORT"] : OS_HTTP_HOST = (ENV.has_key?("OS_HTTP_HOST") ? ENV["OS_HTTP_HOST"] : "172.17.0.1") OS_HTTP_PORT = (ENV.has_key?("OS_HTTP_PORT") ? ENV["OS_HTTP_PORT"] : 8000).to_i32 +SRV_HTTP_HOST = (ENV.has_key?("SRV_HTTP_HOST") ? ENV["SRV_HTTP_HOST"] : "172.17.0.1") +SRV_HTTP_PORT = (ENV.has_key?("SRV_HTTP_PORT") ? ENV["SRV_HTTP_PORT"] : "11300") + CCI_REPOS = (ENV.has_key?("CCI_REPOS") ? ENV["CCI_REPOS"] : "/c") LAB_REPO = "lab-z9" diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 152d549..3f446b2 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -183,6 +183,8 @@ class Sched "LKP_SERVER" => job["LKP_SERVER"]?, "vt" => job["vt"]?, "RESULT_WEBDAV_PORT" => job["RESULT_WEBDAV_PORT"]? || "3080", + "SRV_HTTP_HOST" => SRV_HTTP_HOST, + "SRV_HTTP_PORT" => SRV_HTTP_PORT, }.to_json end -- 2.23.0

1 0

[PATCH compass-ci] src/stats_worker: choice error_id which has the compile file
by Cao Xueliang 19 Jan '21

19 Jan '21

We choice the error_id which has the compile file for more credible git bisect. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/extract-stats/stats_worker.cr | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..4f3bd81 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -82,10 +82,10 @@ class StatsWorker } ) - new_error_ids = check_new_error_ids(error_ids, job_id) - unless new_error_ids.empty? + error_id = choice_error_id(check_new_error_ids(error_ids, job_id)) + if error_id STDOUT.puts "send a delimiter task: job_id is #{job_id}" - @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => new_error_ids.sample, + @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => error_id, "job_id" => job_id, "lab" => LAB}.to_json)) end @@ -93,6 +93,14 @@ class StatsWorker system "echo '#{msg}'" end + def choice_error_id(new_error_ids : Array) + new_error_ids.each do |error_id| + return error_id if /(cpp|c|h):(warning|error)/i =~ error_id + end + + return nil + end + def check_new_error_ids(error_ids : Array, job_id : String) new_error_ids = [] of String error_ids.each do |error_id| -- 2.23.0

2 2

[PATCH compass-ci] 如何登陆测试机调测任务.md: change host-info.yaml to spinlock.yaml
by Zhang Yale 19 Jan '21

19 Jan '21

host-info.yaml looks too cryptic, so use spinlock.yaml as example can be more practical and simple. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- ...03\346\265\213\344\273\273\345\212\241.md" | 29 +++++++++++-------- 1 file changed, 17 insertions(+), 12 deletions(-) diff --git "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" "b/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" index 7a54505..d423bcd 100644 --- "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" +++ "b/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" @@ -12,38 +12,43 @@ ### 方法一：在测试机运行脚本之前进入测试机调测任务 -在job.yaml里加上sshd和sleep字段，测试机在运行脚本之前sleep, 并免密登录进去，手动输入命令或脚本进行调试，以host-info.yaml为例： +在job.yaml里加上sshd和sleep字段，测试机在运行脚本之前sleep, 并免密登录进去，手动输入命令或脚本进行调试，以spinlock.yaml为例： ```yaml - suite: host-info - category: functional + suite: spinlock + category: benchmark + + nr_threads: + - 1 sshd: # sleep at the bottom sleep: 1h - - host-info: + spinlock: ``` ### 方法二：在测试任务运行失败时进入测试机调测任务 -在job.yaml里加上on_fail字段，并在on_fail下加上sshd和sleep字段，测试任务失败后，免密登录进去手动调试，以host-info.yaml为例： +在job.yaml里加上on_fail字段，并在on_fail下加上sshd和sleep字段，测试任务失败后，免密登录进去手动调试，以spinlock.yaml为例： ```yaml - suite: host-info - category: functional + suite: spinlock + category: benchmark + + nr_threads: + - 1 - host-info: + spinlock: on_fail: sshd: - sleep: 1000s + sleep: 1h ``` ## 2.2 提交job.yaml ### 方法一：直接提交 - submit host-info.yaml + submit spinlock.yaml 在收到邮件后，按照邮件提示手动免密登录到执行机调测 @@ -51,6 +56,6 @@ ### 方法二：带参数提交 - submit -m -c host-info.yaml + submit -m -c spinlock.yaml 该方法不用查看邮件，可自动免密登录到测试机 -- 2.23.0

1 0