- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] distro/depends: fix install sysbench pkg depends
by Wei Jihui 27 Jan '21

27 Jan '21

[error] Error: Unable to find a match: mysql-devel [why] openeuler 20.03 has not libmysqlclient-dev: mysql-devel change it to libmariadbclient-dev Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/sysbench-dev | 2 +- distro/depends/sysbench-mysql-dev | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/distro/depends/sysbench-dev b/distro/depends/sysbench-dev index 3fa67deb..f570c463 100644 --- a/distro/depends/sysbench-dev +++ b/distro/depends/sysbench-dev @@ -1,2 +1,2 @@ pkg-config -libmysqlclient-dev +libmariadbclient-dev diff --git a/distro/depends/sysbench-mysql-dev b/distro/depends/sysbench-mysql-dev index ca5b03d8..65633b33 100644 --- a/distro/depends/sysbench-mysql-dev +++ b/distro/depends/sysbench-mysql-dev @@ -1,3 +1,2 @@ pkg-config -libmysqlclient-dev mysql-server -- 2.23.0

2 1

[PATCH compass-ci] providers/lib: introduce context.rb
by Xiao Shenwei 26 Jan '21

26 Jan '21

[why] expand erb templates need a class to save context [how] Context class provider a readable property @info, it will save the context required by erb, and a expand_erb method, it can expand a erb templates based on @info. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/context.rb | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 providers/lib/context.rb diff --git a/providers/lib/context.rb b/providers/lib/context.rb new file mode 100644 index 0000000..1ea36b0 --- /dev/null +++ b/providers/lib/context.rb @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'erb' +require_relative "#{ENV['LKP_SRC']}/lib/hashugar" + +# saves the context required by expand erb templates +class Context + attr_reader :info + + def initialize(mac, hostname, queues) + @info = { + 'mac' => mac, + 'hostname' => hostname, + 'queues' => queues + } + end + + def merge!(hash) + @info.merge!(hash) + end + + def expand_erb(template, context_hash = {}) + @info.merge!(context_hash) + context = Hashugar.new((a)info).instance_eval { binding } + ERB.new(template, nil, '%').result(context) + end +end -- 2.23.0

2 4

[PATCH lkp-tests] jobs: add multi-qemu.yaml multi-docker.yaml
by Xiao Shenwei 26 Jan '21

26 Jan '21

submit the job to a physical machine, then the machine will start some clients to consume job. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- jobs/multi-docker.yaml | 17 +++++++++++++++++ jobs/multi-qemu.yaml | 23 +++++++++++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 jobs/multi-docker.yaml create mode 100644 jobs/multi-qemu.yaml diff --git a/jobs/multi-docker.yaml b/jobs/multi-docker.yaml new file mode 100644 index 000000000..8e42208a9 --- /dev/null +++ b/jobs/multi-docker.yaml @@ -0,0 +1,17 @@ +suite: multi-docker +category: functional +simplify-ci: +runtime: 100d +multi-docker-0: + nr_container: 20 + tbox_group: dc-1g +multi-docker-1: + nr_container: 10 + tbox_group: dc-2g +multi-docker-2: + nr_container: 5 + tbox_group: dc-4g +multi-docker-3: + nr_container: 20 + tbox_group: dc-8g +sleep: diff --git a/jobs/multi-qemu.yaml b/jobs/multi-qemu.yaml new file mode 100644 index 000000000..200f66065 --- /dev/null +++ b/jobs/multi-qemu.yaml @@ -0,0 +1,23 @@ +suite: multi-qemu +category: functional +simplify-ci: +runtime: 100d +multi-qemu-0: + nr_vm: 5 + tbox_group: vm-2p1g +multi-qemu-1: + nr_vm: 5 + tbox_group: vm-2p2g +multi-qemu-2: + nr_vm: 5 + tbox_group: vm-2p4g +multi-qemu-3: + nr_vm: 10 + tbox_group: vm-2p8g +multi-qemu-4: + nr_vm: 5 + tbox_group: vm-2p16g +multi-qemu-5: + nr_vm: 1 + tbox_group: vm-2p32g +sleep: -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/libvirt: add a libvirt provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

request api: http://$SCHED_HOST:$SCHED_PORT/boot.libvirt/mac/$mac Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/run.rb | 60 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100755 providers/libvirt/run.rb diff --git a/providers/libvirt/run.rb b/providers/libvirt/run.rb new file mode 100755 index 0000000..b4f8b4c --- /dev/null +++ b/providers/libvirt/run.rb @@ -0,0 +1,60 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'digest/md5' +require_relative "#{ENV['CCI_SRC']}/lib/log" +require_relative "#{ENV['CCI_SRC']}/lib/sched_client" +require_relative "#{ENV['CCI_SRC']}/providers/lib/context" + +def create_logger(hostname) + filename = "#{hostname}.log" + File.delete(filename) if FileTest.exist?(filename) + Log.new filename +end + +def compute_mac(string) + /(..)(..)(..)(..)(..)/ =~ Digest::MD5.hexdigest(string) + "0a:#{$1}:#{$2}:#{$3}:#{$4}:#{$5}" +end + +def job_exist?(response) + flag = false + if response['job_id'].empty? + puts '----------' + puts 'No job now' + puts '----------' + flag = true + end + return flag +end + +def request_job(context, sched_client, logger) + mac = context.info['mac'] + hostname = context.info['hostname'] + queues = context.info['queues'] + sched_client.register_mac2host(hostname, mac) + sched_client.register_host2queues(hostname, queues) + response = JSON.parse(sched_client.consume_job('libvirt', 'mac', mac)) + if job_exist?(response) + logger.info('No job now') + sched_client.delete_mac2host(mac) + sched_client.delete_host2queues(hostname) + response = nil + end + return response +end + +def main(hostname, queues) + logger = create_logger(hostname) + mac = compute_mac hostname + context = Context.new(mac, hostname, queues) + sched_client = SchedClient.new + response = request_job(context, sched_client, logger) + return if response.nil? + + sched_client.delete_mac2host(mac) + sched_client.delete_host2queues(hostname) +end -- 2.23.0

1 0

[PATCH compass-ci] providers/libvirt: add a libvirt provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

request api: http://$SCHED_HOST:$SCHED_PORT/boot.libvirt/mac/$mac Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/run.rb | 67 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100755 providers/libvirt/run.rb diff --git a/providers/libvirt/run.rb b/providers/libvirt/run.rb new file mode 100755 index 0000000..0ff0d4f --- /dev/null +++ b/providers/libvirt/run.rb @@ -0,0 +1,67 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'digest/md5' +require_relative "#{ENV['CCI_SRC']}/lib/log" +require_relative "#{ENV['CCI_SRC']}/lib/sched_client" +require_relative "#{ENV['CCI_SRC']}/providers/lib/context" + +def create_logger(hostname) + filename = "#{hostname}.log" + File.delete(filename) if FileTest.exist?(filename) + Log.new filename +end + +def compute_mac(string) + /(..)(..)(..)(..)(..)/ =~ Digest::MD5.hexdigest(string) + "0a:#{$1}:#{$2}:#{$3}:#{$4}:#{$5}" +end + +def nojob?(response) + flag = false + if response['job_id'].empty? + puts '----------' + puts 'No job now' + puts '----------' + flag = true + end + return flag +end + +def request(context, sched_client, logger) + mac = context.info['mac'] + hostname = context.info['hostname'] + queues = context.info['queues'] + sched_client.register_mac2host(hostname, mac) + sched_client.register_host2queues(hostname, queues) + response = JSON.parse(sched_client.consume_job('libvirt', 'mac', mac)) + if nojob?(response) + logger.info('No job now') + sched_client.delete_mac2host(mac) + sched_client.delete_host2queues(hostname) + response = nil + end + return response +end + +def main(hostname, queues) + logger = create_logger(hostname) + mac = compute_mac hostname + context = Context.new(mac, hostname, queues) + sched_client = SchedClient.new + response = request(context, sched_client, logger) + return if response.nil? + + # TODO + # based on response load file + # TODO + # generate domain.xml + # TODO + # use ruby-libvirt api start a vm + # clean env + sched_client.delete_mac2host(mac) + sched_client.delete_host2queues(hostname) +end -- 2.23.0

3 4

[PATCH lkp-tests] tests/build-pkg: export V=1 to print verbose output
by Lin Jiaxin 26 Jan '21

26 Jan '21

A developer who really wants to make silent rules enabled by default will calling AM_SILENT_RULES([yes]) in configure.ac. It make compiling packages by default does not show the full compilation/link command lines, but only short "summaries", like this: EPOINFO main/repoinfo.h make all-am make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' CC main/ctags-cmd.o make[1]: Leaving directory '/tmp/lkp/uctags-git/src/ctags' make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' CC main/libctags_a-args.o However, we need the full compilation/link command lines, like this: ./misc/gen-repoinfo main/repoinfo.h make all-am make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' gcc -DHAVE_CONFIG_H -I. -I. -I. -I./main -I./peg -DHAVE_PACKCC -DUSE_SYSTEM_STRNLEN -DHAVE_REPOINFO_H -D_FORTIFY_SOURCE=2 -std=gnu99 -Wall -I/usr/include/libxml2 -march=native -O2 -pipe -fstack-protector-strong -fno-plt -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -MT main/ctags-cmd.o -MD -MP -MF main/.deps/ctags-cmd.Tpo -c -o main/ctags-cmd.o `test -f 'main/cmd.c' || echo './'`main/cmd.c make[1]: Leaving directory '/tmp/lkp/uctags-git/src/ctags' make[1]: Entering directory '/tmp/lkp/uctags-git/src/ctags' mv -f main/.deps/ctags-cmd.Tpo main/.deps/ctags-cmd.Po make V=1 will produce verbose output, so export V=1 in build-pkg script. refer to: https://www.gnu.org/software/automake/manual/html_node/Automake-Silent-Rule… https://lists.gnu.org/archive/html/bug-autoconf/2012-01/msg00009.html Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/build-pkg b/tests/build-pkg index 5344da2a..f3545a19 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -157,6 +157,8 @@ build_source_pkg() local mark="true" [ "${os}" == "archlinux" ] && local mark="pacman" + export V=1 + PACMAN="${mark}" BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf -- 2.23.0

2 1

[PATCH compass-ci] doc/manual: optimize borrow-machine.en.md
by Luan Shengde 26 Jan '21

26 Jan '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/borrow-machine.en.md | 341 +++++++++----------------------- 1 file changed, 98 insertions(+), 243 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index 6a1fe19..36823c3 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -1,246 +1,101 @@ -# Prerequisites - -Ensure that you have performed the following operations according to the [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc…: - -- Send an email to apply for an account. -- Receive an email from compass-ci-robot(a)qq.com. -- Configure the local environment. - -# Applying for a Test Machine (VM) - -1. Generate a local RSA private-public key pair. - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. Select the YAML file as required. - - A **lkp-tests** folder is stored in each user directory `/home/${USER}`. - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. Submit the YAML file and connect to the test machine (VM). - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170593 - query=>{"job_id":["z9.170593"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170593","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593"} - {"job_id": "z9.170593", "result_root": "/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593", "job_state": "set result root"} - {"job_id": "z9.170593", "job_state": "boot"} - {"job_id": "z9.170593", "job_state": "download"} - {"time":"2020-11-26 14:45:06","mac":"0a-1f-0d-3c-91-5c","ip":"172.18.156.13","job_id":"z9.170593","state":"running","testbox":"vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - {"job_state":"running","job_id":"z9.170593"} - {"job_id": "z9.170593", "state": "set ssh port", "ssh_port": "51840", "tbox_name": "vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:51840' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Thu Nov 26 06:44:18 CST 2020 - - System load: 0.83 - Processes: 107 - Memory used: 6.1% - Swap used: 0.0% - Usage On: 89% - IP address: 172.18.156.13 - Users online: 1 - - - - root@vm-2p8g ~# - ``` - - For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. - -4. Return the test machine (VM) after use. - - ```shell - root@vm-2p8g ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` - -# Applying for a Test Machine (Physical Machine) - -1. Generate a local RSA private-public key pair. - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. Select the YAML file as required. - - A **lkp-tests** folder is stored in each user directory `/home/${USER}`. - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. Submit the YAML file and connect to the test machine (physical machine). - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=taishan200-2280-2s64p-256g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170594 - query=>{"job_id":["z9.170594"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170594","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594"} - {"job_id": "z9.170594", "result_root": "/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594", "job_state": "set result root"} - {"job_id": "z9.170594", "job_state": "boot"} - {"job_id": "z9.170594", "job_state": "download"} - {"time":"2020-11-26 14:51:56","mac":"84-46-fe-26-d3-47","ip":"172.168.178.48","job_id":"z9.170594","state":"running","testbox":"taishan200-2280-2s64p-256g--a5"} - {"job_state":"running","job_id":"z9.170594"} - {"job_id": "z9.170594", "state": "set ssh port", "ssh_port": "50420", "tbox_name": "taishan200-2280-2s64p-256g--a5"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:50420' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Thu Nov 26 14:51:59 CST 2020 - - System load: 1.31 - Processes: 1020 - Memory used: 5.1% - Swap used: 0.0% - Usage On: 3% - IP address: 172.168.178.48 - Users online: 1 - - - - root@taishan200-2280-2s64p-256g--a5 ~# - ``` - - For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. - -4. Return the test machine (physical machine) after use. - - ```shell - root@taishan200-2280-2s64p-256g--a5 ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` +# Prepare + +- Apply account +- Config default yaml files + +If have not completed above works, reference to [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… to finish it. + +# Apply testbox + +## 1. Generate local ssh pub key + + Use the command below to check the pub key exists: + + ls ~/.ssh/*.pub + + Generate one if you don't have one now: + + ssh-keygen + +## 2. Select the job yaml + + We have provided various examples for you in dir ~/lkp-tests/jobs: + + You can filtrate examples for borrowing machine with command below: + + cd ~/lkp-tests/jobs + ls borrow* + +## 3. Submit job + + Command to submit jobs: + + for DCs: + + submit -c -m testbox=dc-2g os_mount=container docker_image=centos:8 borrow-1h.yaml + + for VMs: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml + + for HWs: + + submit -c -m testbox=taishan200-2280-2s48p-256g borrow-1h.yaml + + - You can view the logs in real time for the job with the command above。 + - You will directly login the testbox if the job runs successfully. + - And you will receive an email that contains login command and server configuration information. + - Only within the period of borrowing machine, you can access the textbox with the login command. + +## 4. Return testbox + + Reruan manually(recommended): + + Manually execute 'reboot' in time to return the testbox. + Avoid a waste of computer resource with no-load running. + + Automatically return on maturity: + + The testbox will be returned automatically if it has expired its service life. + + - All testboxes will be returned if it has been executed command 'reboot'. + - The testbox cannot be accessed any more after it has been returned. + - Apply a new one if you want continue to use it. # FAQ -* How Do I Change the Duration of Keeping the Test Machine when Applying for It? - - ```shell - hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml - suite: borrow - testcase: borrow - - runtime: 1h - ssh_pub_key: <%= - begin - File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp - rescue - nil - end - %> - sshd: - # sleep at the bottom - sleep: - hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the runtime field. - hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml - # After changing the value, submit the request again. - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - ``` - -* Guide to the **submit** Command - - Reference: [submit-job.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma… - -* What Are the testbox Options? - - For details about the testbox options, visit https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts. - - > ![](./../public_sys-resources/icon-note.gif) **Note** - > - > VM testbox: vm-xxx - > - > PM testbox: taishan200-2280-xxx - - > ![](./../public_sys-resources/icon-notice.gif) **Notice** - > -> - If the testbox of a physical machine ends with `--axx`, a physical machine is specified. If a task is already in the task queue of the physical machine, the borrow task you submitted will not be processed until the previous task in the queue is completed. - > - If the testbox of a physical machine does not end with `-axx`, no physical machine is specified. In this case, the borrow task you submitted will be immediately allocated to idle physical machines in the cluster for execution. - -* How Do I Borrow the Specified Operating System? - - For details about the supported `os`, `os_arch`, and `os_version`, see [os-os\_verison-os\_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/…. +## Customize the borrowing time + + Find key in the yaml file and edit its value according to your requirement. + + - The borrowing period can be calculated in days and hours. + - The maximum period is no more than 10 day. + +## Guidance for command submit + + See the usage and options for command 'submit' the command below: + + submit -h + + Reference the following line to learn the advanced usage for command 'submit': + + [submit detailed usage](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/sub… + +## Available testbox + + For a full list of testbox, reference to https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts + + >![](./../public_sys-resources/icon-note.gif) **instruction:** + > + > - DCs: dc-xxx + > - VMs: vm-xxx + > - HWs: taishan200-2280-xxx + + >![](./../public_sys-resources/icon-notice.gif) **attention:** + > - It means that you choosed a specified physical machine if the testbox name is end with `--axx`. + > - You will need to wait if there are already tasks in the task queue for the machine. + > - Your job will be randomly assigned to a machine that meets the requirements if the testbox name is not end with '-axx'. + +## Specify the OS + + About supportted `os`, `os_arch`, `os_version`, reference to [os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/ma… -- 2.23.0

1 0

[PATCH lkp-tests] lib/log.sh: add square bracket
by Liu Yinsi 26 Jan '21

26 Jan '21

before: 2021-01-26 14:53:36 ERROR -- cifs module is not loaded in after: 2021-01-26 15:00:27 [ERROR] -- cifs module is not loaded in [why] use square bracket make log more obvious. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/log.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/log.sh b/lib/log.sh index 5acf61825..207795ece 100644 --- a/lib/log.sh +++ b/lib/log.sh @@ -2,25 +2,25 @@ log() { - echo "$(date +'%F %T') INFO -- $*" + echo "$(date +'%F %T') [INFO] -- $*" } log_info() { - echo "$(date +'%F %T') INFO -- $*" + echo "$(date +'%F %T') [INFO] -- $*" } log_debug() { - echo "$(date +'%F %T') DEBUG -- $*" + echo "$(date +'%F %T') [DEBUG] -- $*" } log_warn() { - echo "$(date +'%F %T') WARN -- $*" >&2 + echo "$(date +'%F %T') [WARN] -- $*" >&2 } log_error() { - echo "$(date +'%F %T') ERROR -- $*" >&2 + echo "$(date +'%F %T') [ERROR] -- $*" >&2 } -- 2.23.0

1 0

[PATCH lkp-tests] feat: support a new field for local disk: rootfs_disk
by Yu Chuan 26 Jan '21

26 Jan '21

[Why] We now need auto install iso to the specific disk of testbox, so in order to avoid conflict with rootfs_partition, we support a new field for local disk: rootfs_disk. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- filters/run_on_local_disk | 8 ++++---- lib/bootstrap.sh | 23 +++++++++++++++++++++++ lkp-exec/qemu | 7 ++++++- 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/filters/run_on_local_disk b/filters/run_on_local_disk index 389d1ca17fdc..2092bd2b6028 100755 --- a/filters/run_on_local_disk +++ b/filters/run_on_local_disk @@ -1,11 +1,11 @@ #!/bin/bash -# check if testbox has rootfs_partition to run the job +# check if testbox has rootfs_partition or rootfs_disk to run the job if [[ "$run_on_local_disk" == "yes" || "$run_on_local_disk" == "true" ]]; then - if [[ -z "$rootfs_partition" ]]; then - echo "rm $job_file due to no rootfs_partition" - echo "run_on_local_disk needs rootfs_partition" + if [[ -z "$rootfs_partition" && -z "$rootfs_disk" ]]; then + echo "rm $job_file due to no rootfs_partition or rootfs_disk" + echo "run_on_local_disk needs rootfs_partition or rootfs_disk" exit 1 fi fi diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index b04ea3dd48b7..6c0055bd19f5 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -490,6 +490,12 @@ wait_load_disk() ls $rootfs_partition >/dev/null 2>&1 && return # eg: LABEL=LKP-ROOTFS blkid | grep -q ${rootfs_partition#*=} && return + + # eg: /dev/disk/by-id/ata-WDC_WD1002FAEX-00Z3A0_WD-WCATRC577623 + ls $rootfs_disk >/dev/null 2>&1 && return + # eg: LABEL=LKP-ROOTFS + blkid | grep -q ${rootfs_disk#*=} && return + sleep 1 done @@ -515,6 +521,23 @@ mount_rootfs() mkdir -p $ROOTFS_DIR/tmp CACHE_DIR=$ROOTFS_DIR/tmp cleanup_pkg_cache $CACHE_DIR + elif [ -n "$rootfs_disk" ]; then + # wait for the machine to load the disk + wait_load_disk || { + # skipping following test if disk can't be load + echo "can't load the disk $rootfs_disk, skip testing..." + set_job_state 'load_disk_fail' + return 1 + } + ROOTFS_DIR=/opt/rootfs + mkdir -p $ROOTFS_DIR + mount $rootfs_disk $ROOTFS_DIR || { + mkfs.btrfs -f $rootfs_disk + mount $rootfs_disk $ROOTFS_DIR + } + mkdir -p $ROOTFS_DIR/tmp + CACHE_DIR=$ROOTFS_DIR/tmp + cleanup_pkg_cache $CACHE_DIR else CACHE_DIR=/tmp/cache mkdir -p $CACHE_DIR diff --git a/lkp-exec/qemu b/lkp-exec/qemu index 9e4b576cf50f..1cbd316f6146 100755 --- a/lkp-exec/qemu +++ b/lkp-exec/qemu @@ -91,14 +91,16 @@ replace_script_partition_val() local nr_hdd_vdisk="$(echo $hdd_partitions | wc -w)" local nr_ssd_vdisk="$(echo $ssd_partitions | wc -w)" local nr_swap_vdisk="$(echo $swap_partitions | wc -w)" + local nr_rootfs_vdisk="$(echo $rootfs_partition | wc -w)" + [ -z "$rootfs_partition" ] && nr_rootfs_vdisk="$(echo $rootfs_disk | wc -w)" [[ $nr_hdd_partitions ]] || nr_hdd_partitions=$nr_hdd_vdisk [[ $nr_ssd_partitions ]] || nr_ssd_partitions=$nr_ssd_vdisk VDISK_NUM=$((nr_hdd_partitions+nr_ssd_partitions+nr_swap_vdisk+nr_rootfs_vdisk)) - [[ "$hdd_partitions$ssd_partitions$swap_partitions$rootfs_partition" =~ '/dev/vda' ]] && return + [[ "$hdd_partitions$ssd_partitions$swap_partitions$rootfs_partition$rootfs_disk" =~ '/dev/vda' ]] && return if ((VDISK_NUM)); then local index=0 @@ -115,6 +117,9 @@ replace_script_partition_val() -e "s%export swap_partitions=.*%export swap_partitions='${vdisk_swap_val}'%" \ -e "s%export rootfs_partition=.*%export rootfs_partition='${vdisk_rootfs_val}'%" \ $job_script + + [ -z "$rootfs_partition" ] && + sed -i -e "s%export rootfs_disk=.*%export rootfs_disk='${vdisk_rootfs_val}'%" $job_script fi } -- 2.23.0

3 5

[PATCH compass-ci] update apply account email address
by Li Yuanchao 26 Jan '21

26 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- README.en.md | 2 +- README.zh.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.en.md b/README.en.md index 632a480..fa88305 100644 --- a/README.en.md +++ b/README.en.md @@ -44,7 +44,7 @@ If a new error ID is generated, the bisect is automatically triggered to locate **Logging in to the Test Environment** -1. Send an email to the compass-ci-reply(a)qq.com to [apply for an account](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/a…. +1. Send an email to the compass-ci-robot(a)qq.com to [apply for an account](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/a…. 2. [Complete the environment configuration](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma… as instructed by the email. 3. Add the **sshd** field to the test task, submit the task, and [log in to the test environment](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu…. diff --git a/README.zh.md b/README.zh.md index 1339ef0..0e6cfb6 100644 --- a/README.zh.md +++ b/README.zh.md @@ -47,7 +47,7 @@ Compass-CI �� git repos�� **��** -1. �� compass-ci-reply(a)qq.com ��[��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.md)�� +1. �� compass-ci-robot(a)qq.com ��[��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.md)�� 2. ��[��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E6%9C%AC%E5%9C%B0%E5%AE%89%E8%A3%85compass-ci%E5%AE%A2%E6%88%B7%E7%AB%AF.md)�� 3. �� sshd ��[��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E5%A6%82%E4%BD%95%E7%99%BB%E5%BD%95%E6%B5%8B%E8%AF%95%E6%9C%BA%E8%B0%83%E6%B5%8B%E4%BB%BB%E5%8A%A1.md)�� -- 2.23.0

1 0