- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] docker-rootfs/common: add comment for set password file.
by Wang Chenglong 04 Jan '21

04 Jan '21

[Why] add password file is easy to ignore. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/common | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/container/docker-rootfs/common b/container/docker-rootfs/common index fe37da9..2ee16c8 100755 --- a/container/docker-rootfs/common +++ b/container/docker-rootfs/common @@ -33,7 +33,8 @@ usage() dst_rootfs_new_abspath: destination absolute path to create for rootfs Example: - ./run /tmp/openEuler-docker/openEuler-docker.${aarch}.tar.xz /tmp/openeuler-rootfs/ + ./run /tmp/openEuler-docker/openEuler-docker.${aarch}.tar.xz /tmp/openeuler-rootfs/ + # please ensure $HOME/.config/compass-ci/rootfs-passwd exists " } -- 2.23.0

3 4

[PATCH v3 compass-ci] docker-rootfs: add README.md
by Wang Chenglong 04 Jan '21

04 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/docker-rootfs/README.md diff --git a/container/docker-rootfs/README.md b/container/docker-rootfs/README.md new file mode 100644 index 0000000..1b6f42f --- /dev/null +++ b/container/docker-rootfs/README.md @@ -0,0 +1,18 @@ +## Use the tool to create a new nfsroot for os + +## Usage: + ./run <src_docker_file_abspath> <dst_rootfs_new_abspath> + + src_docker_file_abspath: source .tar.xz file absolute path with suffix: [ tar.xz ]. + dst_rootfs_new_abspath: destination absolute path to create for rootfs. + +## Example: + ./run /tmp/openEuler-docker/openEuler-docker.aarch64.tar.xz /tmp/openeuler-rootfs/ + # please ensure $HOME/.config/compass-ci/rootfs-passwd exists + +## Some configuration items: + ./packages-to-install + # If you want to pre-install the software, you can write the package names in ./packages-to-install. + + $HOME/.config/compass-ci/rootfs-passwd + # Set the password for the image into this file. -- 2.23.0

2 2

[PATCH v4 compass-ci] docker-rootfs: add README.md
by Wang Chenglong 04 Jan '21

04 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/docker-rootfs/README.md diff --git a/container/docker-rootfs/README.md b/container/docker-rootfs/README.md new file mode 100644 index 0000000..e7b5084 --- /dev/null +++ b/container/docker-rootfs/README.md @@ -0,0 +1,18 @@ +# Use the tool to create a new nfsroot for os + +## Usage: + ./run <src_docker_file_abspath> <dst_rootfs_new_abspath> + + src_docker_file_abspath: source .tar.xz file absolute path with suffix: [ tar.xz ]. + dst_rootfs_new_abspath: destination absolute path to create for rootfs. + +## Example: + ./run /tmp/openEuler-docker/openEuler-docker.aarch64.tar.xz /tmp/openeuler-rootfs/ + # Please ensure $HOME/.config/compass-ci/rootfs-passwd exists. + +## Some configuration items: + ./packages-to-install + # If you want to pre-install the software, you can write the package names in ./packages-to-install. + + $HOME/.config/compass-ci/rootfs-passwd + # Set the password for the image into this file. -- 2.23.0

1 0

[PATCH compass-ci 2/2] qcow2rootfs: fix rootfs-dir permission error
by Yu Chuan 03 Jan '21

03 Jan '21

[Error Msg] '01/03/21_04:25:49' Compressing file to /root/extract/rootfs-dir/qcow2-out.tar.gz Please wait until command returned... <r.gz compress:gzip numericowner:true xattrs:true selinux:true acls:true libguestfs: error: /root/extract/rootfs-dir/qcow2-out.tar.gz: Permission denied libguestfs: error: file receive cancelled by daemon Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/qcow2rootfs/bin/main | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/qcow2rootfs/bin/main b/container/qcow2rootfs/bin/main index b6ef7238f24a..c7527b794b89 100755 --- a/container/qcow2rootfs/bin/main +++ b/container/qcow2rootfs/bin/main @@ -34,7 +34,7 @@ unzip_qcow2_xz() { extract_rootfs() { echo "[WARNNING] Temporarily change owner of qcow2 file" chown "$EXTRACT_USER":"$EXTRACT_USER" "$QCOW2_FILE" - chmod 775 "$ROOTFS_DIR" + chmod 777 "$ROOTFS_DIR" echo "[$(date +'%D %T')] Converting .qcow2 to .tar.gz, please wait several minutes..." su - "$EXTRACT_USER" -c " -- 2.23.0

1 0

[PATCH compass-ci 1/2] qcow2rootfs: generate pkgs record file for result rootfs
by Yu Chuan 03 Jan '21

03 Jan '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/qcow2rootfs/bin/cci-generate-pkgs | 15 +++++++++++++++ container/qcow2rootfs/bin/main | 6 ++++++ 2 files changed, 21 insertions(+) create mode 100755 container/qcow2rootfs/bin/cci-generate-pkgs diff --git a/container/qcow2rootfs/bin/cci-generate-pkgs b/container/qcow2rootfs/bin/cci-generate-pkgs new file mode 100755 index 000000000000..e6a974cd0f41 --- /dev/null +++ b/container/qcow2rootfs/bin/cci-generate-pkgs @@ -0,0 +1,15 @@ +#!/bin/bash + +RESULT_FILE=/etc/pkgs-$(date "+%Y-%m-%d-%H-%M-%S").list + +command -v rpm >/dev/null && { + rpm -qa > $RESULT_FILE +} + +command -v dpkg >/dev/null && { + dpkg -l > $RESULT_FILE +} + +chmod 644 $RESULT_FILE + +exit 0 diff --git a/container/qcow2rootfs/bin/main b/container/qcow2rootfs/bin/main index b70ec7766649..b6ef7238f24a 100755 --- a/container/qcow2rootfs/bin/main +++ b/container/qcow2rootfs/bin/main @@ -74,6 +74,12 @@ pre_config_rootfs() { if [ -f ${ROOTFS_DIR}/etc/selinux/config ]; then sed -i -r 's/SELINUX=enforcing/SELINUX=disabled/g' "$ROOTFS_DIR/etc/selinux/config" fi + + # generate /etc/pkgs-$(date "+%Y-%m-%d-%H-%M-%S").list file for result rootfs + echo "generate /etc/pkgs-$(date "+%Y-%m-%d-%H-%M-%S").list" + cp $EXTRACT_ROOT/bin/cci-generate-pkgs $ROOTFS_DIR + chroot $ROOTFS_DIR /cci-generate-pkgs + rm /$ROOTFS_DIR/cci-generate-pkgs } clean_environment() { -- 2.23.0

1 0

[PATCH v2 lkp-tests] lib/bootstrap.sh: 'run-ipconfig' step should be before the check step
by Yu Chuan 03 Jan '21

03 Jan '21

[Why] The `run-ipconfig` step is the IP configuration step for initramfs, which should be executed immediately after checking the initramfs-tools file for existence. [Error Msg] + [ ! -e /usr/share/initramfs-tools/scripts/functions ] + get_net_devices + local net_devices + local i + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + [ p0s1/ != /sys/class/net/enp0s1/ ] + basename /sys/class/net/enp0s1/ + net_devices= enp0s1 + [ != /sys/class/net/lo/ ] + continue + echo enp0s1 + local net_devices= enp0s1 + [ -z ] + warn_no_eth0 + [ -f /proc/config.gz ] + return + echo ls /sys/class/net + ls /sys/class/net + is_virt + [ -n ] + [ -n ] + has_cmd virt-what + command -v virt-what + grep -q -w hypervisor /proc/cpuinfo + return 0 + export NO_NETWORK=1 + echo export NO_NETWORK=1 due to no net devices + return Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 7d15bb370fa9..770d550724f1 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -160,6 +160,9 @@ setup_network() return } + $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig + network_ok && return + local net_devices=$(get_net_devices) if [ -z "$net_devices" ]; then @@ -181,9 +184,6 @@ setup_network() fi fi - $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig - network_ok && return - local err_msg='IP-Config: Auto-configuration of network failed' dmesg | grep -q -F "$err_msg" || { # Include $err_msg in the error message so that it matches -- 2.23.0

1 0

[PATCH lkp-tests] lib/bootstrap.sh: 'run-ipconfig' step should be before the check step
by Yu Chuan 03 Jan '21

03 Jan '21

[Before] test job: result/borrow/2020-12-31/vm-2p8g/debian-sid-aarch64/3600/z9.275666 key log: + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + cat /sys/class/net/enp0s1//operstate + [ down = up ] [After] test job: result/borrow/2020-12-31/vm-2p8g/debian-sid-aarch64/3600/z9.275673 key log: + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + cat /sys/class/net/enp0s1//operstate + [ up = up ] Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 7d15bb370fa9..770d550724f1 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -160,6 +160,9 @@ setup_network() return } + $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig + network_ok && return + local net_devices=$(get_net_devices) if [ -z "$net_devices" ]; then @@ -181,9 +184,6 @@ setup_network() fi fi - $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig - network_ok && return - local err_msg='IP-Config: Auto-configuration of network failed' dmesg | grep -q -F "$err_msg" || { # Include $err_msg in the error message so that it matches -- 2.23.0

2 2

git count
by Wu Fengguang 31 Dec '20

31 Dec '20

Cao Xueliang is our code star again, thanks xueliang! 22 Cao Xueliang 21 Luan Shengde 16 Wu Zhende 13 Wei Jihui 13 Li Yuanchao 11 Hu Xuejiao 11 Ren Wen 10 Lu Weitao 9 Liu Shaofei 7 Lin Jiaxin 7 Lu Kaiyi 6 Liu Yinsi 6 Cui Yili 6 Wang Yong 5 Li Ping 4 Xiao Shenwei 3 Wu Fengguang 3 Zhang Yu 3 Yu Chuan 2 Xu Xijian 2 Wang Chenglong 1 Zhang Yale 0 Bai Jing 0 Tong Qunfeng 0 Du Kaitian 0 Sun Yukui 0 Zhang Dewan 0 Shi Zhichao

1 0

[PATCH v2 compass-ci 2/2] mail-robot: optimize assign-account-email
by Luan Shengde 31 Dec '20

31 Dec '20

set different greetings for mail-robot and bisect Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index ee3b574..b71b3fe 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -4,13 +4,19 @@ # frozen_string_literal: true def build_apply_account_email(my_info) + email_greeting = if my_info['bisect'] + 'We have automatically created the following information for you.' + else + 'Thank you for joining us.' + end + email_msg = <<~EMAIL_MESSAGE To: #{my_info['my_email']} Subject: [compass-ci] Account Ready Dear #{my_info['my_name']}, - Thank you for joining us. + #{email_greeting} You can use the following info to submit jobs: -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/assign_account_client: optimize
by Luan Shengde 31 Dec '20

31 Dec '20

add new key for bisect way to apply account use rest-client instead of linux command 'curl' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/assign_account_client.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/assign_account_client.rb b/lib/assign_account_client.rb index d7e0623..4831545 100755 --- a/lib/assign_account_client.rb +++ b/lib/assign_account_client.rb @@ -7,6 +7,7 @@ require 'json' require 'mail' require 'set' require 'optparse' +require 'rest-client' require_relative '../container/defconfig' require_relative 'es_client' require_relative '../container/mail-robot/lib/assign-account-email' @@ -52,7 +53,8 @@ class AutoAssignAccount apply_info['is_update_account'] = true unless apply_info['my_login_name'].nil? apply_info['lab'] = LAB - account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') + assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" + account_info_str = RestClient.post assign_account_url, apply_info.to_json JSON.parse account_info_str end @@ -69,6 +71,7 @@ class AutoAssignAccount end def send_mail + @my_info['bisect'] = true message = build_apply_account_email(@my_info) %x(curl -XPOST "#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text" -d "#{message}") end -- 2.23.0

1 0

[PATCH compass-ci 2/2] mail-robot: optimize assign-account-email
by Luan Shengde 31 Dec '20

31 Dec '20

set different greetings for mail-robot and bisect Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index ee3b574..b71b3fe 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -4,13 +4,19 @@ # frozen_string_literal: true def build_apply_account_email(my_info) + email_greeting = if my_info['bisect'] + 'We have automatically created the following information for you.' + else + 'Thank you for joining us.' + end + email_msg = <<~EMAIL_MESSAGE To: #{my_info['my_email']} Subject: [compass-ci] Account Ready Dear #{my_info['my_name']}, - Thank you for joining us. + #{email_greeting} You can use the following info to submit jobs: -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/assign_account_client: optimize
by Luan Shengde 31 Dec '20

31 Dec '20

add new key for bisect way to apply account use rest-client instead of linux command 'curl' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/assign_account_client.rb | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/lib/assign_account_client.rb b/lib/assign_account_client.rb index d7e0623..15d3f35 100755 --- a/lib/assign_account_client.rb +++ b/lib/assign_account_client.rb @@ -7,6 +7,7 @@ require 'json' require 'mail' require 'set' require 'optparse' +require 'rest-client' require_relative '../container/defconfig' require_relative 'es_client' require_relative '../container/mail-robot/lib/assign-account-email' @@ -52,7 +53,8 @@ class AutoAssignAccount apply_info['is_update_account'] = true unless apply_info['my_login_name'].nil? apply_info['lab'] = LAB - account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') + assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" + account_info_str = RestClient.post assign_account_url, apply_info.to_json JSON.parse account_info_str end @@ -69,6 +71,7 @@ class AutoAssignAccount end def send_mail + @my_info['bisect'] = true message = build_apply_account_email(@my_info) %x(curl -XPOST "#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text" -d "#{message}") end @@ -81,3 +84,12 @@ class AutoAssignAccount send_mail end end + +my_info = { + 'my_email' => 'shdluan(a)163.com', + 'my_name' => 'lsd', + 'my_commit_url' => '' +} + +test = AutoAssignAccount.new(my_info) +test.send_account -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result: send bisect report to author email
by Cao Xueliang 31 Dec '20

31 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index f0f0194..209a28d 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -18,8 +18,9 @@ class MailBisectResult @pkgbuild_repo = bisect_info['pkgbuild_repo'] @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] @git_commit = GitCommit.new(@repo, @commit_id) + @to = @git_commit.author_email # now send mail to review - @to = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com' + @bcc = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' end def create_send_email @@ -37,6 +38,7 @@ class MailBisectResult data = <<~BODY To: #{@to} + Bcc: #{@bcc} Subject: #{subject} Hi #{(a)git_commit.author_name}, -- 2.23.0

1 0

[PATCH compass-ci] delimiter: use docker to run build-pkg job
by Cao Xueliang 31 Dec '20

31 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/delimiter.rb | 2 +- src/delimiter/utils.rb | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/delimiter/delimiter.rb b/src/delimiter/delimiter.rb index bf36db3..1afffbc 100644 --- a/src/delimiter/delimiter.rb +++ b/src/delimiter/delimiter.rb @@ -25,7 +25,7 @@ class Delimiter next end - %x(#{LKP_SRC}/sbin/submit bad_job_id=#{task['job_id']} error_id=#{task['error_id'].inspect} bisect.yaml) + %x(#{LKP_SRC}/sbin/submit bad_job_id=#{task['job_id']} error_id=#{task['error_id'].inspect} bisect.yaml queue='dc-bisect') rescue StandardError => e puts e sleep(60) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 49d48ee..de5a302 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -135,10 +135,16 @@ module Utils job['my_email'] = account_info['my_email'] job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id - job['testbox'] = job['config'] ? 'vm-2p32g' : 'vm-2p16g' + job['testbox'] = job['config'] ? 'dc-16g' : 'dc-8g' + job['os'] = 'openeuler' + job['os_arch'] = 'aarch64' + job['os_mount'] = 'container' + job['os_version'] = '20.03-pre' + job['docker_image'] = 'openeuler:20.03-pre' - job.delete('error_ids') job.delete('start_time') + job.delete('tbox_group') + job.delete('error_ids') job.delete('job_state') job.delete('end_time') job.delete('loadavg') -- 2.23.0

1 0

[PATCH v4 compass-ci] doc/tutorial.md: fix abnormal display in markdown
by Lu Kaiyi 31 Dec '20

31 Dec '20

the middle part in * and * is italics for markdown document, if want show '*' in this condition, add ` in both sides of path. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- doc/tutorial.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..bf72630 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -258,7 +258,7 @@ job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAM 4) 备注目前提供 archlinux 下已有 PKGBUILD 项目构建测试，暂不支持自主导入 PKGBUILD 文件。 - PKGBUILD 存放路径： /srv/git/archlinux/*/*/PKGBUILD。 + PKGBUILD 存放路径： `/srv/git/archlinux/*/*/PKGBUILD`。 ### depends -- 2.23.0

1 0

[PATCH v2 compass-ci] doc: add runtime field in borrow-*.yml and update borrow-machine.*.md
by Zhang Yale 31 Dec '20

31 Dec '20

User can specify the borrow time in "runtime" field Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 7 ++++--- doc/manual/borrow-machine.zh.md | 9 +++++---- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index bed9fcc..b598bec 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -203,6 +203,7 @@ Ensure that you have performed the following operations according to the [apply- suite: borrow testcase: borrow + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 3f500e2..5288377 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # 使用vim来修改你的sleep字段的值 + # 使用vim来修改你的runtime字段的值 hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # 修改完毕后重新submit即可 hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

2 1

[PATCH v3 compass-ci] sparrow/4-docker/buildall: build dist image when it doesn't exist
by Xu Xijian 31 Dec '20

31 Dec '20

If the dist docker image doesn't exist for some reason, run build script to create it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 32801d1..ca4406b 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -32,7 +32,8 @@ do_one() ( cd $container [ "$container_name" == 'ssh-r' ] && exit - if [ "$action" != "reboot" ]; then + docker images | grep -wq "$container_name" + if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/4-docker/buildall: build dist image when it doesn't exist
by Xu Xijian 31 Dec '20

31 Dec '20

If the dist docker image doesn't exist for some reason, run build script to create it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 32801d1..9e66ac9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -32,7 +32,8 @@ do_one() ( cd $container [ "$container_name" == 'ssh-r' ] && exit - if [ "$action" != "reboot" ]; then + docker images | grep -w -q "$container_name" + if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

1 1

[PATCH lkp-tests] tests/build-pkg: fix to generate the directory of *.cgz file
by Liu Shaofei 31 Dec '20

31 Dec '20

[why] Since the file upload path saved by ES is inconsistent with the file path generated by the build-pkg script, the *.cgz file upload failed. [error] ==> /tmp/stdout <== renamed '/initrd/build-pkg/nfs/openeuler/aarch64/20.03-SP1/kubelet/HEAD.cgz.tmp' -> '/initrd/build-pkg/nfs/openeuler/aarch64/20.03-SP1/kubelet/HEAD.cgz' ==> Leaving fakeroot environment. ==> Finished making: kubernetes 1.18.5-1 (Wed Dec 30 16:50:17 CST 2020) create soft link: latest.cgz -> HEAD.cgz ==> /tmp/stderr <== curl: (22) The requested URL returned error: 403 Forbidden Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/build-pkg b/tests/build-pkg index 801649cf4..e21397ee6 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -88,6 +88,9 @@ request_pkg() create_cgz_name() { + pkgname="${pkgname}" + [ -n "${pkgbase}" ] && pkgname="${pkgbase}" + cgz_path="$PKG_MNT/${pack_to}/${pkgname}" cgz_name="${cgz_path}/${upstream_commit}.cgz" pkg_args="-A --check --skippgpcheck" -- 2.23.0

3 5

[PATCH compass-ci] doc: add runtime field in borrow-*.yml and update borrow-machine.*.md
by Zhang Yale 31 Dec '20

31 Dec '20

User can specify the borrow time in "runtime" field Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 7 ++++--- doc/manual/borrow-machine.zh.md | 9 +++++---- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index bed9fcc..b598bec 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -203,6 +203,7 @@ Ensure that you have performed the following operations according to the [apply- suite: borrow testcase: borrow + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 3f500e2..5288377 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v3 lkp-tests] optimize the deploy-cci script
by Hu Xuejiao 31 Dec '20

31 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- tests/deploy-cci | 52 ++++++++++++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 22 deletions(-) diff --git a/tests/deploy-cci b/tests/deploy-cci index 55835544..8e6acbac 100755 --- a/tests/deploy-cci +++ b/tests/deploy-cci @@ -2,30 +2,38 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -# redirect remote-repository address to "/srv/git/gitee.com" -if [ -n "$GIT_SERVER" ]; then - cat >> /etc/gitconfig <<-EOF - [url "git://$GIT_SERVER/gitee.com"] - insteadOf=https://gitee.com - EOF -fi +export_env() +{ + export initrd_http_host=$INITRD_HTTP_HOST + export initrd_http_port=$INITRD_HTTP_PORT + export gitcache_host=$GITCACHE_HOST + export gitcache_port=$GITCACHE_PORT +} -export initrd_http_host=$INITRD_HTTP_HOST -export initrd_http_port=$INITRD_HTTP_PORT -export gitcache_host=$GITCACHE_HOST -export gitcache_port=$GITCACHE_PORT +deploy() +{ + # redirect remote-repository address to "/srv/git/gitee.com" + if [ -n "$GIT_SERVER" ]; then + cat >> /etc/gitconfig <<-EOF + [url "git://$GIT_SERVER/gitee.com"] + insteadOf=https://gitee.com + EOF + fi -mkdir -p /c -cd /c || exit -git clone https://gitee.com/wu_fengguang/compass-ci.git + mkdir -p /c && cd /c + git clone https://gitee.com/wu_fengguang/compass-ci.git -cd /c/compass-ci || exit -[ -n "$compass_commitid" ] && git reset --hard "$compass_commitid" + cd /c/compass-ci || exit + [ -n "$compass_commit_id" ] && git reset --hard "$compass_commit_id" -# Need mount tmpfs to fix compass-ci/container/os-nfs/start error -# "exportfs: /exports/result does not support NFS export" -mount -t tmpfs none /srv + # Need mount tmpfs to fix compass-ci/container/os-nfs/start error + # "exportfs: /exports/result does not support NFS export" + mount -t tmpfs none /srv -# test the script for one-click deployment -cd /c/compass-ci/sparrow || exit -./install-tiny + # test the script for one-click deployment + cd /c/compass-ci/sparrow || exit + ./install-tiny +} + +export_env +deploy -- 2.23.0

1 0

[PATCH v3 lkp-tests 2/2] pkg: add httpress
by Wei Jihui 31 Dec '20

31 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/httpress/PKGBUILD | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 pkg/httpress/PKGBUILD diff --git a/pkg/httpress/PKGBUILD b/pkg/httpress/PKGBUILD new file mode 100644 index 00000000..8891d6c7 --- /dev/null +++ b/pkg/httpress/PKGBUILD @@ -0,0 +1,17 @@ +pkgname=httpress +pkgver=1 +pkgrel=1 +url='http://ftp.us2.freshrpms.net/linux/RPM/epel/7/aarch64/Packages/h/httpress-1…' +arch=('aarch64') +license=('BSD') +source=("https://download.fedoraproject.org/pub/epel/7/SRPMS/Packages/h/httpress-1.1…") +md5sums=('SKIP') + +package() +{ + # The rpm from source maybe not be available + # httpress-1.1.0-13.fc31.aarch64.rpm has been validated to be available + rpmpkg_path="${pkgdir}/opt/rpms/" + mkdir -p "${rpmpkg_path}" + cp "${srcdir}/"*.rpm "${rpmpkg_path}" +} -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/2] distro/depends: add httpress
by Wei Jihui 31 Dec '20

31 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/httpress | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 distro/depends/httpress diff --git a/distro/depends/httpress b/distro/depends/httpress new file mode 100644 index 00000000..73fc57c6 --- /dev/null +++ b/distro/depends/httpress @@ -0,0 +1,2 @@ +numactl +ethtool -- 2.23.0

1 0

[PATCH compass-ci] lib/es_query.rb: parse es_result to a Hash result
by Lu Kaiyi 31 Dec '20

31 Dec '20

[why] make es result more easy access. [example] input: result_hash = {} es_result = [{"key"=>"build-pkg", "doc_count"=>354526, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>5708}, {"key"=>"finished", "doc_count"=>1033}, {"key"=>"incomplete", "doc_count"=>204}, {"key"=>"submit", "doc_count"=>136}, ... output: result_hash = {"build-pkg"=> {"failed"=>5708, "finished"=>1033, "incomplete"=>204, "submit"=>136, "OOM"=>11, "post_run"=>2}, "cci-depends"=> {"finished"=>1785, "failed"=>675, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 29d50e9..dcd735b 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -129,7 +129,9 @@ class ESQuery result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] return nil if result.empty? - result + result_hash = {} + parse_fields(result, result_hash) + result_hash end end @@ -217,3 +219,46 @@ def build_aggs_from_fields(fields, aggs_hash) build_aggs_from_fields(fields, aggs_hash['aggs']["all_#{field}"]) end + +# input: +# result_hash = {} +# es_result = +# [{"key"=>"build-pkg", +# "doc_count"=>354526, +# "all_job_state"=> +# {"doc_count_error_upper_bound"=>0, +# "sum_other_doc_count"=>0, +# "buckets"=> +# [{"key"=>"failed", "doc_count"=>5708}, +# {"key"=>"finished", "doc_count"=>1033}, +# {"key"=>"incomplete", "doc_count"=>204}, +# {"key"=>"submit", "doc_count"=>136}, +# ... +# +# output: +# result_hash = +# {"build-pkg"=> +# {"failed"=>5708, +# "finished"=>1033, +# "incomplete"=>204, +# "submit"=>136, +# "OOM"=>11, +# "post_run"=>2}, +# "cci-depends"=> +# {"finished"=>1785, +# "failed"=>675, +# ... +def parse_fields(es_result, result_hash) + es_result.each do |result| + key = result['key'] + sub_field = result.keys.detect { |field| field.start_with?('all_') } + + if sub_field + result_hash[key] = {} + all_field = result[sub_field]['buckets'] + parse_fields(all_field, result_hash[key]) + else + result_hash[key] = result['doc_count'] + end + end +end -- 2.23.0

2 1

[PATCH compass-ci] matrix2.rb: filter groups to make compare faster
by Li Yuanchao 31 Dec '20

31 Dec '20

When do compare, after group jobs, there are many one_size group, which can not find another job to make a group. Delete such groups to save time. But, if there is no size > 1 group, 3 one_size groups will be kept to avoid 'No Data'. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/matrix2.rb | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 14e9558..0e80858 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -123,18 +123,37 @@ end def combine_group_query_data(job_list, dims) suites_hash = {} groups = auto_group(job_list, dims) + have_multi_member = multi_member?(groups) + one_size_count = 0 groups.each do |group_key, value| - suite_list = [] - value.each do |dimension_key, jobs| - groups[group_key][dimension_key], suites = create_matrix(jobs) - suite_list.concat(suites) + if value.size < 2 + one_size_count += 1 + if have_multi_member || one_size_count > 3 + groups.delete(group_key) + next + end end - suites_hash[group_key] = suite_list + get_groups_matrix(groups, group_key, value, suites_hash) end - return groups, suites_hash end +def get_groups_matrix(groups, group_key, value, suites_hash) + suite_list = [] + value.each do |dimension_key, jobs| + groups[group_key][dimension_key], suites = create_matrix(jobs) + suite_list.concat(suites) + end + suites_hash[group_key] = suite_list +end + +def multi_member?(groups) + groups.each do |_key, value| + return true if value.size > 1 + end + return false +end + # input: # 1. query results(job_list) from es_query that will be auto group by auto_group_by_template() # 2. params from user's template include: -- 2.23.0

2 3