December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] auto_submit.yaml: add os archlinux for buildpkg
by Li Yuanchao 10 Dec '20

10 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index b53b647..24e524b 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -5,4 +5,5 @@ c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=initramfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

3 2

[PATCH compass-ci] container/git-mirror: add openssh
by Li Yuanchao 10 Dec '20

10 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/git-mirror/Dockerfile b/container/git-mirror/Dockerfile index 0828be5..8b2a50a 100644 --- a/container/git-mirror/Dockerfile +++ b/container/git-mirror/Dockerfile @@ -13,7 +13,7 @@ RUN apk update && \ echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ apk upgrade && \ apk add --no-cache git && \ - apk add ruby-dev make gcc g++ + apk add ruby-dev make gcc g++ openssh RUN umask 002 && \ gem install bunny json PriorityQueue elasticsearch --no-rdoc --no-ri -- 2.23.0

1 0

[PATCH lkp-tests] tests/*: use curl to upload cgz file
by Wang Yong 10 Dec '20

10 Dec '20

mount directory need OS install nfs-utils/cifs-utils firstly, now only use curl to upload cgz file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/build-pkg | 12 ------------ tests/cci-depends | 7 ------- tests/cci-makepkg | 14 -------------- 3 files changed, 33 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 8a138055f..fcb10f11f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -18,7 +18,6 @@ ## See: https://wiki.archlinux.org/index.php/Makepkg . $LKP_SRC/lib/debug.sh -. $LKP_SRC/lib/env.sh . $LKP_SRC/distro/common check_vars() @@ -31,13 +30,6 @@ check_vars() [ -n "$os_mount" ] || die "os_mount is empty" } -mount_pkg_mnt() -{ - [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" - } -} - mount_dest() { # the same image is mounted to cifs and nfs, the generated cgz files @@ -47,8 +39,6 @@ mount_dest() PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" - - is_docker || mount_pkg_mnt } get_pkgfile() @@ -140,8 +130,6 @@ build_source_pkg() [ "$?" == 0 ] || exit 1 create_softlink - - is_docker || exit 0 upload_to_target_dir ${PKG_MNT}/${pack_to}/${pkgname} } diff --git a/tests/cci-depends b/tests/cci-depends index ffa585b8c..42d5ef676 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -7,7 +7,6 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/misc-base.sh -. $LKP_SRC/lib/mount.sh [ -n "$benchmark" ] || die "benchmark is empty" [ -n "$os_mount" ] || die "os_mount is empty" @@ -29,11 +28,6 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO -is_docker || { - is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" -} - umask 002 [[ -d "$pack_to" ]] || { @@ -62,5 +56,4 @@ fixup_arch_install $pack_arch pack_benchmark_deps -is_docker || exit 0 upload_to_target_dir ${pack_to} diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 0354af89d..a50a40c20 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -13,7 +13,6 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/misc-base.sh . $LKP_SRC/lib/env.sh -. $LKP_SRC/lib/mount.sh [ -n "$benchmark" ] || die "benchmark is empty" [ -n "$os_mount" ] || die "os_mount is empty" @@ -34,17 +33,6 @@ pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" -mount_pkg_mnt() -{ - is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" -} - -is_docker || { - [ -n "$LKP_SERVER" ] && mount_pkg_mnt -} - - curl_pkgfile() { curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ @@ -148,8 +136,6 @@ build_source_pkg() [ "$?" == 0 ] || exit 1 update_softlink - - is_docker || exit 0 upload_to_target_dir ${sync_dest} } -- 2.23.0

1 0

[PATCH compass-ci] job: rename pub_key to ssh_pub_key
by Wu Fengguang 10 Dec '20

10 Dec '20

As top level job key, the name should be more exact. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/borrow-machine.en.md | 2 +- doc/manual/borrow-machine.zh.md | 2 +- src/lib/job.cr | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index 4c3b348..b9c2bc4 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -203,7 +203,7 @@ Ensure that you have performed the following operations according to the [apply- suite: borrow testcase: borrow - pub_key: <%= + ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index d5f9c50..06d4db6 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -207,7 +207,7 @@ suite: borrow testcase: borrow - pub_key: <%= + ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/src/lib/job.cr b/src/lib/job.cr index 7746889..7ded6d8 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -199,7 +199,7 @@ class Job end private def set_sshr_info - return unless @hash["pub_key"]? + return unless @hash["ssh_pub_key"]? self["sshr_port"] = ENV["SSHR_PORT"] self["sshr_port_base"] = ENV["SSHR_PORT_BASE"] @@ -211,7 +211,7 @@ class Job end private def set_my_ssh_pubkey - pub_key = @hash["pub_key"]?.to_s + pub_key = @hash["ssh_pub_key"]?.to_s update_account_my_pub_key(pub_key) @hash["my_ssh_pubkey"] = @account_info["my_ssh_pubkey"] -- 2.23.0

1 0

[PATCH compass-ci] delimiter: submit a job to handle the delimiter task
by Cao Xueliang 10 Dec '20

10 Dec '20

Before: we call the function to handle the delimiter task. After: we submit a job to handle the delimiter task. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter.rb | 17 +++++------------ src/delimiter/constants.rb | 2 +- src/delimiter/delimiter.rb | 12 +++--------- 3 files changed, 9 insertions(+), 22 deletions(-) diff --git a/src/delimiter.rb b/src/delimiter.rb index bb73c5f..bd40775 100644 --- a/src/delimiter.rb +++ b/src/delimiter.rb @@ -2,18 +2,11 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -require_relative './delimiter/constants' require_relative './delimiter/delimiter' -START_PROCESS_COUNT.times do - begin - Process.fork do - delimiter = Delimiter.new - delimiter.start_delimit - end - rescue StandardError => e - puts e - end +begin + delimiter = Delimiter.new + delimiter.start_delimit +rescue StandardError => e + puts e end - -sleep() diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index c243df5..ebf90cc 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -2,7 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -START_PROCESS_COUNT = 10 +LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' diff --git a/src/delimiter/delimiter.rb b/src/delimiter/delimiter.rb index 67a621d..3dcb3de 100644 --- a/src/delimiter/delimiter.rb +++ b/src/delimiter/delimiter.rb @@ -21,20 +21,14 @@ class Delimiter # consume delimiter task queue task = consume_delimiter_queue unless task - sleep(2) + sleep(60) next end - # find first bad commit based on the task - git_bisect = GitBisect.new task - result = git_bisect.find_first_bad_commit - - # send mail - mbr = MailBisectResult.new result - mbr.create_send_email + %x(#{LKP_SRC}/sbin/submit job_id=#{task['job_id']} error_id=#{task['error_id']}) rescue StandardError => e puts e - sleep(30) + sleep(60) end end end -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: fix URLs in files
by Wu Fengguang 10 Dec '20

10 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 4 ++-- doc/manual/borrow-machine.en.md | 4 ++-- doc/manual/borrow-machine.zh.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index c1b5206..d9a03c8 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -36,7 +36,7 @@ Which contains account information for you: ## step 4: try it out -Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… ## notice: @@ -44,4 +44,4 @@ In case that you received a failed email with message as below: The repo url for your commit is not in our upstream-repo list. -Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… +Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index e177269..4c3b348 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -1,6 +1,6 @@ # Prerequisites -Ensure that you have performed the following operations according to the [apply-for-an-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/mas…: +Ensure that you have performed the following operations according to the [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc…: - Send an email to apply for an account. - Receive an email from compass-ci(a)qq.com. @@ -223,7 +223,7 @@ Ensure that you have performed the following operations according to the [apply- * Guide to the **submit** Command - Reference: [submit Command Description.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… + Reference: [submit-job.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma… * What Are the testbox Options? diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 578bfe3..d5f9c50 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -227,7 +227,7 @@ * Submit命令指导 - 参考文档：[submit命令详解.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/… + 参考文档：[submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… * testbox有什么可选项 -- 2.23.0

1 0

[PATCH lkp-tests 2/3] iso2rootfs: add three required fields when submit test job.yaml
by Yu Chuan 10 Dec '20

10 Dec '20

[Why] Since the ability `verify account information` was added on submit job.yaml, so add three required fields when submit test job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 52 ++++++++++++++++++++++++++++++++++---------- tests/iso2rootfs | 14 ++++++++++-- 2 files changed, 52 insertions(+), 14 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 9123bef24ae5..c67faf44147d 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,35 +1,63 @@ suite: iso2rootfs category: benchmark iso2rootfs: + ################# # iso related fields to be used to generate rootfs + ################# + iso_os: iso_arch: iso_version: + ################# # place the result rootfs related fields - ## 1. result rootfs will be placed in the following localtion on the - ## remote file server: - ## - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ - ## 2. remote file server protocols current supported: - ## - nfs - ## - cifs + ################# + + # 1. Result rootfs will be placed in the following location on the + # remote file server: + # - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ + # 2. Remote file server protocols current supported: + # - nfs + # - cifs rootfs_protocol: rootfs_server: rootfs_path: + ################# # config rootfs related fields - ## supported fields: - ## - dns: will config /etc/resolv.conf. - ## - no_selinux: will disable selinux in /etc/selinux/config. - ## - no_fstab: will comment all line in /etc/fstab. + ################# + + # you can config add some configurations of the result rootfs. + # supported fields: + # - dns: will config /etc/resolv.conf. + # - no_selinux: will disable selinux in /etc/selinux/config. + # - no_fstab: will comment all line in /etc/fstab. config_rootfs: + ################# # dailybuild related fields + ################# + + # dailybuild_iso_url_file: + # - The `dailybuild iso url file` content is the url of a iso. + # - The iso checksum file also exists on the network, and the checksum file path is "{dailybuild_iso_url}.check256sum" + # - demo: + # root@localhost ~% curl http://1.1.1.1/dailybuilds/openEuler-20.03-LTS-aarch64-LTS/release_iso + # http://1.1.1.1//dailybuilds/1970-01-01-00-00-00/openEuler-20.03-LTS-aarch64… dailybuild_iso_url_file: + ################# # submit test yaml related fields - ## 1. you can add as many jobs as you like. - ## 2. the following three fields is requied for every test job. + ################# + + # three required fields when submit test yaml + submit_email: + submit_name: + submit_uuid: + + # submit target tests yaml + ## 1. You can add as many jobs as you like. + ## 2. The following three fields is required for every test job. test1_yaml: test1_os_mount: test1_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index c03235eb2d2b..dbb409d16923 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -448,11 +448,21 @@ EOF ############ submit test job yaml ############ generate_global_yaml() { + [ -n "${submit_email}" ] || die "cannot find value of var: submit_email" + [ -n "${submit_name}" ] || die "cannot find value of var: submit_name" + [ -n "${submit_uuid}" ] || die "cannot find value of var: submit_uuid" + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" mkdir -p "$(dirname "${config_yaml}")" - echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" - echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" + + cat <<-EOF > "${config_yaml}" + SCHED_HOST: ${SCHED_HOST} + SCHED_PORT: ${SCHED_PORT} + my_email: ${submit_email} + my_name: ${submit_name} + my_uuid: ${submit_uuid} + EOF } generate_submit_append_str() -- 2.23.0

1 0

[PATCH lkp-tests] jobs/rpmbuild-pkg.yaml: change the rpm_repo field to level-2 parameter
by Zhang Yale 10 Dec '20

10 Dec '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/rpmbuild-pkg.yaml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/jobs/rpmbuild-pkg.yaml b/jobs/rpmbuild-pkg.yaml index 8b5fc39b..19fd5ec2 100644 --- a/jobs/rpmbuild-pkg.yaml +++ b/jobs/rpmbuild-pkg.yaml @@ -1,6 +1,5 @@ suite: rpmbuild-pkg category: functional -rpm_repo: - rpmbuild-pkg: + rpm_repo: -- 2.23.0

1 0

[PATCH compass-ci 4/4] docker-rootfs: list of packages that need to install
by Wang Chenglong 10 Dec '20

10 Dec '20

write the software in this file, it will install into rootfs. the kernel and kernel-devel is necessary. They provide the default kernel as well as kernel source code. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/packages-to-install | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 container/docker-rootfs/packages-to-install diff --git a/container/docker-rootfs/packages-to-install b/container/docker-rootfs/packages-to-install new file mode 100644 index 0000000..d7e890f --- /dev/null +++ b/container/docker-rootfs/packages-to-install @@ -0,0 +1,3 @@ +kernel +kernel-devel +openssh-server -- 2.23.0

2 1

Re: [PATCH v2 lkp-tests] daemon/sshd: remove $testbox and format adjustment
by Zhang Yale 10 Dec '20

10 Dec '20

On Wed, Dec 09, 2020 at 05:09:42PM +0800, Li Ping wrote: >On Wed, Dec 09, 2020 at 03:09:06PM +0800, Zhang Yale wrote: >>Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> >>--- >> daemon/sshd | 12 +++++++----- >> 1 file changed, 7 insertions(+), 5 deletions(-) >> >>diff --git a/daemon/sshd b/daemon/sshd >>index c7d7b60c..db013005 100755 >>--- a/daemon/sshd >>+++ b/daemon/sshd >>@@ -8,8 +8,8 @@ >> . $LKP_SRC/lib/http.sh >> >> sshr_ip=$LKP_SERVER >>-[ -n "$sshr_port" ] || sshr_port=5050 >>-[ -n "$sshr_port_len" ] || sshr_port_len=2000 >>+[ -n "$sshr_port" ] || sshr_port=5050 >>+[ -n "$sshr_port_len" ] || sshr_port_len=2000 >> [ -n "$sshr_port_base" ] || sshr_port_base=50000 >> >> run_ssh() >>@@ -24,11 +24,12 @@ run_ssh() >> data_success() >> { >> data="To: $my_email >>-Subject: [NOTIFY Compass-ci] $testbox ready to use >>+Subject: [NOTIFY Compass-ci] Applying machine success > > Applying for a machine ... ok Thanks, Yale > >> >> Dear $my_username: >>+ >> Thanks for your participation in software ecosystem! >>- According to your application, $testbox has been provisioned. >>+ According to your application, the test machine has been provisioned. >> The datails are as follows: >> >> Login: >>@@ -50,9 +51,10 @@ Compass-Ci >> data_failure() >> { >> data="To: $my_email >>-Subject: [NOTIFY Compass-ci] Applying $testbox failed >>+Subject: [NOTIFY Compass-ci] Applying machine failed >> >> Dear $my_username: >>+ >> Sorry to inform you that your application failed. >> You may need to wait a moment, or check whether your pub_key exists. >> >>-- >>2.23.0 >>

1 0