January 2021 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 3/3] distro/depends: add monitor-job depends
by Wei Jihui 30 Jan '21

30 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/monitor-job | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 distro/depends/monitor-job diff --git a/distro/depends/monitor-job b/distro/depends/monitor-job new file mode 100644 index 00000000..5bfafa87 --- /dev/null +++ b/distro/depends/monitor-job @@ -0,0 +1,3 @@ +git +ruby-dev +build-essential -- 2.23.0

1 0

[PATCH lkp-tests 2/3] jobs: add monitor-job.yaml
by Wei Jihui 30 Jan '21

30 Jan '21

[usage] add this to CCI_SRC/sbin/auto_submit.yaml, like: repo: - submit monitor-job.yaml os=... monitor_repo=x job_yaml=x job_testbox=x Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- jobs/monitor-job.yaml | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 jobs/monitor-job.yaml diff --git a/jobs/monitor-job.yaml b/jobs/monitor-job.yaml new file mode 100644 index 00000000..9b4d4d69 --- /dev/null +++ b/jobs/monitor-job.yaml @@ -0,0 +1,7 @@ +suite: monitor-job +category: functional + +monitor-job: + monitor_repo: + job_yaml: + job_testbox: -- 2.23.0

1 0

[PATCH lkp-tests 1/3] tests: add monitor-job
by Wei Jihui 30 Jan '21

30 Jan '21

monitor-job is for monitor user repos(monitor_repo), if user's repos has new commit, auto submit target job yaml(job_yaml). Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/monitor-job | 63 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100755 tests/monitor-job diff --git a/tests/monitor-job b/tests/monitor-job new file mode 100755 index 00000000..5fd2d73c --- /dev/null +++ b/tests/monitor-job @@ -0,0 +1,63 @@ +#!/bin/sh +# - monitor_repo +# - job_yaml +# - job_testbox + +. $LKP_SRC/lib/log.sh + +set_scheduler_config() +{ + mkdir -p /etc/compass-ci/defaults + cat > /etc/compass-ci/defaults/$lab.yaml <<-EOF + SCHED_HOST: $SCHED_HOST + SCHED_PORT: $SCHED_PORT + lab: $lab + my_email: $my_email + EOF +} + +install_gem_pkgs() +{ + gem sources -c -a https://gems.ruby-china.com/ + gem install activesupport git rest-client elasticsearch +} + +build_my_info() +{ + git clone git://$GIT_SERVER/gitee.com/wu_fengguang/compass-ci.git "$CCI_SRC" + ruby "$CCI_SRC"/sbin/build-my-info.rb -e "$my_email" -n "$my_name" -l "$lab" + + git clone git://$GIT_SERVER/gitee.com/wu_fengguang/lkp-tests.git "$LKP_SRC" +} + +download_monitor_repo() +{ + if [ -n "$monitor_repo" ]; then + git clone git://"$LKP_SERVER"/"$monitor_repo" + else + log_error "monitor_repo is not specified" && exit 1 + fi + + test_code=${monitor_repo##*/} + cp -r "$test_code"/* $LKP_SRC/ +} + +submit_job() +{ + yaml=${job_yaml##*/} + "$LKP_SRC"/sbin/submit -a "$yaml" testbox="$job_testbox" +} + +main() +{ + local CCI_SRC=/c/compass-ci + local LKP_SRC=/c/lkp-tests + + install_gem_pkgs + set_scheduler_config + build_my_info + download_monitor_repo + submit_job +} + +main -- 2.23.0

1 0

[PATCH compass-ci] update progress
by Wang Chenglong 30 Jan '21

30 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..b354700 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,22 +1,22 @@ ``` -os os_arch os_version supported-os_mount -openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, - mainline-dailybuild cifs/nfs, +os os_arch os_version supported-os_mount +openeuler aarch64 1.0 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs + mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, 20.03-sp1-dailybuild cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, -centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, +centos aarch64 7 container + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initfamfs x86 sid cifs/nfs, -archlinux aarch64 2020-11-12 cifs/nfs, +archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

1 0

[PATCH compass-ci] scheduler/find_job_boot: make sure of consuming delimiter@localhost subqueue at first
by Cao Xueliang 29 Jan '21

29 Jan '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/find_job_boot.cr | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index f8d7caa..9e2898e 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -75,13 +75,15 @@ class Sched end idle_queues = [] of String + delimiter_queues = [] of String default_queues.each do |queue| idle_queues << "#{queue}/idle" + delimiter_queues << "#{queue}/delimiter@localhost" end - all_queues = rand_queues(sub_queues) + idle_queues + all_queues = delimiter_queues + rand_queues(sub_queues) + idle_queues - return all_queues + return all_queues.uniq end def get_job_from_queues(queues, testbox) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: add how to test OSS project
by Wu Fengguang 29 Jan '21

29 Jan '21

Some people apply account by just adding his personal project's git url to upstream-repos. Which is not enough -- we only offer free account to projects that are in our test coverage. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 3e253d0..fda2eba 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -3,7 +3,7 @@ You'll need an account to submit jobs. The account is mainly for tracking your resource consumptions. We have limited pool of testboxes, so there have to be some usage control. -We offer free accounts to OSS (Open Source Software) contributors. +We offer free accounts to OSS (Open Source Software) contributors where the OSS project is in our test coverage. You can prove yourself as OSS contributor by providing "my oss commit" in the below email. We offer free accounts to our collaborators too. If you are one of our @@ -26,10 +26,15 @@ Email template: --- - please replace the 3 parts in {{ }} with your information. + - the email name should be in English or Chinese Pinyin, as: "David Rientjes" in the template. + - my oss commit: a git commit URL that has your name and email. The git repo should be in our test coverage and registered in [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + If your contributed OSS project is not in upstream-repos, you may refer to + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + to add it to compass-ci testing. ## step 2: receive an email -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow/install-tiny: skip steps to deploy compass-ci server
by Liu Yinsi 29 Jan '21

29 Jan '21

[how] before: execute "./install-tiny", locally deploy compass-ci, can submit job, run containers services, run qemu test. after: execute "./install-tiny server reboot", locally deploy compass-ci server, just run containers servers. [why] some user who locally deploy compass-ci, the host which just run containers can only access intranet, so skip some steps that need to access internet when deploy, can submit job and run qemu test will in other machine, use $LKP_SRC/setup/simply-ci to deploy. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-tiny | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/sparrow/install-tiny b/sparrow/install-tiny index cb47cf5..414c5bc 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -4,10 +4,19 @@ # For your reference only. # It's better to run the below scripts step by step. +export server=$1 +export action=$2 + +server_skip() +{ + local deploy_script=$1 + [ "$server" != "server" ] && ${deploy_script} +} + [[ $CCI_SRC ]] || export CCI_SRC=$(cd $(dirname $(realpath $0)); git rev-parse --show-toplevel) cd $CCI_SRC/sparrow || exit -0-package/install +server_skip 0-package/install 0-package/common 1-storage/tiny 1-storage/permission @@ -15,9 +24,9 @@ cd $CCI_SRC/sparrow || exit 2-network/iptables 2-network/nfs 2-network/cifs -3-code/git +server_skip 3-code/git 3-code/dev-env . /etc/profile.d/compass.sh -4-docker/buildall -5-build/ipxe +4-docker/buildall $action +server_skip 5-build/ipxe 7-systemd/systemd-setup -- 2.23.0

3 7

[PATCH v3 compass-ci 2/2] sparrow: call download-rootfs script
by Liu Yinsi 29 Jan '21

29 Jan '21

For locall deploy compass-ci, call $CCI_SRC/sbin/download-rootfs to download default os rootfs openeuler aarch64 20.03. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/os-ready | 5 +++++ sparrow/install-tiny | 1 + 2 files changed, 6 insertions(+) create mode 100755 sparrow/5-build/os-ready diff --git a/sparrow/5-build/os-ready b/sparrow/5-build/os-ready new file mode 100755 index 0000000..48a740b --- /dev/null +++ b/sparrow/5-build/os-ready @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/download-rootfs diff --git a/sparrow/install-tiny b/sparrow/install-tiny index a291549..5f026e5 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -19,4 +19,5 @@ cd $CCI_SRC/sparrow || exit . /etc/profile.d/compass.sh 4-docker/buildall 5-build/ipxe +5-build/os-ready 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sbin: add script for download os rootfs
by Liu Yinsi 29 Jan '21

29 Jan '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 46 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100644 index 0000000..d5ea328 --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,46 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs + +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +export os=$1 +export os_arch=$2 +export os_version=$3 + +: ${os:=openeuler} +: ${os_arch:=aarch64} +: ${os_version:=20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p /srv/os/${rootfs_dir} + cd /srv/os/$rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${rootfs_dir}.cgz" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

1 0

[PATCH v2 lkp-tests] setup/simplify-ci: add param $server_ip
by Liu Yinsi 29 Jan '21

29 Jan '21

[how] before: all containers services ip is the host locally server ip after: if $server_ip is not null, all containers services ip is $server_ip [why] if container services deploy in other server, add param $server_ip will easy to deploy, rather than change config yaml manually. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- setup/simplify-ci | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/setup/simplify-ci b/setup/simplify-ci index df18272f3..4b26e3bcb 100755 --- a/setup/simplify-ci +++ b/setup/simplify-ci @@ -2,8 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${SCHED_HOST:=172.17.0.1} -: ${SCHED_PORT:=3000} +server_ip=$1 git_ci() { @@ -21,7 +20,7 @@ dev_env() { export sched_host=$SCHED_HOST export sched_port=$SCHED_PORT - 3-code/dev-env + 3-code/dev-env $server_ip } install_env() -- 2.23.0

1 1