- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] lkp-tests/*: fix cci jobs upload in container
by Wang Yong 26 Nov '20

26 Nov '20

docker can not use mount operation, so now use curl to upload cci-depends/cci-makepkg cgz file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 10 ++++++++++ tests/cci-depends | 12 ++++++++++-- tests/cci-makepkg | 22 ++++++++++++++++++---- 3 files changed, 38 insertions(+), 6 deletions(-) diff --git a/distro/common b/distro/common index b48dfe5f1..ad6fc23e5 100755 --- a/distro/common +++ b/distro/common @@ -81,3 +81,13 @@ share_debian_package() exit 0 } + +upload_dest_cgz() +{ + . $LKP_SRC/lib/upload.sh + + local dir="$1" + local file="$(ls -F ${dir} | grep -v "@")" + + upload_one_curl "${dir}/${file}" "${dir}" >/dev/null +} diff --git a/tests/cci-depends b/tests/cci-depends index d5c21541d..087808bce 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -29,8 +29,10 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO -is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} -mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +is_docker || { + is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +} umask 002 @@ -59,3 +61,9 @@ fixup_preinstall fixup_arch_install $pack_arch pack_benchmark_deps + +if [ $? = "0" ] && is_docker; then + upload_dest_cgz ${pack_to} +else + exit 0 +fi diff --git a/tests/cci-makepkg b/tests/cci-makepkg index d08f23718..90fce8541 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -34,11 +34,17 @@ pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" -[ -n "$LKP_SERVER" ] && { - is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +mount_pkg_mnt() +{ + is_mount_point ${PKG_MNT} && umount ${PKG_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +} + +is_docker || { + [ -n "$LKP_SERVER" ] && mount_pkg_mnt } + curl_pkgfile() { curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ @@ -140,7 +146,15 @@ build_source_pkg() PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg - [ "$?" == 0 ] && update_softlink + [ "$?" == 0 ] || exit 1 + + update_softlink + + if is_docker; then + upload_dest_cgz ${sync_dest} + else + exit 0 + fi } build_source_pkg -- 2.23.0

1 0

[PATCH v4 lkp-tests] stats/crystal: add sysbench-mysql.cr
by Zhang Yu 26 Nov '20

26 Nov '20

Make ruby2crystal convert Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- stats/crystal/sysbench-mysql.cr | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100755 stats/crystal/sysbench-mysql.cr diff --git a/stats/crystal/sysbench-mysql.cr b/stats/crystal/sysbench-mysql.cr new file mode 100755 index 00000000..b9b6c8f1 --- /dev/null +++ b/stats/crystal/sysbench-mysql.cr @@ -0,0 +1,18 @@ +#!/usr/bin/env crystal + +# SQL statistics: +# queries performed: +# read: 210212086 +# write: 60060591 +# other: 30030296 +# total: 300302973 +# transactions: 15015147 (25024.26 per sec.) +# queries: 300302973 (500485.16 per sec.) +# ignored errors: 2 (0.00 per sec.) +# reconnects:0 (0.00 per sec.) + +while (line = STDIN.gets) + next unless line =~ /transactions:.*\(\s*(\S+)/ + + puts "transactions: #{$1}" +end -- 2.23.0

1 0

[PATCH compass-ci] doc: add how-to-create-a-PKGBUILD.md
by Liu Shaofei 26 Nov '20

26 Nov '20

Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- doc/how-to-create-a-PKGBUILD.md | 79 +++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 doc/how-to-create-a-PKGBUILD.md diff --git a/doc/how-to-create-a-PKGBUILD.md b/doc/how-to-create-a-PKGBUILD.md new file mode 100644 index 0000000..01e1084 --- /dev/null +++ b/doc/how-to-create-a-PKGBUILD.md @@ -0,0 +1,79 @@ +# 如何编写PKGBUILD + +## PKGBUILD是什么？ + + PKGBUILD是一个shell脚本，makepkg通过PKGBUILD中包含的指令，生成包含二进制文件和安装指令的软件包。 + +## PKGBUILD包含什么？ + + PKGBUILD包含两部分内容：变量和函数 + + 1）、定义变量 + + - pkgname: 必须定义，表示软件包的名称; + - pkgver: 必须定义，表示软件包的版本号； + - pkgrel: 必须定义，表示软件包的发布号； + - arch: 必须定义，表示软件包使用的架构序列； + - depends: 可选字段，软件测试运行时需要的依赖包名称； + - makedepends: 可选字段，构建软件包时需要的文件列表； + - source: 可选字段，指定每个源文件的MD5哈希值，用于构建过程中验证源文件的完整性。 + + 2）、定义函数 + + - package函数 + package函数必须定义，此函数用于将文件安装到将成为构建包的根目录的目录中； + - prepare函数 + 定义可选的prepare函数，在其中执行用于准备构建源码的操作； + - build函数 + 定义可选的build函数，用于编译和/或构建源码； + - check函数 + 定义可选的check函数，用于运行程序包的测试套件。 + + 注意：srcdir 和 pkgdir + + - srcdir: 提取或复制源文件的目录，所有打包功能都在srcdir目录内部运行； + - pkgdir: 构建软件包的根目录，仅在package函数中使用。 + +## 如何编写PKGBUILD？ + + 1）、创建PKGBUILD，文件名必须以“PKGBUILD”命名。 + + `touch PKGBUILD` + + 2）、使用vim打开PKGBUILD，编写PKGBUILD文件内容。 + + 如下我们给出一个PKGBUILD例子： + + pkgname=zstd + pkgver=1.4.4 + pkgrel=2 + arch=('i686' 'x86_64' 'aarch64') + url='https://github.com/facebook/zstd' + license=('custom:BSD3' 'GPL2') + depends=('xz' 'zlib' 'lz4') + makedepends=('git') + source=('git://github.com/facebook/zstd.git#branch=dev') + md5sums=('SKIP') + + pkgver() { + cd "$srcdir/$pkgname" + git describe --long --tags | sed 's/$[^-]*-g$/r\1/;s/-/./g;s/^v//g' + } + + build() { + cd "$srcdir/$pkgname" + make + make -C contrib/pzstd + } + + package() { + cd "$srcdir/$pkgname" + make PREFIX="/usr" DESTDIR="$pkgdir/" install + install -D -m755 contrib/pzstd/pzstd "$pkgdir/usr/bin/pzstd" + install -D -m644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" + } + +## 参考 + + https://www.archlinux.org/pacman/PKGBUILD.5.html + https://git.archlinux.org/pacman.git/plain/proto/PKGBUILD.proto -- 2.23.0

1 0

[PATCH compass-ci] doc: add how-to-create-a-PKGBUILD
by Liu Shaofei 26 Nov '20

26 Nov '20

Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- doc/how-to-create-a-PKGBUILD.md | 81 +++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 doc/how-to-create-a-PKGBUILD.md diff --git a/doc/how-to-create-a-PKGBUILD.md b/doc/how-to-create-a-PKGBUILD.md new file mode 100644 index 0000000..2230b5f --- /dev/null +++ b/doc/how-to-create-a-PKGBUILD.md @@ -0,0 +1,81 @@ +# ��PKGBUILD + +## PKGBUILD�� + + PKGBUILD��shell��makepkg��PKGBUILD�� + +## PKGBUILD�� + + PKGBUILD�� + + 1�� + + - pkgname: ��; + - pkgver: �� + - pkgrel: �� + - arch: �� + - depends: �� + - makedepends: �� + - source: ��MD5�� + + 2�� + + - package�� + package�� + - prepare�� + ��prepare�� + - build�� + ��build��/�� + - check�� + ��check�� + + ��srcdir �� pkgdir + + - srcdir: ��srcdir�� + - pkgdir: ��package�� + +## ��PKGBUILD�� + + 1��PKGBUILD��PKGBUILD�� + + `touch PKGBUILD` + + 2��vim��PKGBUILD��PKGBUILD�� + + ��PKGBUILD�� + + pkgname=zstd + pkgver=1.4.4 + pkgrel=2 + arch=('i686' 'x86_64' 'aarch64') + url='https://github.com/facebook/zstd' + license=('custom:BSD3' 'GPL2') + depends=('xz' 'zlib' 'lz4') + makedepends=('git') + source=('git://github.com/facebook/zstd.git#branch=dev') + md5sums=('SKIP') + + pkgver() { + cd "$srcdir/$pkgname" + git describe --long --tags | sed 's/$[^-]*-g$/r\1/;s/-/./g;s/^v//g' + } + + build() { + cd "$srcdir/$pkgname" + make + make -C contrib/pzstd + } + + package() { + cd "$srcdir/$pkgname" + make PREFIX="/usr" DESTDIR="$pkgdir/" install + install -D -m755 contrib/pzstd/pzstd "$pkgdir/usr/bin/pzstd" + install -D -m644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" + } + +## �� + + https://www.archlinux.org/pacman/PKGBUILD.5.html + https://git.archlinux.org/pacman.git/plain/proto/PKGBUILD.proto + + -- 2.23.0

1 1

[PATCH v2 compass-ci] add a document for adding open source project
by Li Yuanchao 26 Nov '20

26 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- ...-compass-ci-to-test-open-source-project.md | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 doc/how-to-use-compass-ci-to-test-open-source-project.md diff --git a/doc/how-to-use-compass-ci-to-test-open-source-project.md b/doc/how-to-use-compass-ci-to-test-open-source-project.md new file mode 100644 index 0000000..ad4f9b0 --- /dev/null +++ b/doc/how-to-use-compass-ci-to-test-open-source-project.md @@ -0,0 +1,53 @@ +使用compass-ci平台测试开源仓库，需要完成 +1. 将仓库url添加到upstream-repos仓库中 +2. 添加自己的测试case + +### 首先，将想要测试的代码仓信息以yaml格式文件的形式添加到upstream-repos仓库 + +仓库地址为：https://gitee.com/wu_fengguang/upstream-repos + +你可以先fork这个仓库，新建文件之后，通过Pull Request的方式完成添加。 + +以仓库backlight为例： + +它的url为：https://github.com/baskerville/backlight +1. 以首字母和仓库名创建路径 +``` +mkdir -p b/backlight +``` +2. 在该目录下新建同名文件backlight： +``` +cd b/backlight +touch backlight +``` +3. 使用vim写入url： +``` +vim backlight +``` +内容格式为 +``` +--- +url: +- https://github.com/baskerville/backlight +``` +请保持格式一致，若不确定，可参考upstream-repos仓库中已有文件。 + +### 然后，将测试case以yaml文件的形式添加到lkp-tests仓库的jobs目录 + +仓库地址为 https://gitee.com/wu_fengguang/lkp-tests + +这其中有一些已经适配好的测试case，如果你想要测试的case其中正好有，那就可以直接使用。使用方式以iperf为例。 + +iperf是已经适配的一个测试case，在lkp-tests仓库的jobs下有一个iperf.yaml文件，里面有一些测试的基本参数。 + +然后，你要在compass-ci仓库下面的 sbin/auto_submit.yaml 文件中添加 +``` +b/backlight/backlight: +- testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml +``` +并通过Pull Request的方式提交。相关参数含义见参考文档 + +参考文档： +- [compass-ci测试平台使用教程--submit命令详解](https://gitee.com/wu_fengguang/compass-ci/t… +- [如何适配测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-test… + -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/result-webdav: adjust the value of second_dir
by Cui Yili 26 Nov '20

26 Nov '20

[why] 1. when the uri starts with "//", etc., the previous regular expression did not get the second level directory correctly. 2. when the uploaded data package does not meet the format of "xxxx_yyyymmdd.cgz", uploading is also allowed, but no symbolic link. [how] 1. modify the variable second_dir regular expression。 2. make conditional judgments on link_name Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/nginx.conf | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/container/result-webdav/nginx.conf b/container/result-webdav/nginx.conf index c880522..e036e77 100644 --- a/container/result-webdav/nginx.conf +++ b/container/result-webdav/nginx.conf @@ -54,17 +54,21 @@ http { path = string.match(uri,"%g*/") upload_file = string.match(uri, ".+/([^/]*%.%w+)$") - second_dir = string.match(uri, "/.+/(.*)/.+") + second_dir = string.match(uri, "/*%w+/(%w+)/.+") if (second_dir == "pkg") then - link_name = "latest.cgz" + link_name = "latest" else - link_name = string.match(upload_file, "(.*)_%d+%.cgz")..".cgz" + link_name = string.match(upload_file, "(.*)_%d+%.cgz") + end + + if (link_name ~= '') + then + link_name = link_name..".cgz" + root_patch = "/srv/"..path + io.popen("cd "..root_patch.."&& ln -sf "..upload_file.." "..link_name) end - - root_patch = "/srv/"..path - io.popen("cd "..root_patch.."&& ln -sf "..upload_file.." "..link_name) } } -- 2.23.0

1 0

[PATCH compass-ci] doc: add how-to-apply-a-test-env.md
by Yu Chuan 26 Nov '20

26 Nov '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/how-to-apply-a-test-env.md | 252 +++++++++++++++++++++++++++++++++ 1 file changed, 252 insertions(+) create mode 100644 doc/how-to-apply-a-test-env.md diff --git a/doc/how-to-apply-a-test-env.md b/doc/how-to-apply-a-test-env.md new file mode 100644 index 000000000000..a703587f38cd --- /dev/null +++ b/doc/how-to-apply-a-test-env.md @@ -0,0 +1,252 @@ +# 1. 前提条件 +------------- + +- 已向compass-ci服务团队申请账号，且已登录到account-vm服务器上。 + + 参考资料：[apply for jumper account](./apply-ssh-account.md) + + +# 2. 申请测试机（虚拟机） +------------------------ + +## 2.1 生成本地RSA公私钥对 + +```shell +hi684@account-vm ~% ssh-keygen -t rsa +Generating public/private rsa key pair. +Enter file in which to save the key (/home/hi684/.ssh/id_rsa): +Created directory '/home/hi684/.ssh'. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /home/hi684/.ssh/id_rsa. +Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. +The key fingerprint is: +SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm +The key's randomart image is: ++---[RSA 2048]----+ +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| ++----[SHA256]-----+ +hi684@account-vm ~% ls -hla .ssh +total 16K +drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . +drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. +-rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa +-rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub +``` + +## 2.2 根据需求选择yaml + +每位用户家目录下面都存放了一个lkp-tests的文件夹 + +```shell +hi684@account-vm ~% cd lkp-tests/jobs +hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* +-rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml +-rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml +-rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml +``` + +## 2.3 提交yaml并连接测试机（虚拟机） + +```shell +hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml +2020-11-26 14:44:25 +0800 WARN -- skip non-executable /home/hi684/lkp-tests/monitors/plain/wait +submit borrow-1h.yaml, got job_id=z9.170593 +query=>{"job_id":["z9.170593"]} +connect to ws://172.168.131.2:11310/filter +{"job_id":"z9.170593","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593"} +{"job_id": "z9.170593", "result_root": "/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593", "job_state": "set result root"} +{"job_id": "z9.170593", "job_state": "boot"} +{"job_id": "z9.170593", "job_state": "download"} +{"time":"2020-11-26 14:45:06","mac":"0a-1f-0d-3c-91-5c","ip":"172.18.156.13","job_id":"z9.170593","state":"running","testbox":"vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} +{"job_state":"running","job_id":"z9.170593"} +{"job_id": "z9.170593", "state": "set ssh port", "ssh_port": "51840", "tbox_name": "vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} +Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts +Warning: Permanently added '[172.168.131.2]:51840' (ECDSA) to the list of known hosts. +Last login: Wed Sep 23 11:10:58 2020 + + +Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 + +System information as of time: Thu Nov 26 06:44:18 CST 2020 + +System load: 0.83 +Processes: 107 +Memory used: 6.1% +Swap used: 0.0% +Usage On: 89% +IP address: 172.18.156.13 +Users online: 1 + + + +root@vm-2p8g ~# +``` + +更多关于`submit命令如何使用`、`testbox都有什么可选项`、`如何borrow指定的操作系统`，请参见文章末尾Faq。 + +## 2.4 使用完毕退还测试机（虚拟机） + +```shell +root@vm-2p8g ~# reboot +Connection to 172.168.131.2 closed by remote host. +Connection to 172.168.131.2 closed. +hi684@account-vm ~/lkp-tests/jobs% +``` + + +# 3. 申请测试机（物理机） +------------------------ + +## 3.1 生成本地RSA公私钥对 + +```shell +hi684@account-vm ~% ssh-keygen -t rsa +Generating public/private rsa key pair. +Enter file in which to save the key (/home/hi684/.ssh/id_rsa): +Created directory '/home/hi684/.ssh'. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /home/hi684/.ssh/id_rsa. +Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. +The key fingerprint is: +SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm +The key's randomart image is: ++---[RSA 2048]----+ +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| +|xxxxxxxxxxxxxxxxx| ++----[SHA256]-----+ +hi684@account-vm ~% ls -hla .ssh +total 16K +drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . +drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. +-rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa +-rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub +``` + +## 3.2 根据需求选择yaml + +每位用户家目录下面都存放了一个lkp-tests的文件夹 + +```shell +hi684@account-vm ~% cd lkp-tests/jobs +hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* +-rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml +-rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml +-rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml +``` + +## 3.3 提交yaml并连接测试机（物理机） + +```shell +hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=taishan200-2280-2s64p-256g borrow-1h.yaml +2020-11-26 14:49:13 +0800 WARN -- skip non-executable /home/hi684/lkp-tests/monitors/plain/wait +submit borrow-1h.yaml, got job_id=z9.170594 +query=>{"job_id":["z9.170594"]} +connect to ws://172.168.131.2:11310/filter +{"job_id":"z9.170594","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594"} +{"job_id": "z9.170594", "result_root": "/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594", "job_state": "set result root"} +{"job_id": "z9.170594", "job_state": "boot"} +{"job_id": "z9.170594", "job_state": "download"} +{"time":"2020-11-26 14:51:56","mac":"84-46-fe-26-d3-47","ip":"172.168.178.48","job_id":"z9.170594","state":"running","testbox":"taishan200-2280-2s64p-256g--a5"} +{"job_state":"running","job_id":"z9.170594"} +{"job_id": "z9.170594", "state": "set ssh port", "ssh_port": "50420", "tbox_name": "taishan200-2280-2s64p-256g--a5"} +Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts +Warning: Permanently added '[172.168.131.2]:50420' (ECDSA) to the list of known hosts. +Last login: Wed Sep 23 11:10:58 2020 + + +Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 + +System information as of time: Thu Nov 26 14:51:59 CST 2020 + +System load: 1.31 +Processes: 1020 +Memory used: 5.1% +Swap used: 0.0% +Usage On: 3% +IP address: 172.168.178.48 +Users online: 1 + + + +root@taishan200-2280-2s64p-256g--a5 ~# +``` + +更多关于`submit命令如何使用`、`testbox都有什么可选项`、`如何borrow指定的操作系统`，请参见文章末尾Faq。 + +## 3.4 使用完毕退还测试机（物理机） + +```shell +root@taishan200-2280-2s64p-256g--a5 ~# reboot +Connection to 172.168.131.2 closed by remote host. +Connection to 172.168.131.2 closed. +hi684@account-vm ~/lkp-tests/jobs% +``` + +# 4 Faq +------- + +## 4.1 如何自行修改申请时长？ + +```shell +hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml +suite: borrow +testcase: borrow + +sshd: + pub_key: <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> +# sleep at the bottom +sleep: 1h +hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml +sleep: 1h +# 使用vim来修改你的sleep字段的值 +hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml +# 修改完毕后重新submit即可 +hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml +``` + +## 4.2 Submit命令指导 + +参考文档：[compass-ci测试平台使用教程--submit命令详解.md](./compass-ci测试平台使用教程--submit命令详解.md) + +## 4.3 testbox都有什么可选项？ + +testbox可选项见：https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts + +> 说明： +> 虚拟机的testbox: +> - vm-xxx +> +> 物理机的testbox: +> - taishan200-2280-xxx + + +> 注意： +> - 物理机的testbox若选择以`--axx`结尾的，则表示指定到了具体的某一个物理机。若此物理机任务队列中已经有任务在排队，则需要等待队列中前面的任务执行完毕后，才会轮到你提交的borrow任务。 +> - 物理机的testbox若不选择以`-axx`结尾的，表示不指定具体的某一个物理机。则此时集群中的空闲物理机会即时被分配执行你的borrow任务。 + +## 4.4 如何borrow指定的操作系统？ + +关于支持的`os`, `os_arch`, `os_version`，参见：[os-os_verison-os_arch.md](./job/os-os_verison-os_arch.md) -- 2.23.0

1 0

[PATCH v6 lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- .../merge_yaml/merge_included_files.yaml | 1 + .../merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit/merge_yaml/template_yaml | 3 +++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml create mode 100644 spec/submit/merge_yaml/template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files.yaml b/spec/submit/merge_yaml/merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH v3 compass-ci] doc/manual: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...\273\273\345\212\241\347\273\223\346\236\234" | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..7a8cf8b --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,16 @@ +# �� + ��https://compass-ci.openeuler.org/jobs�� + ��suite��id��job state�� + +# ��compass-ci�� +## job.yaml�� + job.yaml��job�� + �� + +## output�� + output��check_exit_code��0�� + +## stats.json�� + �� + compass-ci��.json�� + stats.json��json�� -- 2.23.0

1 0

[PATCH v3 compass-ci] doc/job: add runtime.md
by Zhang Yale 26 Nov '20

26 Nov '20

Add a document for introducing the runtime field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/job/runtime.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 doc/job/runtime.md diff --git a/doc/job/runtime.md b/doc/job/runtime.md new file mode 100644 index 0000000..8e720f8 --- /dev/null +++ b/doc/job/runtime.md @@ -0,0 +1,32 @@ +# runtime + +Meaning: +- Run time for test: seconds(s), hours(h) or days(d). +- Limit the test's run time in $runtime. + +Usage example: + +```bash +#!/bin/sh +# - runtime + +## delay for a specified amount of time +echo sleep started +exec sleep ${1:-$runtime} +echo sleep finished +``` + +`runtime` is a parameter of test script. +For any test case needing parameter(s), +need such comment line and follow exactly the +same format `# - parameter`. + +```yaml +suite: borrow +testcase: borrow + +runtime: 1d + +sleep: +``` + -- 2.23.0

1 0

[PATCH v2 compass-ci] fill 0 if standard_deviation is nil
by Lu Weitao 26 Nov '20

26 Nov '20

[Why] With template compare mode, it's not friendly for echart that return null for nil standard_deviation of a metric like: { "tables": { "vmstat.system.cs": { "standard_deviation": { "dimensions": ["compare_dimension", "openeuler 20.03"], "source": [ ["4G", "8G", "16G"], ["openeuler 20.03", 23, null, null] ] } } } } [How] fill 0 if standard_deviation is nil. Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 1550435..dd9b86e 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -461,7 +461,7 @@ def assign_metric_values(metrics_values, dim, metric, values) metrics_values[metric]['standard_deviation'] ||= {} metric_value = get_values(values, true) metrics_values[metric]['average'][dim] = metric_value[:average] - metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] + metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] || 0 end def assign_metric_change(metrics_values) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...273\273\345\212\241\347\273\223\346\236\234" | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..db6cc75 --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,17 @@ +# �� + ��https://compass-ci.openeuler.org�� + ��id��job state�� + +## job.yaml�� + job.yaml��job�� + �� + +## output�� + output��check_exit_code��0�� + +## �� + �� + +## stats.json�� + ��xxx.json�� + stats.json��json�� -- 2.23.0

3 2

[PATCH v2 compass-ci] doc/manual: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...3\273\345\212\241\347\273\223\346\236\234" | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..855e9be --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,19 @@ +# �� + ��https://compass-ci.openeuler.org�� + ��suite��id��job state�� + +# ��compass-ci�� +## job.yaml�� + job.yaml��job�� + �� + +## output�� + output��check_exit_code��0�� + +## �� + �� + ��iperf,��iperf��iperf�� + +## stats.json�� + ��xxx.json�� + stats.json��json�� -- 2.23.0

1 0

[PATCH compass-ci] kernel_version.md: update kernel information
by Xu Xijian 26 Nov '20

26 Nov '20

1. delete redundant soft links of kernel files I've created before and update this information in kernel_version.md. 2. the kernel version of centos/aarch64/7.6.1810 we use has been changed into 4.14.0-115.el7.0.1.aarch64. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index 3e94ca8..4a193fb 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -6,10 +6,10 @@ Meaning: - If kernel_version is not given by users, it will use the default one. - Here are the kernel versions we support for different os: openeuler/aarch64/1.0 - - 4.19.90-2001(default) + - 4.19.90-vhulk2001.1.0.0026.aarch64(default) openeuler/aarch64/20.03 - - 4.19.90-2003(default) + - 4.19.90-2003.4.0.0036.oe1.aarch64(default) - 4.19.90-mysql - 4.19.90-nginx @@ -18,7 +18,7 @@ Meaning: - 5.8.0-1-arm64 centos/aarch64/7.6.1810 - - 4.14.0-115.el7a.0.1(default) + - 4.14.0-115.el7.0.1.aarch64(default) centos/aarch64/7.8.2003 - 4.18.0-147.8.1.el7(default) @@ -29,10 +29,10 @@ Meaning: Related files: - In initramfs boot process, every kernel version is related with a vmlinuz, module and headers. - Files like below under $boot_dir, an example $boot_dir can be "/srv/os/openeuler/aarch64/20.03/boot". -├── headers-4.19.90-2003.cgz -├── modules-4.19.90-2003.cgz -├── vmlinuz-4.19.90-2003 -└── vmlinuz -> vmlinuz-4.19.90-2003 +├── headers-4.19.90-2003.4.0.0036.oe1.aarch64.cgz +├── modules-4.19.90-2003.4.0.0036.oe1.aarch64.cgz +├── vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 +└── vmlinuz -> vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 Usage example: -- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 +- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003.4.0.0036.oe1.aarch64 -- 2.23.0

1 0

[PATCH compass-ci] fill 0 for nil standard_deviation with template compare mode
by Lu Weitao 26 Nov '20

26 Nov '20

[Why] With template compare mode, it's not friendly for echart that return null for nil standard_deviation of a metric like: { "tables": { "vmstat.system.cs": { "standard_deviation": { "dimensions": ["compare_dimension", "openeuler 20.03"], "source": [ ["4G", "8G", "16G"], ["openeuler 20.03", 23, null, null] ] } } } } [How] fill 0 for nil standard_deviation, instead of "null" Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 1550435..dd9b86e 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -461,7 +461,7 @@ def assign_metric_values(metrics_values, dim, metric, values) metrics_values[metric]['standard_deviation'] ||= {} metric_value = get_values(values, true) metrics_values[metric]['average'][dim] = metric_value[:average] - metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] + metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] || 0 end def assign_metric_change(metrics_values) -- 2.23.0

2 2

[PATCH v3 compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..02161f0 --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,49 @@ +# Introduction of compare feature + The compare feature used to analyze the results of different jobs, showing the performance waves and changes + under different influencing factors, for users to analyze performance influencing factor. + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope. + choose at least one option as filter. + +### dimension + Dimension can select: os, os_version, os_arch, suite, tbox_group. + Within filter, we will compare all different job result by dimension + and keep other test conditions are same. + +## example: + filter: suite = iperf + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

1 0

[PATCH v2 compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..94b139a --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,49 @@ +# Introduction of compare feature + The compare feature used to analyze the results of different jobs, showing the performance waves and changes + under different influencing factors, for users to analyze performance influencing factor. + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope. + choose at least one option as filter. + +### dimension + Dimension can select: os, os_version, os_arch, suite, tbox_group. + Within filter, we will compare all different job result by dimension + and keep other test conditions are same. + +## example: + filter: suite = iperf | os_arch = aarch64 | tbox_group = vm-2p8g + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

1 0

[PATCH compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..24b8f73 --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,46 @@ +# Introduction of compare feature + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope + +### dimension + dimension can select: os, os_version, os_arch, suite, box_group. + within compare scope, we will compare all defferent job result by dimension + and keep other test conditions are same. + +## example: + filter: suite=iperf | os_arch=aarch63 | tbox_group=2p8g + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

3 4

[PATCH v6 lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template_yaml | 3 +++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/template_yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH v3 lkp-tests] spec/submit_spec.rb: submit job_on_fail.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to debug job_on_fail.yaml, and after the task is failed, it is be waiting state. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/job_on_fail/job_on_fail.yaml | 5 +++++ .../job_on_fail/job_on_fail-3000.yaml | 22 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 3 files changed, 31 insertions(+) create mode 100644 spec/submit/job_on_fail/job_on_fail.yaml create mode 100644 spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml diff --git a/spec/submit/job_on_fail/job_on_fail.yaml b/spec/submit/job_on_fail/job_on_fail.yaml new file mode 100644 index 00000000..bf68ff40 --- /dev/null +++ b/spec/submit/job_on_fail/job_on_fail.yaml @@ -0,0 +1,5 @@ +suite: atomic + +on_fail: + sleep: 3000 +atomic: diff --git a/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml b/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml new file mode 100644 index 00000000..3d7ad9b5 --- /dev/null +++ b/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml @@ -0,0 +1,22 @@ +--- +suite: atomic +on_fail: + sleep: 3000 +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + sleep: 3000 + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..52d9a158 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'job on fail' do + submit_job('job_on_fail') + end end -- 2.23.0

1 0

[PATCH compass-ci] add a document for adding open source project
by Li Yuanchao 26 Nov '20

26 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- ...-compass-ci-to-test-open-source-project.md | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 doc/how-to-use-compass-ci-to-test-open-source-project.md diff --git a/doc/how-to-use-compass-ci-to-test-open-source-project.md b/doc/how-to-use-compass-ci-to-test-open-source-project.md new file mode 100644 index 0000000..219cd0c --- /dev/null +++ b/doc/how-to-use-compass-ci-to-test-open-source-project.md @@ -0,0 +1,51 @@ +### 首先，将想要测试的代码仓信息以yaml格式文件的形式添加到upstream-repos仓库 + +仓库地址为：https://gitee.com/wu_fengguang/upstream-repos。 + +你可以先fork这个仓库，新建文件之后，通过Pull Request的方式完成添加。 + +以仓库backlight为例： + +它的url为：https://github.com/baskerville/backlight +1. 以首字母和仓库名创建路径 +``` +mkdir -p b/backlight +``` +2. 在该目录下新建同名文件backlight： +``` +cd b/backlight +touch backlight +``` +3. 使用vim写入url： +``` +vim backlight +``` +内容格式为 +``` +--- +url: +- https://github.com/baskerville/backlight +``` +请保持格式一致，若不确定，可参考upstream-repos仓库中已有文件。 + +pkgbuild_repo字段暂不需要，因为自助添加pkgbuild的功能尚未开放。 + +### 然后，将测试case以yaml文件的形式添加到lkp-tests仓库的jobs目录 + +仓库地址为 https://gitee.com/wu_fengguang/lkp-tests + +这其中有一些已经适配好的测试case，如果你想要测试的case其中正好有，那就可以直接使用。使用方式以iperf为例。 + +iperf是已经适配的一个测试case，在lkp-tests仓库的jobs下有一个iperf.yaml文件，里面有一些测试的基本参数。 + +然后，你要在compass-ci仓库下面的 sbin/auto_submit.yaml 文件中添加 +``` +b/backlight/backlight: +- testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml +``` +并通过Pull Request的方式提交。 + +参考文档： +- [compass-ci测试平台使用教程--submit命令详解](https://gitee.com/wu_fengguang/compass-ci/t… +- [如何适配测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-test… + -- 2.23.0

1 0

[PATCH compass-ci] doc: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...273\273\345\212\241\347\273\223\346\236\234" | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..c4a2a55 --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,17 @@ +# 如何查看任务结果 + 完测试用例执行完成后，可以登录到https://compass-ci.openeuler.org查看用例的结果。 + 在网站上可以根据任务id等内容搜索到自己的任务，点击任务对应的job state列即可查看。 + +## job.yaml文件 + job.yaml文件中部分字段是用户提交上来的，还有一部分字段是根据提交的job自动添加进来。 + 此文件包含了测试任务需要的所有参数。 + +## output文件 + output文件记录了用例的执行过程，文件最后部分一般会有check_exit_code这个状态码，非0时代表测试用例错误。 + +## 测试项结果文件 + 测试过程中不仅会执行测试用例，还可能会监控系统信息，这些测试项会生成一个与测试项同名的文件，记录它们命令与输出。 + +## stats.json文件 + 对于测试项结果文件，会对每一个文件进行提取，生成与文件同名的xxx.json文件。 + stats.json是所有的json文件的汇总。 -- 2.23.0

2 1

[PATCH v2 compass-ci 4/4] assign-account: attach new direcroty for the container
by Luan Shengde 26 Nov '20

26 Nov '20

[why] enable execute command: usermod -s shell to enable/disable login permission Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/start | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/container/assign-account/start b/container/assign-account/start index 2290982..793191a 100755 --- a/container/assign-account/start +++ b/container/assign-account/start @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -. $CCI_SRC/container/defconfig.sh +. ../defconfig.sh docker_rm assign-account @@ -11,8 +11,7 @@ cmd=( --restart=always --name=assign-account -d - -v /etc/passwd:/etc/passwd:ro - -v /etc/group:/etc/group:ro + -v /etc:/etc:rw -v /srv/account-info:/opt/account_data/:rw -v /home:/home:rw -p 29999:29999 @@ -20,14 +19,3 @@ cmd=( ) "${cmd[@]}" - -cat << EOF -port: 29999 ---- -Client: -./assign-tool.rb email_file - -email subject: "apply ssh account" - -Refer to compass-ci/doc/apply-ssh-account.md for more information. -EOF -- 2.23.0

1 0

[PATCH v2 compass-ci 3/4] assign-account: build email message for updating configuration
by Luan Shengde 26 Nov '20

26 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../assign-account/build-update-email.rb | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100755 container/assign-account/build-update-email.rb diff --git a/container/assign-account/build-update-email.rb b/container/assign-account/build-update-email.rb new file mode 100755 index 0000000..40d6b3b --- /dev/null +++ b/container/assign-account/build-update-email.rb @@ -0,0 +1,43 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_update_message(email, account_info, conf_info) + message = if conf_info['enable_login'] + + <<~EMAIL_MESSAGE + To: #{email} + Subject: [compass-ci] account update notice + + Dear user: + + Your account has been update successfully. + You can use the following command to login the server. + + Login command: + ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} + + Account password: + #{account_info['my_password']} + + regards + compass-ci + EMAIL_MESSAGE + else + <<~EMAIL_MESSAGE + To: #{email} + Subject: [compass-ci] account expiration notice + + Dear user: + + your account has expired. + consult the manager for help if you want to continue using the account. + + regards + compass-ci + EMAIL_MESSAGE + end + + return message +end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/4] assign-account: add new function and optimize
by Luan Shengde 26 Nov '20

26 Nov '20

[why] meet new options for answerback-email meet update configuration function for answerback-email [how] add new function for the new options for answerback-email optimize current function to enable update Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 117 +++++++++++++++---- 1 file changed, 92 insertions(+), 25 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 0e91057..95bec14 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -29,11 +29,14 @@ API: call graph: setup_jumper_account_info - read_account_info - build_account_name + read account + read_account_info + build_account_info + reread_account_info read_jumper_info config_default_yaml config_authorized_key + permit_login_config generate_ssh_key the returned data for setup_jumper_account_info like: @@ -41,12 +44,14 @@ the returned data for setup_jumper_account_info like: "my_login_name" => "login_name", "password" => "password", "jumper_host" => "0.0.0.0", - "jumper_port" => "10000" + "jumper_port" => "10000", + "my_jumper_pubkey" => my_jumper_pubkey } =end require 'fileutils' +require 'yaml' # get jumper and account info class AccountStorage @@ -95,14 +100,21 @@ class AccountStorage end def setup_jumper_account_info - login_name, password = read_account_info + if @data.key?('is_update_account') && @data['is_update_account'] + login_name = @data['my_login_name'] + password = reread_account_info + else + login_name, password = read_account_info + end + jumper_host, jumper_port = read_jumper_info - pub_key = @data['my_ssh_pubkey'][0] - ssh_dir = File.join('/home/', login_name, '.ssh') - config_authorized_key(login_name, pub_key, ssh_dir) unless pub_key.nil? + config_authorized_key(login_name) config_default_yaml(login_name) - my_jumper_pubkey = generate_ssh_key(login_name, ssh_dir) if @data['gen_sshkey'].eql? true + permit_login_config(login_name) + if @data.key?('gen_sshkey') && @data['gen_sshkey'] + my_jumper_pubkey = generate_ssh_key(login_name) + end jumper_account_info = { 'my_login_name' => login_name, @@ -115,34 +127,89 @@ class AccountStorage return jumper_account_info end - def generate_ssh_key(login_name, ssh_dir) + def permit_login_config(login_name) + if @data.key?('enable_login') && @data['enable_login'] + %x(usermod -s /usr/bin/zsh #{login_name}) + else + %x(usermod -s /sbin/nologin #{login_name}) + end + end + + def generate_ssh_key(login_name) + ssh_dir = File.join('/home/', login_name, '.ssh') Dir.mkdir ssh_dir, 0o700 unless File.exist? ssh_dir + pub_key_file = File.join(ssh_dir, 'id_rsa.pub') + + return if File.exist?(pub_key_file) && \ + @data['my_ssh_pubkey'].include?(File.read(pub_key_file).strip) + %x(ssh-keygen -f "#{ssh_dir}/id_rsa" -N '' -C "#{login_name}@account-vm") - %x(chown -R #{login_name}:#{login_name} #{ssh_dir}) - File.read("/home/#{login_name}/.ssh/id_rsa.pub") + + FileUtils.chown_R(login_name, login_name, ssh_dir) + File.read("/home/#{login_name}/.ssh/id_rsa.pub").strip end - def config_default_yaml(login_name) + def touch_default_yaml(login_name) default_yaml_dir = File.join('/home', login_name, '.config/compass-ci/defaults') - FileUtils.mkdir_p default_yaml_dir + FileUtils.mkdir_p default_yaml_dir unless File.exist? default_yaml_dir + + default_yaml = File.join(default_yaml_dir, 'account.yaml') + FileUtils.touch default_yaml unless File.exist? default_yaml + default_yaml + end + def config_default_yaml(login_name) + default_yaml = touch_default_yaml(login_name) + + account_yaml = YAML.load_file(default_yaml) || {} # my_email, my_name, my_uuid is required to config default yaml file # they are added along with 'my_ssh_pubkey' when sending assign account request - File.open("#{default_yaml_dir}/account.yaml", 'a') do |file| - file.puts "my_email: #{@data['my_email']}" - file.puts "my_name: #{@data['my_name']}" - file.puts "my_uuid: #{@data['my_uuid']}" - end - %x(chown -R #{login_name}:#{login_name} "/home/#{login_name}/.config") + account_yaml['my_email'] = @data['my_email'] + account_yaml['my_name'] = @data['my_name'] + account_yaml['my_uuid'] = @data['my_uuid'] + + f = File.new(default_yaml, 'w') + f.puts account_yaml.to_yaml + f.close + FileUtils.chown_R(login_name, login_name, "/home/#{login_name}/.config") + end + + def config_authorized_key(login_name) + pub_key = @data['my_ssh_pubkey'][0] + + return if pub_key.nil? + return if pub_key.strip.end_with?('account-vm') + + ssh_dir = File.join('/home/', login_name, '.ssh') + Dir.mkdir ssh_dir, 0o700 unless File.exist? ssh_dir + + authorized_file = File.join(ssh_dir, 'authorized_keys') + FileUtils.touch authorized_file unless File.exist? authorized_file + + store_pubkey(ssh_dir, login_name, authorized_file, pub_key) end - def config_authorized_key(login_name, pub_key, ssh_dir) - Dir.mkdir ssh_dir, 0o700 - Dir.chdir ssh_dir - f = File.new('authorized_keys', 'w') + def store_pubkey(ssh_dir, login_name, authorized_file, pub_key) + authorized_keys = File.read(authorized_file).split("\n") + + return if authorized_keys.include?(pub_key.strip) + + f = File.new(authorized_file, 'a') f.puts pub_key f.close - File.chmod 0o600, 'authorized_keys' - %x(chown -R #{login_name}:#{login_name} #{ssh_dir}) + + FileUtils.chown_R(login_name, login_name, ssh_dir) + File.chmod 0o600, authorized_file + end + + def reread_account_info + my_login_name_file = File.join(@account_dir, 'assigned-users', @data['my_login_name']) + + message = "No such assigned account exists: #{my_login_name_file}." + raise message unless File.exist? my_login_name_file + + password = File.readlines(my_login_name_file)[0].chomp + + return password end end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/4] answerback-email: add new options
by Luan Shengde 26 Nov '20

26 Nov '20

add option for answerback-email tool -n|--name name specify my_name for user when assigning account -l|--login value enable/disable user login allowed values: y|Y|n|N -u|--update enable update configuration for assigned account -u option enable update for: my_name my_ssh_pubkey generate sshkey on jumper reset user's login permission Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 153 +++++++++++++++---- 1 file changed, 123 insertions(+), 30 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 5ddc451..40284e4 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -14,6 +14,7 @@ require 'optparse' require_relative '../defconfig' require_relative '../../lib/es_client' require_relative 'build-send-account-email' +require_relative 'build-update-email' names = Set.new %w[ JUMPER_HOST @@ -24,7 +25,7 @@ names = Set.new %w[ defaults = relevant_defaults(names) -JUMPER_HOST = defaults['JUMPER_HOST'] || 'api.compass-ci.openeuler.org' +JUMPER_HOST = defaults['JUMPER_HOST'] JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 @@ -33,25 +34,61 @@ my_info = { 'my_email' => nil, 'my_name' => nil, 'my_commit_url' => nil, - 'my_uuid' => %x(uuidgen).chomp, - 'my_ssh_pubkey' => [], - 'gen_sshkey' => false + 'my_uuid' => nil, + 'my_login_name' => nil, + 'my_ssh_pubkey' => [] } -def init_info(mail_content, my_info) +# stdin_info is used to store infos added with option: +# -e email_address +# -n name +# -s pubkey_file +# email_info is used to store infos parsed from email_file with option: +# -f email_file +# my_info_es is used to store info read from ES when update user +# when use --update option +# stdin_info, email_info, my_info_es have different priority +# stdin_info > email_info > my_info_es +# when assigning account or update conf for account +# if they have the same key, my_info will use the value with higher priority. +# conf_info is used to store keys used to config the account +stdin_info = {} +email_info = {} +my_info_es = {} +conf_info = { + 'gen_sshkey' => false, + 'enable_login' => true, + 'is_update_account' => false +} + +def init_info(mail_content, email_info, my_info) my_info['my_email'] = mail_content.from[0] - my_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') + email_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') return if mail_content.attachments.empty? - my_info['my_ssh_pubkey'] << mail_content.attachments[0].body.decoded + email_info['new_email_pubkey'] = mail_content.attachments[0].body.decoded.strip +end + +def read_my_login_name(my_email, my_info_es) + my_account_info_str = %x(curl -XGET localhost:9200/accounts/_doc/#{my_email}) + my_account_info = YAML.safe_load my_account_info_str + message = "No such email found from the ES: #{my_email}" + raise message unless my_account_info['found'] + + my_info_es.update my_account_info['_source'] end options = OptionParser.new do |opts| - opts.banner = 'Usage: answerback-mail.rb [-e|--email email] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey]\n" - opts.banner += " -e or -f is required\n" + opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|Y|n|N] [--update]\n" + opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' + opts.banner += "[-g|--gen-sshkey] [--login y|Y|n|N] [--update]\n" + opts.banner += " -e|-f is required when applying account or updating account\n" + opts.banner += " -n is required when assigning account with -e\n" opts.banner += " -s is optional when use -e\n" - opts.banner += ' -g is optional, used to generate sshkey for user' + opts.banner += " -g is optional, used to generate sshkey for user\n" + opts.banner += " -u is required when updating an account\n" + opts.banner += ' -l is optional, used to enable/disable login permission' opts.separator '' opts.separator 'options:' @@ -60,18 +97,37 @@ options = OptionParser.new do |opts| my_info['my_email'] = email_address end + opts.on('-n name', '--name name', 'appoint name') do |name| + stdin_info['my_name'] = name + end + opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| - my_info['my_ssh_pubkey'] << File.read(pub_key_file).chomp + stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| mail_content = Mail.read(email_file) - init_info(mail_content, my_info) + init_info(mail_content, email_info, my_info) end - opts.on('-g', '--gen-sshkey', 'generate jumper ras public/private key and return pubkey') do - my_info['gen_sshkey'] = true + opts.on('-g', '--gen-sshkey', 'generate jumper rsa public/private key and return pubkey') do + conf_info['gen_sshkey'] = true + end + + opts.on('-u', '--update', 'updata configurations') do + read_my_login_name(my_info['my_email'], my_info_es) + conf_info['is_update_account'] = true + end + + opts.on('-l value', '--login value', 'enable/disable login, value: y|Y|n|N') do |value| + if value.downcase == 'y' + conf_info['enable_login'] = true + elsif value.downcase == 'n' + conf_info['enable_login'] = false + else + raise 'invalid parameter, please use y|Y|n|N' + end end opts.on_tail('-h', '--help', 'show this message') do @@ -82,33 +138,70 @@ end options.parse!(ARGV) -def apply_account(my_info) - account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{my_info.to_json}') +def apply_account(my_info, conf_info) + apply_info = {} + apply_info.update my_info + apply_info.update conf_info + + account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') JSON.parse account_info_str end -def send_account(my_info) +def check_my_email(my_info) message = "No email address specified\n" - message += "use -e to add a email address\n" - message += 'or use -f to add a email file' - raise message if my_info['my_email'].nil? + message += "use -e to add an email address for applying account\n" + message += 'or use -f to add an email file' - account_info = apply_account(my_info) - my_info['my_login_name'] = account_info['my_login_name'] + raise message if my_info['my_email'].empty? +end +def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) + new_email_pubkey = email_info.delete 'new_email_pubkey' + new_stdin_pubkey = stdin_info.delete 'new_ssh_pubkey' + new_pubkey = new_stdin_pubkey || new_email_pubkey + + my_info.update my_info_es unless my_info_es.empty? + my_info.update email_info unless email_info.empty? + my_info.update stdin_info unless stdin_info.empty? + + return if new_pubkey.nil? + return if my_info['my_ssh_pubkey'].include? new_pubkey + + my_info['my_ssh_pubkey'].insert(0, new_pubkey) +end + +def build_my_info_from_account_info(my_info, account_info, conf_info) unless account_info['my_jumper_pubkey'].nil? - my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'].chomp + return if my_info['my_ssh_pubkey'][-1] == account_info['my_jumper_pubkey'] + + my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'] end - my_info.delete 'gen_sshkey' + my_info['my_login_name'] = account_info['my_login_name'] unless conf_info['is_update_account'] +end + +def send_account(my_info, conf_info, email_info, my_info_es, stdin_info) + check_my_email(my_info) + my_info['my_uuid'] = %x(uuidgen).chomp unless conf_info['is_update_account'] + build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) + + message = 'No my_name found, please use -n to add one' + raise message if my_info['my_name'].nil? + + account_info = apply_account(my_info, conf_info) + build_my_info_from_account_info(my_info, account_info, conf_info) + store_account_info(my_info) - send_mail(my_info, account_info) + send_mail(my_info, account_info, conf_info) end -def send_mail(my_info, account_info) - message = build_message(my_info['my_email'], account_info) - +def send_mail(my_info, account_info, conf_info) + message = if conf_info['is_update_account'] + build_update_message(my_info['my_email'], account_info, conf_info) + else + build_message(my_info['my_email'], account_info) + end %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end @@ -117,4 +210,4 @@ def store_account_info(my_info) es.put_source_by_id(my_info['my_email'], my_info) end -send_account(my_info) +send_account(my_info, conf_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH compass-ci] sched: fix cast from Nil to Hash(k, v) failed
by Cao Xueliang 26 Nov '20

26 Nov '20

The program {} => Nil in the commit 4301d1ece960dd0eff1f2e10c2d967e0e6150643 of lkp-tests, sched need to adapt to this case. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 378fd00..d88aa40 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -460,11 +460,11 @@ class Job private def get_program_params program_params = Hash(String, JSON::Any).new - if @hash["monitors"]? + if @hash["monitors"]? != nil program_params.merge!(@hash["monitors"].as_h) end - if @hash["pp"]? + if @hash["pp"]? != nil program_params.merge!(@hash["pp"].as_h) end -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: fix when to regard the job as a cluster job
by Ren Wen 26 Nov '20

26 Nov '20

If a job has a cluster field and its' value is not "cs-localhost" or empty, will regard it as a cluster job. When running a cluster job with cluster value "localhost", its' actions depend on field "node_roles", default is "server client". Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 8838e40..2836275 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -9,10 +9,8 @@ class Sched job = Job.new(job_content, job_content["id"]?) job["commit_date"] = get_commit_date(job) - # it is not a cluster job if cluster field is empty or - # field's prefix is 'cs-localhost' cluster_file = job["cluster"] - if cluster_file.empty? || cluster_file.starts_with?("cs-localhost") + if cluster_file.empty? || cluster_file == "cs-localhost" return submit_single_job(job) else cluster_config = get_cluster_config(cluster_file, -- 2.23.0

1 0

[PATCH v5 lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 3 +++ .../template/template-defaults.yaml | 19 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 5 files changed, 46 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/template/template-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/template/template-defaults.yaml b/spec/submit/merge_yaml/template/template-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/template/template-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH compass-ci] update document: apply-ssh-account.md
by Luan Shengde 26 Nov '20

26 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 44 ++++++++-------------------------------- 1 file changed, 8 insertions(+), 36 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index a4902f5..bb9ec76 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -1,35 +1,6 @@ +# apply account -# apply for jumper account - -## purpose - -The jumper account is used to login the jumper VM, where you can -- submit a job -- ssh into a testbox -- read output of the job - -## steps overview -### manually assign account -for internal user -1. run assign account tool - usage: - answerback-mail.rb [-e|--email email] [-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey] - - login command: - ssh -p port account@server_ip - -2. login authentication - - use pub key - - use password (if no pub key) - -3. submit jobs - - prepare job.yaml - - submit job - - command: - submit job.yaml - -### apply account with email -for external user +## steps 1. send apply account email - send apply account email to: compass-ci(a)qq.com attention: @@ -52,17 +23,18 @@ for external user my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… # attach your ssh pub key to the email as attachment -2. successfully applied account email - it will return an email with following information: +2. receive email from compass-ci(a)qq.com + the email contains following information: - my_email - my_name - my_uuid - SCHED_HOST - SCHED_PORT -3. environment configuration +3. local environment configuration follow steps in the email to finish the following configuration - setup default yaml ~/.config/compass-ci/default/account.yaml - - download and install lkp-tests - - prepare to submit jobs + +for more information: how to use submit + https://gitee.com/openeuler/compass-ci/blob/master/doc/compass-ci测试平台使用教程--… -- 2.23.0

1 0

[PATCH v2 lkp-tests] spec/submit_spec.rb: submit debug_yaml/job_fail.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to debug job.yaml, and after the task is failed, it is be waiting state. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/debug_yaml/job_fail.yaml | 5 +++++ .../debug_yaml/job_fail/job_fail-3000.yaml | 22 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 3 files changed, 31 insertions(+) create mode 100644 spec/submit/debug_yaml/job_fail.yaml create mode 100644 spec/submit/debug_yaml/job_fail/job_fail-3000.yaml diff --git a/spec/submit/debug_yaml/job_fail.yaml b/spec/submit/debug_yaml/job_fail.yaml new file mode 100644 index 00000000..16dcae3f --- /dev/null +++ b/spec/submit/debug_yaml/job_fail.yaml @@ -0,0 +1,5 @@ +suite: atomic + +on_fail: + sleep: 3000 +atomic: diff --git a/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml new file mode 100644 index 00000000..3d7ad9b5 --- /dev/null +++ b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml @@ -0,0 +1,22 @@ +--- +suite: atomic +on_fail: + sleep: 3000 +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + sleep: 3000 + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d9b931e2 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'debug yaml spec' do + submit_job('debug_yaml') + end end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/5] openeuler: configure files about build image with one command
by Wang Chenglong 26 Nov '20

26 Nov '20

[Contains] openeule-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

2 2

[PATCH v4 lkp-tests 2/2] stats/crystal: mysql-server to mysql-server.cr
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/mysql-server.cr | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 stats/crystal/mysql-server.cr diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr new file mode 100644 index 00000000..99b24b67 --- /dev/null +++ b/stats/crystal/mysql-server.cr @@ -0,0 +1,20 @@ +#!/usr/bin/env crystal + +results = [] of String + +while (line = STDIN.gets) + case line + # The output is as below: + # 200706 14:35:48 [ 99%] main.subquery_sj_innodb_all w8 [ pass ] 140 + # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 + # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 + when /\[\s*\d+%\]/ + results << line.split(']')[1].split[0] + ".pass: 1" if line.includes?("[ pass ]") + results << line.split(']')[1].split[0] + ".fail: 1" if line.includes?("[ fail ]") + results << line.split(']')[1].split[0] + ".skip: 1" if line.includes?("[ skipped ]") + end +end + +results.each do |item| + puts item +end -- 2.23.0

1 0

[PATCH v4 lkp-tests 1/2] stats/mysql-server: remove redundant whitespace
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/mysql-server | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/stats/mysql-server b/stats/mysql-server index 037d1f0f..ca31f9ea 100755 --- a/stats/mysql-server +++ b/stats/mysql-server @@ -1,5 +1,4 @@ #!/usr/bin/env ruby -# frozen_string_literal: true LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) @@ -16,9 +15,9 @@ while (line = STDIN.gets) # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 when /\[\s*\d+%\]/ - results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') - results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') - results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') + results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') + results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') + results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') end end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/5] openeuler: the link for one command to build openeuler initramfs image
by Wang Chenglong 26 Nov '20

26 Nov '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/openeuler/aarch64/20.03-LTS/build | 1 + rootfs/initramfs/openeuler/aarch64/20.09/build | 1 + 2 files changed, 2 insertions(+) create mode 120000 rootfs/initramfs/openeuler/aarch64/20.03-LTS/build create mode 120000 rootfs/initramfs/openeuler/aarch64/20.09/build diff --git a/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build b/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build new file mode 120000 index 0000000..f04acac --- /dev/null +++ b/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build @@ -0,0 +1 @@ +../../../../../container/osimage/openeuler/run \ No newline at end of file diff --git a/rootfs/initramfs/openeuler/aarch64/20.09/build b/rootfs/initramfs/openeuler/aarch64/20.09/build new file mode 120000 index 0000000..f04acac --- /dev/null +++ b/rootfs/initramfs/openeuler/aarch64/20.09/build @@ -0,0 +1 @@ +../../../../../container/osimage/openeuler/run \ No newline at end of file -- 2.23.0

2 2

[PATCH v2 compass-ci] web_backend.rb: fix : compare_groups_matrices needs arg suites_list
by Li Yuanchao 26 Nov '20

26 Nov '20

compare_groups_matrices needs three args, but now only get two, lost suites_list. divide method to pass rubocop limit Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 1f7224e..b5b06b4 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -147,6 +147,20 @@ def get_es_must(params) must end +def _get_group_matrices(query, dimension) + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + while matrices.empty? + from += size + break if from > total + + query[:from] = from + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + end + [matrices, suites_list] +end + def get_groups_matrices(conditions, dimension, must, size, from) must += build_mutli_field_subquery_body(conditions) count_query = { query: { bool: { must: must } } } @@ -166,27 +180,18 @@ def get_groups_matrices(conditions, dimension, must, size, from) }] } - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - while matrices.empty? - from += size - break if from > total - - query[:from] = from - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - end - matrices + matrices, suites_list = _get_groups_matrices(query, dimension) + [matrices, suites_list] end def get_compare_body(params) dimension, conditions = get_dimension_conditions(params) must = get_es_must(params) - groups_matrices = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) + groups_matrices, suites_list = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) if !groups_matrices || groups_matrices.empty? body = 'No Data.' else - body = compare_group_matrices(groups_matrices, { no_print: true }) + body = compare_group_matrices(groups_matrices, suites_list, { no_print: true }) body = 'No Difference.' if !body || body.empty? end return body -- 2.23.0

2 1

[PATCH v5 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 3 +++ .../template/template-defaults.yaml | 19 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 5 files changed, 46 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/template/template-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/template/template-defaults.yaml b/spec/submit/merge_yaml/template/template-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/template/template-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH compass-ci] job.cr: delete redundant "#{}" for string
by Xu Xijian 26 Nov '20

26 Nov '20

JobHelper.service_path() will return string value, so "#{}" is unnecessary for it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 378fd00..aa7ffc3 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -366,14 +366,14 @@ class Job private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{linux_vmlinuz_path}")}" + JobHelper.service_path("#{linux_vmlinuz_path}") end private def common_initrds temp_initrds = [] of String temp_initrds << "#{INITRD_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_INITRD}/lkp/#{lkp_initrd_user}/lkp-#{os_arch}.cgz")}" + JobHelper.service_path("#{SRV_INITRD}/lkp/#{lkp_initrd_user}/lkp-#{os_arch}.cgz") temp_initrds << "#{SCHED_HTTP_PREFIX}/job_initrd_tmpfs/#{id}/job.cgz" return temp_initrds @@ -384,13 +384,13 @@ class Job osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{osimage_dir}/current")}" + JobHelper.service_path("#{osimage_dir}/current") temp_initrds << "#{INITRD_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz") temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path(self["linux_modules_initrd"])}" + JobHelper.service_path(self["linux_modules_initrd"]) temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path(self["linux_headers_initrd"])}" + JobHelper.service_path(self["linux_headers_initrd"]) temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? @@ -402,7 +402,7 @@ class Job temp_initrds = [] of String temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/initrd.lkp")}" + JobHelper.service_path("#{SRV_OS}/#{os_dir}/initrd.lkp") return temp_initrds end -- 2.23.0

2 1

[PATCH v2 compass-ci 5/5] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 26 Nov '20

26 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command, and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build --- container/osimage/openeuler/run | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..dde9985 --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/openeuler/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) + +cmd=( + docker run + --name init_docker + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +load_docker_image +"${cmd[@]}" +cp_image_to_host +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

1 0

[PATCH v2 compass-ci 4/5] openeuler: generate the cgz file in container
by Wang Chenglong 26 Nov '20

26 Nov '20

Install packages, set rootfs, pack images package and other actions in docker. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/create-image | 37 ++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/osimage/openeuler/create-image diff --git a/container/osimage/openeuler/create-image b/container/osimage/openeuler/create-image new file mode 100755 index 0000000..7402b2b --- /dev/null +++ b/container/osimage/openeuler/create-image @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +install_packages() +{ + yum -y install --skip-broken $(</tmp/packages-to-install) + yum clean all +} + +setup_rootfs() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + rm -f /etc/machine-id /var/lib/dbus/machine-id + systemd-machine-id-setup +} + +setup_login() { + [ -n "$ROOT_PASSWD" ] || return + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" +} + +pack_cgz() +{ + echo "Packing package. Please wait." + find / ! -path "/tmp/${IMAGE_NAME}" | grep -vf /tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > /tmp/${IMAGE_NAME} + chmod 644 /tmp/${IMAGE_NAME} +} + +install_packages +setup_rootfs +setup_login +pack_cgz -- 2.23.0

1 0

[PATCH v2 compass-ci 3/5] openeuler: the function executed in host
by Wang Chenglong 26 Nov '20

26 Nov '20

Generate the root password, download and load the Docker image, and copy the image package to the host. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/lib | 45 +++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 container/osimage/openeuler/lib diff --git a/container/osimage/openeuler/lib b/container/osimage/openeuler/lib new file mode 100755 index 0000000..507b5b5 --- /dev/null +++ b/container/osimage/openeuler/lib @@ -0,0 +1,45 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(pwd)" +os_name="$(echo $DIR | awk -F "/" '{print $(NF - 2)}')" +os_version="$(echo $DIR | awk -F "/" '{print $NF}')" +docker_url="https://repo.openeuler.org/openEuler-${os_version}/docker_img/aarch64/openE…" +docker_tar_file="openEuler-${os_version}-docker.aarch64.tar.xz" +image=$(echo "${os_name}-${os_version,,}") +IMAGE_NAME="${os_name}-${os_version}-$(date +"%Y%m%d").cgz" + +export_root_passwd() +{ + local root_pwd_file="$HOME/.config/compass-ci/rootfs-passwd" + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs-passwd" + exit 1 + } + + export ROOT_PASSWD=$(cat "$root_pwd_file") +} + +load_docker_image() +{ + if [[ "$os_version" == '20.09' || "$os_version" == '20.03-LTS' ]]; then + [ -f "$HOME/$docker_tar_file" ] || wget -O "$HOME/$docker_tar_file" "$docker_url" + fi + + docker load -i "$HOME/$docker_tar_file" + + if [ $? = 0 ]; then + echo "finish downloading image" + else + echo "[ERROR] Pull docker image failed." + exit 1 + fi +} + +cp_image_to_host() +{ + docker cp -a init_docker:/tmp/${IMAGE_NAME} $HOME/ + echo "result: $(ls $HOME/${IMAGE_NAME})" +} -- 2.23.0

1 0

[PATCH lkp-tests] spec/submit_spec.rb: submit debug_yaml/job_fail.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

[why] We will use 'rspec spec/submit_spec.rb' to debug job.yaml, and after the task is failed, it is be waiting state. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/debug_yaml/job_fail.yaml | 6 +++++ .../debug_yaml/job_fail/job_fail-3000.yaml | 22 +++++++++++++++++++ 2 files changed, 28 insertions(+) create mode 100644 spec/submit/debug_yaml/job_fail.yaml create mode 100644 spec/submit/debug_yaml/job_fail/job_fail-3000.yaml diff --git a/spec/submit/debug_yaml/job_fail.yaml b/spec/submit/debug_yaml/job_fail.yaml new file mode 100644 index 00000000..86967162 --- /dev/null +++ b/spec/submit/debug_yaml/job_fail.yaml @@ -0,0 +1,6 @@ +suite: atomic + +on_fail: + sleep: 3000 + +atomic: diff --git a/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml new file mode 100644 index 00000000..3d7ad9b5 --- /dev/null +++ b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml @@ -0,0 +1,22 @@ +--- +suite: atomic +on_fail: + sleep: 3000 +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + sleep: 3000 + atomic: + cpufreq_governor: +monitors: -- 2.23.0

2 1

[PATCH compass-ci 2/5] openeuler: the function executed in host
by Wang Chenglong 26 Nov '20

26 Nov '20

Generate the root password, download and load the Docker image, and copy the image package to the host. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/lib | 45 +++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 container/osimage/openeuler/lib diff --git a/container/osimage/openeuler/lib b/container/osimage/openeuler/lib new file mode 100755 index 0000000..cf291ea --- /dev/null +++ b/container/osimage/openeuler/lib @@ -0,0 +1,45 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(pwd)" +os_name="$(echo $DIR |awk -F "/" '{print $(NF - 2)}')" +os_version="$(echo $DIR |awk -F "/" '{print $NF}')" +docker_url="https://repo.openeuler.org/openEuler-${os_version}/docker_img/aarch64/openE…" +docker_tar="openEuler-${os_version}-docker.aarch64.tar.xz" +image=$(echo "${os_name}-${os_version}" | tr '[A-Z]' '[a-z]') +IMAGE_NAME="${os_name}-${os_version}-$(date +"%Y%m%d").cgz" + +export_root_passwd() { + local root_pwd_file="$HOME/.config/compass-ci/rootfs-passwd" + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs-passwd" + exit 1 + } + + export ROOT_PASSWD=$(cat "$root_pwd_file") +} + +pull_docker_image() +{ + if [ "$os_version" == '20.09' ]; then + [ -f "$HOME/$docker_tar" ] || wget -O "$HOME/$docker_tar" "$docker_url" + elif [ "$os_version" == '20.03-LTS' ]; then + [ -f "$HOME/$docker_tar" ] || wget -O "$HOME/$docker_tar" "$docker_url" + fi + + docker load -i "$HOME/$docker_tar" + if [ $? = 0 ]; then + echo "finish downloading image" + else + echo "[ERROR] Pull docker image failed." + exit 1 + fi +} + +docker_cp_image() +{ + docker cp -a init_docker:/tmp/${IMAGE_NAME} $HOME/ + echo "result: $(ls $HOME/${IMAGE_NAME})" +} -- 2.23.0

4 6

[PATCH compass-ci 2/3] web_backend.rb: fix : compare_groups_matrices needs arg suites_list
by Li Yuanchao 26 Nov '20

26 Nov '20

compare_groups_matrices needs three args, but now only get two, lost suites_list. divide method to pass rubocop limit Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 1f7224e..90c8a58 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -147,6 +147,20 @@ def get_es_must(params) must end +def _get_group_matrices(query, dimension) + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + while matrices.empty? + from += size + break if from > total + + query[:from] = from + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + end + [matrices, suites_list] +end + def get_groups_matrices(conditions, dimension, must, size, from) must += build_mutli_field_subquery_body(conditions) count_query = { query: { bool: { must: must } } } @@ -166,27 +180,17 @@ def get_groups_matrices(conditions, dimension, must, size, from) }] } - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - while matrices.empty? - from += size - break if from > total - - query[:from] = from - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - end - matrices + matrices, suites_list = _get_groups_matrices(query, dimension) end def get_compare_body(params) dimension, conditions = get_dimension_conditions(params) must = get_es_must(params) - groups_matrices = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) + groups_matrices, suites_list = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) if !groups_matrices || groups_matrices.empty? body = 'No Data.' else - body = compare_group_matrices(groups_matrices, { no_print: true }) + body = compare_group_matrices(groups_matrices, suites_list, { no_print: true }) body = 'No Difference.' if !body || body.empty? end return body -- 2.23.0

1 1

[PATCH compass-ci 1/3] container/git-mirror: fix that can't get group id of committer in z9
by Li Yuanchao 26 Nov '20

26 Nov '20

In z9, 'committer' is only a group, not a user, so command 'id -g committer' can't get id of committer group Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/start | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/git-mirror/start b/container/git-mirror/start index e1d7d8f..a618ec3 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -10,13 +10,13 @@ DOCKER_REPO = '/c/upstream-repos' docker_rm "git-mirror" lkp = %x(id -u lkp) -committer = %x(id -g committer) +group_id = %x(cat /etc/group | grep '^committer' | awk -F ':' '{print $3}') cmd = %W[ docker run --restart=always --name git-mirror - -u #{lkp.to_i}:#{committer.to_i} + -u #{lkp.to_i}:#{group_id.to_i} -d -e CCI_SRC=#{DOCKER_CCI} -e REPO_SRC=#{DOCKER_REPO} -- 2.23.0

1 1

[PATCH compass-ci 3/3] web-backend: fix : wrong method name
by Li Yuanchao 26 Nov '20

26 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 90c8a58..88ca02a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -147,7 +147,7 @@ def get_es_must(params) must end -def _get_group_matrices(query, dimension) +def _get_groups_matrices(query, dimension) result = es_query(query) matrices, suites_list = combine_group_query_data(result, dimension) while matrices.empty? -- 2.23.0

1 0

[PATCH v2 lkp-tests 3/3] iso2rootfs: add comment to iso2rootfs.yaml
by Yu Chuan 26 Nov '20

26 Nov '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index a0ffd280676d..2c4d3a93ff62 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,9 +1,18 @@ suite: iso2rootfs category: benchmark iso2rootfs: + # iso related fields to be used to generate rootfs iso_os: iso_arch: iso_version: + + # place the result rootfs related fields + ## 1. result rootfs will be placed in the following localtion on the + ## remote file server: + ## - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ + ## 2. remote file server protocols current supported: + ## - nfs + ## - cifs rootfs_protocol: rootfs_server: rootfs_path: -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/3] iso2rootfs: delete the unnecessary fields
by Yu Chuan 26 Nov '20

26 Nov '20

[Why] 'INITRD_HTTP_HOST' and 'INITRD_HTTP_PORT' should be added by the user on demand at runtime, no need to merge into the git repository. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 2 -- tests/iso2rootfs | 4 +--- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 4f3e9c83e6cd..a0ffd280676d 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -7,8 +7,6 @@ iso2rootfs: rootfs_protocol: rootfs_server: rootfs_path: - initrd_http_host: - initrd_http_port: # config rootfs related fields ## supported fields: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 761fbb8b70cf..9adbc930b48f 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -49,8 +49,6 @@ check_yaml_vars() "rootfs_protocol" "rootfs_server" "rootfs_path" - "initrd_http_host" - "initrd_http_port" ) for yaml_t in "${yaml_vars[@]}" @@ -190,7 +188,7 @@ check_os() get_cgz() { - local cgz_url="http://${initrd_http_host}:${initrd_http_port}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/iso2rootfs.cgz" + local cgz_url="http://${INITRD_HTTP_HOST}:${INITRD_HTTP_PORT}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/iso2rootfs.cgz" local cgz_name cgz_name=$(basename "${cgz_url}") local cgz_path_t=${ENV_DIR}/${cgz_name} -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/3] iso2rootfs: delete the meaningless field
by Yu Chuan 26 Nov '20

26 Nov '20

[Why] The 'iso2rootfs_cgz' field will always be 'iso2rootfs.cgz', so delete this field, and use the 'iso2rootfs.cgz' directly. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 1 - tests/iso2rootfs | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 1a643b538ac1..4f3e9c83e6cd 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,7 +1,6 @@ suite: iso2rootfs category: benchmark iso2rootfs: - iso2rootfs_cgz: iso_os: iso_arch: iso_version: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index fcaadaa1ba46..761fbb8b70cf 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -43,7 +43,6 @@ check_yaml_vars() "os" "os_arch" "os_version" - "iso2rootfs_cgz" "iso_os" "iso_arch" "iso_version" @@ -191,7 +190,7 @@ check_os() get_cgz() { - local cgz_url="http://${initrd_http_host}:${initrd_http_port}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/${iso2rootfs_cgz}" + local cgz_url="http://${initrd_http_host}:${initrd_http_port}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/iso2rootfs.cgz" local cgz_name cgz_name=$(basename "${cgz_url}") local cgz_path_t=${ENV_DIR}/${cgz_name} -- 2.23.0

1 0

[PATCH v4 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

[why] We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/atomic.yaml | 3 +++ .../merge_yaml/atomic/atomic-defaults.yaml | 19 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 5 files changed, 46 insertions(+) create mode 100644 spec/submit/merge_yaml/atomic.yaml create mode 100644 spec/submit/merge_yaml/atomic/atomic-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/atomic.yaml b/spec/submit/merge_yaml/atomic.yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/atomic.yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/atomic/atomic-defaults.yaml b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..30019c6c --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: atomic.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index a83d2102..92d8fa3d 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -44,4 +44,8 @@ describe 'submit job spec' do it 'link matrix' do submit_job('matrix') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0