- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 27 Nov '20

27 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- .../merge_yaml/merge_included_files.yaml | 1 + .../merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit/merge_yaml/template_yaml | 3 +++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml create mode 100644 spec/submit/merge_yaml/template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files.yaml b/spec/submit/merge_yaml/merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 52d9a158..4096cb73 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -52,4 +52,8 @@ describe 'submit job spec' do it 'job on fail' do submit_job('job_on_fail') end + + it 'merge yaml' do + submit_job('merge_yaml') + end end -- 2.23.0

2 1

[PATCH compass-ci 3/5] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 27 Nov '20

27 Nov '20

1.monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2.query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 536dc9f..d9d6bd6 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -34,6 +34,44 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + job_states = %w[boot extract_finished] + job_states.each do |state| + result_dict = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_dict.nil? + end + return time_list +end + +def es_search(job_id) + es_list = [] + query_field = %w[job_state result_root] + query_field.each do |field| + es = ESQuery.new + result_dict = es.query_by_id(job_id) + es_list.push(result_dict[field]) unless result_dict.nil? + end + return es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

2 2

[PATCH compass-ci] doc: remove extra how-to-create-a-PKGBUILD.md
by Liu Shaofei 27 Nov '20

27 Nov '20

Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- doc/how-to-create-a-PKGBUILD.md | 79 --------------------------------- 1 file changed, 79 deletions(-) delete mode 100644 doc/how-to-create-a-PKGBUILD.md diff --git a/doc/how-to-create-a-PKGBUILD.md b/doc/how-to-create-a-PKGBUILD.md deleted file mode 100644 index 01e1084..0000000 --- a/doc/how-to-create-a-PKGBUILD.md +++ /dev/null @@ -1,79 +0,0 @@ -# 如何编写PKGBUILD - -## PKGBUILD是什么？ - - PKGBUILD是一个shell脚本，makepkg通过PKGBUILD中包含的指令，生成包含二进制文件和安装指令的软件包。 - -## PKGBUILD包含什么？ - - PKGBUILD包含两部分内容：变量和函数 - - 1）、定义变量 - - - pkgname: 必须定义，表示软件包的名称; - - pkgver: 必须定义，表示软件包的版本号； - - pkgrel: 必须定义，表示软件包的发布号； - - arch: 必须定义，表示软件包使用的架构序列； - - depends: 可选字段，软件测试运行时需要的依赖包名称； - - makedepends: 可选字段，构建软件包时需要的文件列表； - - source: 可选字段，指定每个源文件的MD5哈希值，用于构建过程中验证源文件的完整性。 - - 2）、定义函数 - - - package函数 - package函数必须定义，此函数用于将文件安装到将成为构建包的根目录的目录中； - - prepare函数 - 定义可选的prepare函数，在其中执行用于准备构建源码的操作； - - build函数 - 定义可选的build函数，用于编译和/或构建源码； - - check函数 - 定义可选的check函数，用于运行程序包的测试套件。 - - 注意：srcdir 和 pkgdir - - - srcdir: 提取或复制源文件的目录，所有打包功能都在srcdir目录内部运行； - - pkgdir: 构建软件包的根目录，仅在package函数中使用。 - -## 如何编写PKGBUILD？ - - 1）、创建PKGBUILD，文件名必须以“PKGBUILD”命名。 - - `touch PKGBUILD` - - 2）、使用vim打开PKGBUILD，编写PKGBUILD文件内容。 - - 如下我们给出一个PKGBUILD例子： - - pkgname=zstd - pkgver=1.4.4 - pkgrel=2 - arch=('i686' 'x86_64' 'aarch64') - url='https://github.com/facebook/zstd' - license=('custom:BSD3' 'GPL2') - depends=('xz' 'zlib' 'lz4') - makedepends=('git') - source=('git://github.com/facebook/zstd.git#branch=dev') - md5sums=('SKIP') - - pkgver() { - cd "$srcdir/$pkgname" - git describe --long --tags | sed 's/$[^-]*-g$/r\1/;s/-/./g;s/^v//g' - } - - build() { - cd "$srcdir/$pkgname" - make - make -C contrib/pzstd - } - - package() { - cd "$srcdir/$pkgname" - make PREFIX="/usr" DESTDIR="$pkgdir/" install - install -D -m755 contrib/pzstd/pzstd "$pkgdir/usr/bin/pzstd" - install -D -m644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" - } - -## 参考 - - https://www.archlinux.org/pacman/PKGBUILD.5.html - https://git.archlinux.org/pacman.git/plain/proto/PKGBUILD.proto -- 2.23.0

1 0

[PATCH compass-ci 1/5] user-client/maintain: delete redundant fields
by Liu Yinsi 27 Nov '20

27 Nov '20

os, os_version, os_arch, os_mount, testbox will be used in 'submit 'key=value'...', no need write in job yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/iperf-walk-os.yaml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/user-client/maintain/walk-os-test/iperf-walk-os.yaml b/user-client/maintain/walk-os-test/iperf-walk-os.yaml index 647220c..237d006 100644 --- a/user-client/maintain/walk-os-test/iperf-walk-os.yaml +++ b/user-client/maintain/walk-os-test/iperf-walk-os.yaml @@ -7,9 +7,5 @@ if role server: if role client: iperf: protocol: tcp -testbox: vm-2p8g--USER -os: OS -os_arch: OS_ARCH -os_version: OS_VER node_roles: server client LKP_DEBUG_PREFIX: bash -x -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: reduce parameters
by Wu Fengguang 27 Nov '20

27 Nov '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- tests/build-pkg | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 29d35997..bdaef035 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -1,13 +1,16 @@ #!/bin/bash -# - os -# - os_arch -# - os_version -# - os_mount # - pkgbuild_repo # - upstream_repo -# - upstream_url # - upstream_commit # - PKGBUILD_TAG +# upstream_url + +# We'll use the below global env vars, but not treat them as parameter to avoid +# duplicates in result path like /openeuler-20.03-aarch64/openeuler-aarch64-cifs-2003/ +# - os +# - os_arch +# - os_version +# - os_mount ## makepkg is a script that automates the building of packages; it can download and validate source files, ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, -- 2.23.0

1 0

[PATCH v3 lkp-tests] lkp-tests/*: fix cci jobs upload in container
by Wang Yong 27 Nov '20

27 Nov '20

docker can not use mount operation, so now use curl to upload cci-depends/cci-makepkg cgz file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 10 ++++++++++ tests/cci-depends | 9 +++++++-- tests/cci-makepkg | 18 ++++++++++++++---- 3 files changed, 31 insertions(+), 6 deletions(-) diff --git a/distro/common b/distro/common index b48dfe5f1..cdb531403 100755 --- a/distro/common +++ b/distro/common @@ -81,3 +81,13 @@ share_debian_package() exit 0 } + +upload_to_target_dir() +{ + . $LKP_SRC/lib/upload.sh + + local dir="$1" + local file="$(ls -F ${dir} | grep -v "@")" + + upload_one_curl "${dir}/${file}" "${dir}" >/dev/null +} diff --git a/tests/cci-depends b/tests/cci-depends index d5c21541d..ffa585b8c 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -29,8 +29,10 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO -is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} -mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +is_docker || { + is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +} umask 002 @@ -59,3 +61,6 @@ fixup_preinstall fixup_arch_install $pack_arch pack_benchmark_deps + +is_docker || exit 0 +upload_to_target_dir ${pack_to} diff --git a/tests/cci-makepkg b/tests/cci-makepkg index d08f23718..a7a990a59 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -34,11 +34,17 @@ pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" -[ -n "$LKP_SERVER" ] && { - is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +mount_pkg_mnt() +{ + is_mount_point ${PKG_MNT} && umount ${PKG_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +} + +is_docker || { + [ -n "$LKP_SERVER" ] && mount_pkg_mnt } + curl_pkgfile() { curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ @@ -140,7 +146,11 @@ build_source_pkg() PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg - [ "$?" == 0 ] && update_softlink + [ "$?" == 0 ] || exit 1 + update_softlink + + is_docker || exit 0 + upload_to_target_dir ${sync_dest} } build_source_pkg -- 2.23.0

4 5

[PATCH v2 compass-ci 2/2] doc/job/: add os_dir.md
by Xu Xijian 27 Nov '20

27 Nov '20

Add a document for introducing the os_dir field. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/os_dir.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/job/os_dir.md diff --git a/doc/job/os_dir.md b/doc/job/os_dir.md new file mode 100644 index 0000000..cecac49 --- /dev/null +++ b/doc/job/os_dir.md @@ -0,0 +1,10 @@ +# os_dir + +Meaning: +- An os_dir is a field that consists of three fields: os, os_arch, os_version, an example can be openeuler/aarch64/20.03. +- An os_dir is used to consist some paths with concrete os information, here are some paths in scheduler: + $SRV_OS/$os_dir/boot + $SRV_INITRD/osimage/$os_dir + $SRV_INITRD/deps/$mount_type/$os_dir/$program/$program.cgz + $SRV_INITRD/pkg/$mount_type/$os_dir/$program/$program_version.cgz + -- 2.23.0

2 1

[PATCH v1 compass-ci] lib/mail_bisect_result.rb: no need to add from field when send a email
by Cao Xueliang 27 Nov '20

27 Nov '20

The email service will add the from field by itself. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 1e85824..2799ce2 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -37,7 +37,7 @@ class MailBisectResult https://gitee.com/openeuler/compass-ci BODY to = 'caoxl(a)crystal.ci' - @hash = { 'to' => to, 'body' => body, 'subject' => subject, 'from' => 'compass-ci(a)outlook.com' } + @hash = { 'to' => to, 'body' => body, 'subject' => subject } # @hash = { 'to' => @git_commit.author_email, 'body' => body, 'subject' => subject } end -- 2.23.0

1 0

[PATCH compass-ci] lib/mai_bisect_result.rb: no need to add from field when send a email
by Cao Xueliang 27 Nov '20

27 Nov '20

The email service will add the from field by itself. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 1e85824..2799ce2 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -37,7 +37,7 @@ class MailBisectResult https://gitee.com/openeuler/compass-ci BODY to = 'caoxl(a)crystal.ci' - @hash = { 'to' => to, 'body' => body, 'subject' => subject, 'from' => 'compass-ci(a)outlook.com' } + @hash = { 'to' => to, 'body' => body, 'subject' => subject } # @hash = { 'to' => @git_commit.author_email, 'body' => body, 'subject' => subject } end -- 2.23.0

1 1

[PATCH compass-ci] kernel_params.cr: refactor if block to one line
by Xu Xijian 27 Nov '20

27 Nov '20

In crystal language, if the last line of a function has no value, this function will return nil, so the function kernel_console can be refactored into one line. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/kernel_params.cr | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index a558735..2cc8172 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -26,11 +26,7 @@ class Job end private def kernel_console - if os_arch == "x86_64" - return "console=ttyS0,115200 console=tty0" - else - return "" - end + return "console=ttyS0,115200 console=tty0" if os_arch == "x86_64" end private def set_kernel_params -- 2.23.0

1 0