November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 3/3] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command, and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build --- container/osimage/openeuler/run | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..dde9985 --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/openeuler/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) + +cmd=( + docker run + --name init_docker + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +load_docker_image +"${cmd[@]}" +cp_image_to_host +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

2 2

review star
by Wu Fengguang 27 Nov '20

27 Nov '20

Jihui is our review star, thanks Jihui! wfg@crystal /c/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 120 38 52 30 Wei Jihui 118 23 25 18 Cao Xueliang 36 15 27 15 Xiao Shenwei 25 5 11 13 Wang Yong 28 13 18 11 Liu Shaofei 24 9 13 10 Bai Jing 27 12 25 8 Ren Wen 53 13 17 8 Lu Kaiyi 11 5 10 7 Du Kaitian 43 13 33 7 Luan Shengde 24 4 11 6 Yu Chuan 21 6 6 6 Cui Yili 46 8 9 6 Zhang Yu 34 8 9 6 Sun Yukui 5 3 2 4 Zhang Yuhang 26 2 3 3 Lu Weitao 45 6 9 3 Wu Zhende 14 8 15 3 Lin Jiaxin 12 5 5 3 Li Ping 29 9 13 2 Xu Xijian 35 4 4 2 Hu Xuejiao 28 2 3 2 Li Yuanchao 10 3 10 1 Wu Fengguang 38 3 2 1 Zhang Yale 1 0 0 0 Zhang Dewan 13 0 0 0 Liu Yinsi 53 6 9 0 Wang Chenglong

1 0

help star
by Wu Fengguang 27 Nov '20

27 Nov '20

Xueliang is our help star, thanks Xueliang! wfg@crystal /c/todo% bin/help-star people/*/credit/2020-11-27.md 22.6 Cao Xueliang 21.0 Wei Jihui 17.3 Wu Fengguang 16.7 Yu Chuan 13.2 Lu Weitao 12.9 Wu Zhende 9.4 Cui Yili 9.2 Ren Wen 8.6 Du Kaitian 7.9 Xu Xijian 7.9 Wang Yong 7.1 Li Ping 6.7 Liu Yinsi 6.7 Zhang Dewan 6.6 Lu Kaiyi 6.0 Bai Jing 5.6 Luan Shengde 5.3 Lin Jiaxin 5.2 Liu Shaofei 5.0 Li Yuanchao 4.2 Zhang Yale 3.6 Wang Chenglong 2.9 Sun Yukui 2.9 Xiao Shenwei 2.3 Zhang Yuhang 2.0 Zhang Yu 1.1 Hu Xuejiao

1 0

git count
by Wu Fengguang 27 Nov '20

27 Nov '20

Shengde is our code star again, thanks Shengde! wfg@crystal /c/todo% bin/git-count 13 Luan Shengde 11 Li Yuanchao 9 Wu Zhende 9 Wang Yong 8 Xu Xijian 8 Lu Kaiyi 7 Sun Yukui 7 Wei Jihui 7 Zhang Yu 7 Yu Chuan 6 Liu Shaofei 6 Liu Yinsi 6 Zhang Yale 6 Ren Wen 5 Bai Jing 5 Cao Xueliang 5 Xiao Shenwei 5 Wang Chenglong 4 Hu Xuejiao 4 Lu Weitao 4 Cui Yili 3 Lin Jiaxin 2 Wu Fengguang 1 Zhang Yuhang 1 Li Ping

1 0

[PATCH v4 compass-ci 2/3] openeuler: generate the cgz file in container
by Wang Chenglong 27 Nov '20

27 Nov '20

Install packages, set rootfs, pack images package and other actions in docker. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/create-image | 37 ++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/osimage/openeuler/create-image diff --git a/container/osimage/openeuler/create-image b/container/osimage/openeuler/create-image new file mode 100755 index 0000000..631bb14 --- /dev/null +++ b/container/osimage/openeuler/create-image @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +install_packages() +{ + yum -y install --skip-broken $(</tmp/packages-to-install) + yum clean all +} + +setup_rootfs() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + rm -f /etc/machine-id /var/lib/dbus/machine-id + systemd-machine-id-setup +} + +setup_login() { + [ -n "$ROOT_PASSWD" ] || return + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" +} + +pack_cgz() +{ + echo "Packing package. Please wait." + find / ! -path "/tmp/${IMAGE_NAME}" | grep -vf /tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > /tmp/${IMAGE_NAME} + chmod 644 /tmp/${IMAGE_NAME} +} + +install_packages +setup_rootfs +setup_login +pack_cgz -- 2.23.0

1 0

[PATCH v4 compass-ci 1/3] openeuler: configure files about build image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Contains] openeuler-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

1 0

[PATCH lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 27 Nov '20

27 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- .../merge_yaml/merge_included_files.yaml | 1 + .../merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit/merge_yaml/template_yaml | 3 +++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml create mode 100644 spec/submit/merge_yaml/template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files.yaml b/spec/submit/merge_yaml/merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 52d9a158..4096cb73 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -52,4 +52,8 @@ describe 'submit job spec' do it 'job on fail' do submit_job('job_on_fail') end + + it 'merge yaml' do + submit_job('merge_yaml') + end end -- 2.23.0

2 1

[PATCH compass-ci 3/5] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 27 Nov '20

27 Nov '20

1.monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2.query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 536dc9f..d9d6bd6 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -34,6 +34,44 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + job_states = %w[boot extract_finished] + job_states.each do |state| + result_dict = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_dict.nil? + end + return time_list +end + +def es_search(job_id) + es_list = [] + query_field = %w[job_state result_root] + query_field.each do |field| + es = ESQuery.new + result_dict = es.query_by_id(job_id) + es_list.push(result_dict[field]) unless result_dict.nil? + end + return es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

2 2

[PATCH compass-ci] doc: remove extra how-to-create-a-PKGBUILD.md
by Liu Shaofei 27 Nov '20

27 Nov '20

Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- doc/how-to-create-a-PKGBUILD.md | 79 --------------------------------- 1 file changed, 79 deletions(-) delete mode 100644 doc/how-to-create-a-PKGBUILD.md diff --git a/doc/how-to-create-a-PKGBUILD.md b/doc/how-to-create-a-PKGBUILD.md deleted file mode 100644 index 01e1084..0000000 --- a/doc/how-to-create-a-PKGBUILD.md +++ /dev/null @@ -1,79 +0,0 @@ -# 如何编写PKGBUILD - -## PKGBUILD是什么？ - - PKGBUILD是一个shell脚本，makepkg通过PKGBUILD中包含的指令，生成包含二进制文件和安装指令的软件包。 - -## PKGBUILD包含什么？ - - PKGBUILD包含两部分内容：变量和函数 - - 1）、定义变量 - - - pkgname: 必须定义，表示软件包的名称; - - pkgver: 必须定义，表示软件包的版本号； - - pkgrel: 必须定义，表示软件包的发布号； - - arch: 必须定义，表示软件包使用的架构序列； - - depends: 可选字段，软件测试运行时需要的依赖包名称； - - makedepends: 可选字段，构建软件包时需要的文件列表； - - source: 可选字段，指定每个源文件的MD5哈希值，用于构建过程中验证源文件的完整性。 - - 2）、定义函数 - - - package函数 - package函数必须定义，此函数用于将文件安装到将成为构建包的根目录的目录中； - - prepare函数 - 定义可选的prepare函数，在其中执行用于准备构建源码的操作； - - build函数 - 定义可选的build函数，用于编译和/或构建源码； - - check函数 - 定义可选的check函数，用于运行程序包的测试套件。 - - 注意：srcdir 和 pkgdir - - - srcdir: 提取或复制源文件的目录，所有打包功能都在srcdir目录内部运行； - - pkgdir: 构建软件包的根目录，仅在package函数中使用。 - -## 如何编写PKGBUILD？ - - 1）、创建PKGBUILD，文件名必须以“PKGBUILD”命名。 - - `touch PKGBUILD` - - 2）、使用vim打开PKGBUILD，编写PKGBUILD文件内容。 - - 如下我们给出一个PKGBUILD例子： - - pkgname=zstd - pkgver=1.4.4 - pkgrel=2 - arch=('i686' 'x86_64' 'aarch64') - url='https://github.com/facebook/zstd' - license=('custom:BSD3' 'GPL2') - depends=('xz' 'zlib' 'lz4') - makedepends=('git') - source=('git://github.com/facebook/zstd.git#branch=dev') - md5sums=('SKIP') - - pkgver() { - cd "$srcdir/$pkgname" - git describe --long --tags | sed 's/$[^-]*-g$/r\1/;s/-/./g;s/^v//g' - } - - build() { - cd "$srcdir/$pkgname" - make - make -C contrib/pzstd - } - - package() { - cd "$srcdir/$pkgname" - make PREFIX="/usr" DESTDIR="$pkgdir/" install - install -D -m755 contrib/pzstd/pzstd "$pkgdir/usr/bin/pzstd" - install -D -m644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" - } - -## 参考 - - https://www.archlinux.org/pacman/PKGBUILD.5.html - https://git.archlinux.org/pacman.git/plain/proto/PKGBUILD.proto -- 2.23.0

1 0

[PATCH compass-ci 1/5] user-client/maintain: delete redundant fields
by Liu Yinsi 27 Nov '20

27 Nov '20

os, os_version, os_arch, os_mount, testbox will be used in 'submit 'key=value'...', no need write in job yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/iperf-walk-os.yaml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/user-client/maintain/walk-os-test/iperf-walk-os.yaml b/user-client/maintain/walk-os-test/iperf-walk-os.yaml index 647220c..237d006 100644 --- a/user-client/maintain/walk-os-test/iperf-walk-os.yaml +++ b/user-client/maintain/walk-os-test/iperf-walk-os.yaml @@ -7,9 +7,5 @@ if role server: if role client: iperf: protocol: tcp -testbox: vm-2p8g--USER -os: OS -os_arch: OS_ARCH -os_version: OS_VER node_roles: server client LKP_DEBUG_PREFIX: bash -x -- 2.23.0

1 0