- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 2/4] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 27 Nov '20

27 Nov '20

1. monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2. query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index ba142d0..202908e 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,44 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + + %w[boot extract_finished].each do |state| + result_array = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_array.empty? + end + time_list +end + +def es_search(job_id) + es_list = [] + + es = ESQuery.new + result_dict = es.query_by_id(job_id) + %w[job_state result_root].each do |field| + es_list.push(result_dict[field]) unless result_dict.nil? + end + es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

1 0

[PATCH v2 compass-ci 1/4] refactor: user-client/maintain: walk-os-iperf-test
by Liu Yinsi 27 Nov '20

27 Nov '20

1. submit job; 2. get job result by monitor and es query; 3. write job result into report; 4. send report mail; 5. cron task, test once an hours, mail once a week. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 148 +++++++++++------- 1 file changed, 88 insertions(+), 60 deletions(-) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 21e1428..ba142d0 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -1,62 +1,90 @@ -#!/bin/bash -e +#!/usr/bin/env ruby # SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true -# os test rounds -test_times=1 - -test_logs=walk-test.report -test_yaml=iperf-walk-os.yaml -test_os=( - 'openeuler aarch64 20' - 'openeuler aarch64 20.03' - 'centos aarch64 7.6' - 'centos aarch64 7.8' - 'centos aarch64 8.1.1911' - 'debian aarch64 sid' -) - -trap 'echo User Cancelled! && exit' SIGINT - -sec2date() { - date -d@$1 +"%Y%m%d_%H:%M:%S" -} - -logging() { - if [ 'new' == "$1" ] ; then - printf "%-5s %-8s %-25s %-20s %-20s %-8s %-10s\n" ${@:2} > $test_logs - else - printf "%-5s %-8s %-25s %-20s %-20s %-8s %-10s\n" $@ >> $test_logs - fi -} - -create_test_yaml() { - local hwarch os os_arch os_ver cur_os - hwarch=$(arch) - cur_os="$@" - os=$(echo $cur_os |cut -d' ' -f1) - os_arch=$(echo $cur_os |cut -d' ' -f2) - os_ver=$(echo $cur_os |cut -d' ' -f3) - sed "s/USER/$USER/" $test_yaml > test.yaml - sed -i "s/OS_ARCH/$os_arch/" test.yaml - sed -i "s/OS_VER/$os_ver/" test.yaml - sed -i "s/OS/$os/" test.yaml - echo "$hwarch $os/$os_arch/$os_ver" -} - -job_test() { - local job_id bef aft - job_id=$(submit test.yaml |awk '{print $NF}') - bef=$(date +%s) - ./my-qemu.sh - aft=$(date +%s) - logging "$1" "$2" "$3" "$(sec2date $bef)" "$(sec2date $aft)" "$((aft - bef))" "$job_id" -} - -logging new "Index" "hwArch" "OS" "Begin" "End" "Cost/sec" "Job_ID" -for round in $(seq $test_times) -do - for idx in "${!test_os[@]}" - do - job_test $round $(create_test_yaml "${test_os[$idx]}") - done -done +# rootfs test via job, including cifs, initramfs files +# boot way kvm. +# input: job yaml, os, os_version, os_arch, os_mount, testbox. +# output: a test report, including log print time, job ID, os, runtime, job state and url to check logs, and send mail. +# Eg: test report format: +# [2020-11-26 19:22:22 +0800]: ["crystal.143338", "centos", "aarch64", "7.6.1810", "cifs", 2020-11-26 19:21:12 +0800, +# 2020-11-26 19:22:21 +0800, 69, "finished", "/result/iperf-walk-os-test/vm-2p8g/2020-11-26/crystal.143338"] + +require "#{ENV['CCI_SRC']}/lib/es_query" +require "#{ENV['LKP_SRC']}/lib/monitor" +require "#{ENV['CCI_SRC']}/lib/mail_client" + +require 'time' +require 'logger' +require 'rufus-scheduler' + +TEST_OS = ['openeuler aarch64 20.03', + 'centos aarch64 7.6.1810', + 'centos aarch64 7.8.2003', + 'centos aarch64 8.1.1911', + 'debian aarch64 sid', + 'archlinux aarch64 2020-11-12'].freeze + +TEST_YAML = 'iperf-walk-os.yaml' + +TESTBOX = 'vm-2p8g' + +LOG = 'walk-test.report' + +OS_MOUNT = %w[cifs initramfs].freeze + +EMAIL_ADDRESS = (ENV['USER']).to_s.freeze + +def run_qemu + Process.fork do + %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) + end +end + +def submit_job(os_str, os_mount) + `submit --my-queue 'os os_arch os_version=#{os_str}' 'testbox=#{TESTBOX}' 'os_mount=#{os_mount}' #{TEST_YAML}` +end + +def traversal_os_mount + OS_MOUNT.each do |os_mount| + traversal_test_os(os_mount) + end +end + +def traversal_test_os(os_mount) + TEST_OS.each do |os_str| + test_rootfs(os_str, os_mount) + end +end + +def test_rootfs(os_str, os_mount) + message = submit_job(os_str, os_mount).chomp + job_id = message.split('=')[1] + run_qemu + + job_result = if message.include? 'error' + [] + else + find_by_id(job_id) + end + + report_all = job_id.split(', ') + os_str.split + os_mount.split + job_result + write_report(report_all) +end + +def cron_job + test = Rufus::Scheduler.new + test.cron '0 8-22 * * *' do + traversal_os_mount + end + + mail = Rufus::Scheduler.new + mail.cron '0 8 * * 1' do + mail_report + end + + [test, mail].each(&:join) +end + +cron_job -- 2.23.0

1 0

[PATCH v5 compass-ci 3/3] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command, and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build --- container/osimage/openeuler/run | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..5d7da72 --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/openeuler/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) + +cmd=( + docker run + --name initramfs_${image} + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +load_docker_image +"${cmd[@]}" +cp_image_to_host +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] openeuler: generate the cgz file in container
by Wang Chenglong 27 Nov '20

27 Nov '20

Install packages, set rootfs, pack images package and other actions in docker. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/create-image | 37 ++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/osimage/openeuler/create-image diff --git a/container/osimage/openeuler/create-image b/container/osimage/openeuler/create-image new file mode 100755 index 0000000..631bb14 --- /dev/null +++ b/container/osimage/openeuler/create-image @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +install_packages() +{ + yum -y install --skip-broken $(</tmp/packages-to-install) + yum clean all +} + +setup_rootfs() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + rm -f /etc/machine-id /var/lib/dbus/machine-id + systemd-machine-id-setup +} + +setup_login() { + [ -n "$ROOT_PASSWD" ] || return + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" +} + +pack_cgz() +{ + echo "Packing package. Please wait." + find / ! -path "/tmp/${IMAGE_NAME}" | grep -vf /tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > /tmp/${IMAGE_NAME} + chmod 644 /tmp/${IMAGE_NAME} +} + +install_packages +setup_rootfs +setup_login +pack_cgz -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] openeuler: configure files about build image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Contains] openeuler-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

1 0

[PATCH v5 compass-ci] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command, and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build --- container/osimage/openeuler/run | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..5d7da72 --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/openeuler/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) + +cmd=( + docker run + --name initramfs_${image} + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +load_docker_image +"${cmd[@]}" +cp_image_to_host +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

1 1

[PATCH compass-ci 5/5] user-client/maintain: send report mail
by Liu Yinsi 27 Nov '20

27 Nov '20

use send-mail service to mail. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 7ed07c5..f65959d 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -34,6 +34,16 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def send_mail + message = File.read(LOG).chomp + data = " + subject: os rootfs test report + to: #{EMAIL_ADDRESS} + body: '#{message}'" + + %x(curl -X POST 'localhost:11311/send_mail_yaml' -d "#{data}") +end + def write_report(report_all) file = File.open(LOG, 'a') log = Logger.new(file, 'weekly') -- 2.23.0

4 6

[PATCH compass-ci 2/5] refactor: user-client/maintain: walk-os-iperf-test
by Liu Yinsi 27 Nov '20

27 Nov '20

1.submit job; 2.get job result by monitor and es query; 3.write job result into report; 4.send report mail; 5.cron task, test once an hours, mail once a week. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 151 +++++++++++------- 1 file changed, 91 insertions(+), 60 deletions(-) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 21e1428..536dc9f 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -1,62 +1,93 @@ -#!/bin/bash -e +#!/usr/bin/env ruby # SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true -# os test rounds -test_times=1 - -test_logs=walk-test.report -test_yaml=iperf-walk-os.yaml -test_os=( - 'openeuler aarch64 20' - 'openeuler aarch64 20.03' - 'centos aarch64 7.6' - 'centos aarch64 7.8' - 'centos aarch64 8.1.1911' - 'debian aarch64 sid' -) - -trap 'echo User Cancelled! && exit' SIGINT - -sec2date() { - date -d@$1 +"%Y%m%d_%H:%M:%S" -} - -logging() { - if [ 'new' == "$1" ] ; then - printf "%-5s %-8s %-25s %-20s %-20s %-8s %-10s\n" ${@:2} > $test_logs - else - printf "%-5s %-8s %-25s %-20s %-20s %-8s %-10s\n" $@ >> $test_logs - fi -} - -create_test_yaml() { - local hwarch os os_arch os_ver cur_os - hwarch=$(arch) - cur_os="$@" - os=$(echo $cur_os |cut -d' ' -f1) - os_arch=$(echo $cur_os |cut -d' ' -f2) - os_ver=$(echo $cur_os |cut -d' ' -f3) - sed "s/USER/$USER/" $test_yaml > test.yaml - sed -i "s/OS_ARCH/$os_arch/" test.yaml - sed -i "s/OS_VER/$os_ver/" test.yaml - sed -i "s/OS/$os/" test.yaml - echo "$hwarch $os/$os_arch/$os_ver" -} - -job_test() { - local job_id bef aft - job_id=$(submit test.yaml |awk '{print $NF}') - bef=$(date +%s) - ./my-qemu.sh - aft=$(date +%s) - logging "$1" "$2" "$3" "$(sec2date $bef)" "$(sec2date $aft)" "$((aft - bef))" "$job_id" -} - -logging new "Index" "hwArch" "OS" "Begin" "End" "Cost/sec" "Job_ID" -for round in $(seq $test_times) -do - for idx in "${!test_os[@]}" - do - job_test $round $(create_test_yaml "${test_os[$idx]}") - done -done +# rootfs test via job, including cifs, initramfs files +# boot way kvm. +# input: job yaml, os, os_version, os_arch, os_mount, testbox. +# output: a test report, including log print time, job ID, os, runtime, job state and url to check logs, and send mail. +# Eg: test report format: +# [2020-11-26 19:22:22 +0800]: ["crystal.143338", "centos", "aarch64", "7.6.1810", "cifs", 2020-11-26 19:21:12 +0800, +# 2020-11-26 19:22:21 +0800, 69, "finished", "/result/iperf-walk-os-test/vm-2p8g/2020-11-26/crystal.143338"] + +require "#{ENV['CCI_SRC']}/lib/es_query" +require "#{ENV['LKP_SRC']}/lib/monitor" +require 'time' +require 'logger' +require 'rufus-scheduler' + +TEST_OS = ['openeuler aarch64 20.03', + 'centos aarch64 7.6.1810', + 'centos aarch64 7.8.2003', + 'centos aarch64 8.1.1911', + 'debian aarch64 sid', + 'archlinux aarch64 2020-11-12'].freeze + +TEST_YAML = 'iperf-walk-os.yaml' + +TESTBOX = 'vm-2p8g' + +LOG = 'walk-test.report' + +OS_MOUNT = %w[cifs initramfs].freeze + +EMAIL_ADDRESS = (ENV['USER']).to_s.freeze + +def run_qemu + Process.fork do + %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) + end +end + +def submit_job(os_str, os_mount) + `submit 'os os_arch os_version=#{os_str}' testbox=#{TESTBOX}' 'queue=#{TESTBOX}~#{ENV['USER']}' \ + 'os_mount=#{os_mount}' #{TEST_YAML}` +end + +def traversal_os_mount + OS_MOUNT.each do |os_mount| + p os_mount + traversal_test_os(os_mount) + end +end + +def traversal_test_os(os_mount) + TEST_OS.each do |os_str| + test_rootfs(os_str, os_mount) + end +end + +def test_rootfs(os_str, os_mount) + message = submit_job(os_str, os_mount).chomp + puts message + job_id = message.split('=')[1] + run_qemu + + job_result = if message.include? 'error' + [] + else + find_by_id(job_id) + end + + report_all = job_id.split(', ') + os_args + job_result + write_report(report_all) +end + +def cron_job + test = Rufus::Scheduler.new + test.cron '0 8-22 * * *' do + traversal_os_mount + end + + mail = Rufus::Scheduler.new + mail.cron '0 8 * * 1' do + send_mail + end + + [test, mail].each do |job| + job.join + end +end + +cron_job -- 2.23.0

3 4

[PATCH v4 compass-ci 3/3] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command, and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build --- container/osimage/openeuler/run | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..dde9985 --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/openeuler/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) + +cmd=( + docker run + --name init_docker + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +load_docker_image +"${cmd[@]}" +cp_image_to_host +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

2 2

review star
by Wu Fengguang 27 Nov '20

27 Nov '20

Jihui is our review star, thanks Jihui! wfg@crystal /c/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 120 38 52 30 Wei Jihui 118 23 25 18 Cao Xueliang 36 15 27 15 Xiao Shenwei 25 5 11 13 Wang Yong 28 13 18 11 Liu Shaofei 24 9 13 10 Bai Jing 27 12 25 8 Ren Wen 53 13 17 8 Lu Kaiyi 11 5 10 7 Du Kaitian 43 13 33 7 Luan Shengde 24 4 11 6 Yu Chuan 21 6 6 6 Cui Yili 46 8 9 6 Zhang Yu 34 8 9 6 Sun Yukui 5 3 2 4 Zhang Yuhang 26 2 3 3 Lu Weitao 45 6 9 3 Wu Zhende 14 8 15 3 Lin Jiaxin 12 5 5 3 Li Ping 29 9 13 2 Xu Xijian 35 4 4 2 Hu Xuejiao 28 2 3 2 Li Yuanchao 10 3 10 1 Wu Fengguang 38 3 2 1 Zhang Yale 1 0 0 0 Zhang Dewan 13 0 0 0 Liu Yinsi 53 6 9 0 Wang Chenglong

1 0

help star
by Wu Fengguang 27 Nov '20

27 Nov '20

Xueliang is our help star, thanks Xueliang! wfg@crystal /c/todo% bin/help-star people/*/credit/2020-11-27.md 22.6 Cao Xueliang 21.0 Wei Jihui 17.3 Wu Fengguang 16.7 Yu Chuan 13.2 Lu Weitao 12.9 Wu Zhende 9.4 Cui Yili 9.2 Ren Wen 8.6 Du Kaitian 7.9 Xu Xijian 7.9 Wang Yong 7.1 Li Ping 6.7 Liu Yinsi 6.7 Zhang Dewan 6.6 Lu Kaiyi 6.0 Bai Jing 5.6 Luan Shengde 5.3 Lin Jiaxin 5.2 Liu Shaofei 5.0 Li Yuanchao 4.2 Zhang Yale 3.6 Wang Chenglong 2.9 Sun Yukui 2.9 Xiao Shenwei 2.3 Zhang Yuhang 2.0 Zhang Yu 1.1 Hu Xuejiao

1 0

git count
by Wu Fengguang 27 Nov '20

27 Nov '20

Shengde is our code star again, thanks Shengde! wfg@crystal /c/todo% bin/git-count 13 Luan Shengde 11 Li Yuanchao 9 Wu Zhende 9 Wang Yong 8 Xu Xijian 8 Lu Kaiyi 7 Sun Yukui 7 Wei Jihui 7 Zhang Yu 7 Yu Chuan 6 Liu Shaofei 6 Liu Yinsi 6 Zhang Yale 6 Ren Wen 5 Bai Jing 5 Cao Xueliang 5 Xiao Shenwei 5 Wang Chenglong 4 Hu Xuejiao 4 Lu Weitao 4 Cui Yili 3 Lin Jiaxin 2 Wu Fengguang 1 Zhang Yuhang 1 Li Ping

1 0

[PATCH v4 compass-ci 2/3] openeuler: generate the cgz file in container
by Wang Chenglong 27 Nov '20

27 Nov '20

Install packages, set rootfs, pack images package and other actions in docker. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/create-image | 37 ++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/osimage/openeuler/create-image diff --git a/container/osimage/openeuler/create-image b/container/osimage/openeuler/create-image new file mode 100755 index 0000000..631bb14 --- /dev/null +++ b/container/osimage/openeuler/create-image @@ -0,0 +1,37 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +install_packages() +{ + yum -y install --skip-broken $(</tmp/packages-to-install) + yum clean all +} + +setup_rootfs() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + rm -f /etc/machine-id /var/lib/dbus/machine-id + systemd-machine-id-setup +} + +setup_login() { + [ -n "$ROOT_PASSWD" ] || return + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" +} + +pack_cgz() +{ + echo "Packing package. Please wait." + find / ! -path "/tmp/${IMAGE_NAME}" | grep -vf /tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > /tmp/${IMAGE_NAME} + chmod 644 /tmp/${IMAGE_NAME} +} + +install_packages +setup_rootfs +setup_login +pack_cgz -- 2.23.0

1 0

[PATCH v4 compass-ci 1/3] openeuler: configure files about build image with one command
by Wang Chenglong 27 Nov '20

27 Nov '20

[Contains] openeuler-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

1 0

[PATCH lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 27 Nov '20

27 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- .../merge_yaml/merge_included_files.yaml | 1 + .../merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit/merge_yaml/template_yaml | 3 +++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml create mode 100644 spec/submit/merge_yaml/template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files.yaml b/spec/submit/merge_yaml/merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/merge_included_files/merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 52d9a158..4096cb73 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -52,4 +52,8 @@ describe 'submit job spec' do it 'job on fail' do submit_job('job_on_fail') end + + it 'merge yaml' do + submit_job('merge_yaml') + end end -- 2.23.0

2 1

[PATCH compass-ci 3/5] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 27 Nov '20

27 Nov '20

1.monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2.query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 536dc9f..d9d6bd6 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -34,6 +34,44 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + job_states = %w[boot extract_finished] + job_states.each do |state| + result_dict = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_dict.nil? + end + return time_list +end + +def es_search(job_id) + es_list = [] + query_field = %w[job_state result_root] + query_field.each do |field| + es = ESQuery.new + result_dict = es.query_by_id(job_id) + es_list.push(result_dict[field]) unless result_dict.nil? + end + return es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

2 2

[PATCH compass-ci] doc: remove extra how-to-create-a-PKGBUILD.md
by Liu Shaofei 27 Nov '20

27 Nov '20

Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- doc/how-to-create-a-PKGBUILD.md | 79 --------------------------------- 1 file changed, 79 deletions(-) delete mode 100644 doc/how-to-create-a-PKGBUILD.md diff --git a/doc/how-to-create-a-PKGBUILD.md b/doc/how-to-create-a-PKGBUILD.md deleted file mode 100644 index 01e1084..0000000 --- a/doc/how-to-create-a-PKGBUILD.md +++ /dev/null @@ -1,79 +0,0 @@ -# 如何编写PKGBUILD - -## PKGBUILD是什么？ - - PKGBUILD是一个shell脚本，makepkg通过PKGBUILD中包含的指令，生成包含二进制文件和安装指令的软件包。 - -## PKGBUILD包含什么？ - - PKGBUILD包含两部分内容：变量和函数 - - 1）、定义变量 - - - pkgname: 必须定义，表示软件包的名称; - - pkgver: 必须定义，表示软件包的版本号； - - pkgrel: 必须定义，表示软件包的发布号； - - arch: 必须定义，表示软件包使用的架构序列； - - depends: 可选字段，软件测试运行时需要的依赖包名称； - - makedepends: 可选字段，构建软件包时需要的文件列表； - - source: 可选字段，指定每个源文件的MD5哈希值，用于构建过程中验证源文件的完整性。 - - 2）、定义函数 - - - package函数 - package函数必须定义，此函数用于将文件安装到将成为构建包的根目录的目录中； - - prepare函数 - 定义可选的prepare函数，在其中执行用于准备构建源码的操作； - - build函数 - 定义可选的build函数，用于编译和/或构建源码； - - check函数 - 定义可选的check函数，用于运行程序包的测试套件。 - - 注意：srcdir 和 pkgdir - - - srcdir: 提取或复制源文件的目录，所有打包功能都在srcdir目录内部运行； - - pkgdir: 构建软件包的根目录，仅在package函数中使用。 - -## 如何编写PKGBUILD？ - - 1）、创建PKGBUILD，文件名必须以“PKGBUILD”命名。 - - `touch PKGBUILD` - - 2）、使用vim打开PKGBUILD，编写PKGBUILD文件内容。 - - 如下我们给出一个PKGBUILD例子： - - pkgname=zstd - pkgver=1.4.4 - pkgrel=2 - arch=('i686' 'x86_64' 'aarch64') - url='https://github.com/facebook/zstd' - license=('custom:BSD3' 'GPL2') - depends=('xz' 'zlib' 'lz4') - makedepends=('git') - source=('git://github.com/facebook/zstd.git#branch=dev') - md5sums=('SKIP') - - pkgver() { - cd "$srcdir/$pkgname" - git describe --long --tags | sed 's/$[^-]*-g$/r\1/;s/-/./g;s/^v//g' - } - - build() { - cd "$srcdir/$pkgname" - make - make -C contrib/pzstd - } - - package() { - cd "$srcdir/$pkgname" - make PREFIX="/usr" DESTDIR="$pkgdir/" install - install -D -m755 contrib/pzstd/pzstd "$pkgdir/usr/bin/pzstd" - install -D -m644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" - } - -## 参考 - - https://www.archlinux.org/pacman/PKGBUILD.5.html - https://git.archlinux.org/pacman.git/plain/proto/PKGBUILD.proto -- 2.23.0

1 0

[PATCH compass-ci 1/5] user-client/maintain: delete redundant fields
by Liu Yinsi 27 Nov '20

27 Nov '20

os, os_version, os_arch, os_mount, testbox will be used in 'submit 'key=value'...', no need write in job yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/iperf-walk-os.yaml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/user-client/maintain/walk-os-test/iperf-walk-os.yaml b/user-client/maintain/walk-os-test/iperf-walk-os.yaml index 647220c..237d006 100644 --- a/user-client/maintain/walk-os-test/iperf-walk-os.yaml +++ b/user-client/maintain/walk-os-test/iperf-walk-os.yaml @@ -7,9 +7,5 @@ if role server: if role client: iperf: protocol: tcp -testbox: vm-2p8g--USER -os: OS -os_arch: OS_ARCH -os_version: OS_VER node_roles: server client LKP_DEBUG_PREFIX: bash -x -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: reduce parameters
by Wu Fengguang 27 Nov '20

27 Nov '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- tests/build-pkg | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 29d35997..bdaef035 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -1,13 +1,16 @@ #!/bin/bash -# - os -# - os_arch -# - os_version -# - os_mount # - pkgbuild_repo # - upstream_repo -# - upstream_url # - upstream_commit # - PKGBUILD_TAG +# upstream_url + +# We'll use the below global env vars, but not treat them as parameter to avoid +# duplicates in result path like /openeuler-20.03-aarch64/openeuler-aarch64-cifs-2003/ +# - os +# - os_arch +# - os_version +# - os_mount ## makepkg is a script that automates the building of packages; it can download and validate source files, ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, -- 2.23.0

1 0

[PATCH v3 lkp-tests] lkp-tests/*: fix cci jobs upload in container
by Wang Yong 27 Nov '20

27 Nov '20

docker can not use mount operation, so now use curl to upload cci-depends/cci-makepkg cgz file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 10 ++++++++++ tests/cci-depends | 9 +++++++-- tests/cci-makepkg | 18 ++++++++++++++---- 3 files changed, 31 insertions(+), 6 deletions(-) diff --git a/distro/common b/distro/common index b48dfe5f1..cdb531403 100755 --- a/distro/common +++ b/distro/common @@ -81,3 +81,13 @@ share_debian_package() exit 0 } + +upload_to_target_dir() +{ + . $LKP_SRC/lib/upload.sh + + local dir="$1" + local file="$(ls -F ${dir} | grep -v "@")" + + upload_one_curl "${dir}/${file}" "${dir}" >/dev/null +} diff --git a/tests/cci-depends b/tests/cci-depends index d5c21541d..ffa585b8c 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -29,8 +29,10 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO -is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} -mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +is_docker || { + is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +} umask 002 @@ -59,3 +61,6 @@ fixup_preinstall fixup_arch_install $pack_arch pack_benchmark_deps + +is_docker || exit 0 +upload_to_target_dir ${pack_to} diff --git a/tests/cci-makepkg b/tests/cci-makepkg index d08f23718..a7a990a59 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -34,11 +34,17 @@ pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" -[ -n "$LKP_SERVER" ] && { - is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +mount_pkg_mnt() +{ + is_mount_point ${PKG_MNT} && umount ${PKG_MNT} + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" +} + +is_docker || { + [ -n "$LKP_SERVER" ] && mount_pkg_mnt } + curl_pkgfile() { curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ @@ -140,7 +146,11 @@ build_source_pkg() PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg - [ "$?" == 0 ] && update_softlink + [ "$?" == 0 ] || exit 1 + update_softlink + + is_docker || exit 0 + upload_to_target_dir ${sync_dest} } build_source_pkg -- 2.23.0

4 5

[PATCH v2 compass-ci 2/2] doc/job/: add os_dir.md
by Xu Xijian 27 Nov '20

27 Nov '20

Add a document for introducing the os_dir field. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/os_dir.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/job/os_dir.md diff --git a/doc/job/os_dir.md b/doc/job/os_dir.md new file mode 100644 index 0000000..cecac49 --- /dev/null +++ b/doc/job/os_dir.md @@ -0,0 +1,10 @@ +# os_dir + +Meaning: +- An os_dir is a field that consists of three fields: os, os_arch, os_version, an example can be openeuler/aarch64/20.03. +- An os_dir is used to consist some paths with concrete os information, here are some paths in scheduler: + $SRV_OS/$os_dir/boot + $SRV_INITRD/osimage/$os_dir + $SRV_INITRD/deps/$mount_type/$os_dir/$program/$program.cgz + $SRV_INITRD/pkg/$mount_type/$os_dir/$program/$program_version.cgz + -- 2.23.0

2 1

[PATCH v1 compass-ci] lib/mail_bisect_result.rb: no need to add from field when send a email
by Cao Xueliang 27 Nov '20

27 Nov '20

The email service will add the from field by itself. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 1e85824..2799ce2 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -37,7 +37,7 @@ class MailBisectResult https://gitee.com/openeuler/compass-ci BODY to = 'caoxl(a)crystal.ci' - @hash = { 'to' => to, 'body' => body, 'subject' => subject, 'from' => 'compass-ci(a)outlook.com' } + @hash = { 'to' => to, 'body' => body, 'subject' => subject } # @hash = { 'to' => @git_commit.author_email, 'body' => body, 'subject' => subject } end -- 2.23.0

1 0

[PATCH compass-ci] lib/mai_bisect_result.rb: no need to add from field when send a email
by Cao Xueliang 27 Nov '20

27 Nov '20

The email service will add the from field by itself. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 1e85824..2799ce2 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -37,7 +37,7 @@ class MailBisectResult https://gitee.com/openeuler/compass-ci BODY to = 'caoxl(a)crystal.ci' - @hash = { 'to' => to, 'body' => body, 'subject' => subject, 'from' => 'compass-ci(a)outlook.com' } + @hash = { 'to' => to, 'body' => body, 'subject' => subject } # @hash = { 'to' => @git_commit.author_email, 'body' => body, 'subject' => subject } end -- 2.23.0

1 1

[PATCH compass-ci] kernel_params.cr: refactor if block to one line
by Xu Xijian 27 Nov '20

27 Nov '20

In crystal language, if the last line of a function has no value, this function will return nil, so the function kernel_console can be refactored into one line. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/kernel_params.cr | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index a558735..2cc8172 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -26,11 +26,7 @@ class Job end private def kernel_console - if os_arch == "x86_64" - return "console=ttyS0,115200 console=tty0" - else - return "" - end + return "console=ttyS0,115200 console=tty0" if os_arch == "x86_64" end private def set_kernel_params -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code: fix start container error message
by Liu Yinsi 27 Nov '20

27 Nov '20

[why] when start containers send-internet-mail and send-mail, error message: docker: invalid publish opts format (should be name=value but got ':'). [how] add ENV vars defaults values, avoid confusion error message. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 218b6c1..444281e 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -15,6 +15,9 @@ server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${DOCKER_REGISTRY_PORT:=5001} : ${LOGGING_ES_HOST:=$server_ip} : ${LOGGING_ES_PORT:=9202} +: ${SEND_MAIL_PORT:=49000} +: ${LOCAL_SEND_MAIL_PORT:=11311} +: ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF @@ -32,6 +35,9 @@ DOCKER_REGISTRY_HOST: $DOCKER_REGISTRY_HOST DOCKER_REGISTRY_PORT: $DOCKER_REGISTRY_PORT LOGGING_ES_HOST: $LOGGING_ES_HOST LOGGING_ES_PORT: $LOGGING_ES_PORT +SEND_MAIL_PORT: $SEND_MAIL_PORT +LOCAL_SEND_MAIL_PORT: $LOCAL_SEND_MAIL_PORT +LOCAL_ROBOT_EMAIL_ADDRESS: $LOCAL_ROBOT_EMAIL_ADDRESS EOF cat > /etc/profile.d/compass.sh <<EOF -- 2.23.0

1 0

2025

2024

2023

2022

2021

2020

Compass-ci