- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 02/11] feat: step 2 of iso2rootfs: reboot directly in firstboot stage
by Yu Chuan 10 Mar '21

10 Mar '21

[Why] Firstboot stage: the first boot after iso installation is complete. At this stage, the system will run some initial configurations. So we need copy out the rootfs in second boot stage, and do nothing in firstboot stage. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/install-iso-firstboot.yaml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 jobs/install-iso-firstboot.yaml diff --git a/jobs/install-iso-firstboot.yaml b/jobs/install-iso-firstboot.yaml new file mode 100644 index 000000000000..ccec24c1c162 --- /dev/null +++ b/jobs/install-iso-firstboot.yaml @@ -0,0 +1,3 @@ +suite: install-iso-firstboot + +sleep: 1 -- 2.23.0

2 2

[PATCH v3 compass-ci 2/2] providers/multi-qemu: add the function of restarting VMs
by Wu Zhende 10 Mar '21

10 Mar '21

Obtain the VM to be restarted from the mq and restart it Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- providers/multi-qemu | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/providers/multi-qemu b/providers/multi-qemu index c729c8e..5979cc9 100755 --- a/providers/multi-qemu +++ b/providers/multi-qemu @@ -5,6 +5,9 @@ require 'fileutils' require 'optparse' +require 'json' + +require '../lib/mq_client' opt = {} options = OptionParser.new do |opts| @@ -42,6 +45,8 @@ HOSTNAME = opt['hostname_prefix'] || "vm-2p8g.#{ENV['HOSTNAME']}" NR_VM = opt['nr_vm'] || 1 QUEUES = opt['queues'] || "#{ENV['HOSTNAME']}.#{RUBY_PLATFORM.split('-')[0]}" LOG_DIR = '/srv/cci/serial/logs' +MQ_HOST = ENV['MQ_HOST'] || ENV['LKP_SERVER'] || 'localhost' +MQ_PORT = ENV['MQ_PORT'] || 5672 def main(hostname) start_time = record_runtime_log(hostname) @@ -99,7 +104,26 @@ def save_pid(arr) f.close end +def reboot_deadline_machine(hostname) + mq = MQClient.new + queue = mq.queue(hostname, {:durable => true}) + queue.subscribe({:block => true, :manual_ack => true}) do |info, _pro, msg| + puts msg + machine_info = JSON.parse(msg) + job_id = machine_info['job_id'] + system "pkill #{job_id};echo $?" + mq.ack(info) + end +end + def multiqemu + Process.fork do + reboot_deadline_machine(HOSTNAME) + rescue StandardError => e + puts e.backtrace + sleep 5 + end + pids = [] NR_VM.to_i.times do |i| pid = Process.fork do -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] lib/mq_client.rb: add new class for rabbitmq
by Wu Zhende 10 Mar '21

10 Mar '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/mq_client.rb | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100755 lib/mq_client.rb diff --git a/lib/mq_client.rb b/lib/mq_client.rb new file mode 100755 index 0000000..63618f9 --- /dev/null +++ b/lib/mq_client.rb @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'bunny' + +# client for rabbitmq +class MQClient + def initialize(hostname = 'localhost', port = '5672') + @conn = Bunny.new(hostname: hostname, port: port) + @conn.start + @channel = @conn.create_channel + end + + def queue(queue_name, opts = {}) + @channel.queue(queue_name, opts) + end + + def ack(delivery_info) + @channel.ack(delivery_info.delivery_tag) + end +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/bootstrap.sh: change $CCI_SRC to $lab
by Wang Yong 10 Mar '21

10 Mar '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- lib/bootstrap.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 52c9da32a..98c30a26b 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -274,7 +274,7 @@ setup_hosts() is_docker && return suffix="sh.intel.com" - [ -n "${CCI_SRC}" ] && suffix="compass-ci.net" + [ -n "${lab}" ] && suffix="compass-ci.net" # /etc/hosts may be shared when it's NFSROOT and there is no obvious # way to detect if rootfs is already RAM based. So unconditionally -- 2.23.0

1 0

[PATCH v2 lkp-tests] add new type error
by Bai Jing 10 Mar '21

10 Mar '21

[input] Error: Unknown repo: 'powertools' [output] unknown-repo.powertools: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/stats/openeuler_docker b/stats/openeuler_docker index f85c9ca63..b3890f568 100755 --- a/stats/openeuler_docker +++ b/stats/openeuler_docker @@ -9,9 +9,15 @@ while (line = STDIN.gets) # Error: Unable to find a match: docker-registry when /Error: Unable to find a match: (.+)/ $1.split.each do |repo| - repo_set << repo + repo_set << repo end + # Error: Unknown repo: 'powertools' + when /Error: Unknown repo: (.+)/ + repo = $1.delete!("'") + puts "unknown-repo.#{repo}: 1" + puts "unknown-repo.#{repo}.message: #{line}" + # /bin/sh: passwd: command not found when /\/bin\/sh: (.+): command not found/ puts "sh.command-not-found.#{$1}: 1" -- 2.23.0

2 2

[PATCH compass-ci 4/5] src/scheduler: add local mount of kernel_common_params
by Xu Xijian 10 Mar '21

10 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/kernel_params.cr | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index bb5983f..cbafb9f 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -4,8 +4,11 @@ class Job private def kernel_common_params common_params = "user=lkp job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job ip=dhcp" - return "#{common_params} rw" if "#{self.os_mount}" == "local" - return "#{common_params} rootovl ro" + return "#{common_params} rootovl ro" unless "#{self.os_mount}" == "local" + + use_root_partition = "/dev/mapper/os-#{os}_#{os_arch}_#{src_lv_suffix}" if @hash["src_lv_suffix"]? != nil + save_root_partition = "/dev/mapper/os-#{os}_#{os_arch}_#{boot_lv_suffix}" if @hash["boot_lv_suffix"]? != nil + return "#{common_params} local use_root_partition=#{use_root_partition} save_root_partition=#{save_root_partition} rw" end private def kernel_custom_params @@ -20,7 +23,7 @@ class Job "cifs" => "root=cifs://#{OS_HTTP_HOST}#{os_real_path}" + ",guest,ro,hard,vers=1.0,noacl,nouser_xattr,noserverino", "initramfs" => "rdinit=/sbin/init prompt_ramdisk=0", - "local" => "root=/dev/mapper/os-#{self.os}_#{self.os_arch}_#{self.os_version}", + "local" => "root=#{OS_HTTP_HOST}:#{os_real_path}", # root is just used to temporarily mount a root in initqueue stage when lvm is not ready "container" => "", } -- 2.23.0

2 2

[PATCH compass-ci 3/5] container/dracut-initrd: set sysroot in local mount situation
by Xu Xijian 10 Mar '21

10 Mar '21

When mount local filesystem in os starting, we will create or use a local existed logical volume to replace remote filesystem. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- .../dracut-initrd/bin/set-local-sysroot.sh | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 container/dracut-initrd/bin/set-local-sysroot.sh diff --git a/container/dracut-initrd/bin/set-local-sysroot.sh b/container/dracut-initrd/bin/set-local-sysroot.sh new file mode 100644 index 0000000..65e6d3e --- /dev/null +++ b/container/dracut-initrd/bin/set-local-sysroot.sh @@ -0,0 +1,78 @@ +#!/bin/sh + +analyse_kernel_cmdline_params() { + rootfs="$(getarg root=)" + rootfs_src=$(echo $"$rootfs" | sed 's/\///') + timestamp="$(basename $root)" + export rootfs_src timestamp +} + +rsync_src_lv() { + local src_lv="$1" + + [ -e "$src_lv" ] && return # /dev/mapper/os-openeuler-aarch64-20210226104900 + + # create logical volume + src_lv_devname="$(basename $src_lv)" # os-openeuler_aarch64_20210226104900 + lvm lvcreate -L 10G --name "${src_lv_devname#os-}" os # openeuler_aarch64_20210226104900 + mke2fs -t ext4 -F "$src_lv" + + # sync nfsroot to $src_lv + mkdir -p /mnt1 && mount -t nfs "$rootfs_src" /mnt1 + mkdir -p /mnt2 && mount "$src_lv" /mnt2 + cp -a /mnt1/. /mnt2/ + umount /mnt1 /mnt2 + + # change permission of "$src_lv" to readonly + lvm lvchange --permission r "$src_lv" +} + +snapshot_boot_lv() { + local src_lv="$1" + local boot_lv="$2" + + [ "$src_lv" == "$boot_lv" ] && return + + lvm lvremove --force "$boot_lv" + boot_lv_devname="$(basename $boot_lv)" + lvm lvcreate --size 10G --name ${boot_lv_devname#os-} --snapshot "$src_lv" +} + +set_sysroot() { + boot_lv="$1" + umount "$NEWROOT" + mount "$boot_lv" "$NEWROOT" +} + +if ! getargbool 0 local; then + exit 0 +fi + +analyse_kernel_cmdline_params + +sed -i "s/^locking_type = .*/locking_type = 1/" /etc/lvm/lvm.conf + +use_root_partition="$(getarg use_root_partition=)" +if [ -n "$use_root_partition" ]; then + src_lv="$use_root_partition" # "$use_root_partition" "$root_$timestamp", root from getarg root= + rsync_src_lv "$src_lv" +else + src_lv="/dev/mapper/os-openeuler_aarch64_$timestamp" # "os-openeuler_aarch64_$timestamp" + [ -e "src_lv" ] || { + echo "warn dracut: FATAL: no src_lv with local mount, reboot" + reboot + } +fi + +save_root_partition="$(getarg save_root_partition=)" +if [ -n "$save_root_partition" ]; then + boot_lv="$save_root_partition" # "os-openeuler_aarch64_$save_partition" +else + boot_lv="/dev/mapper/os-openeuler_aarch64_default" +fi + +snapshot_boot_lv "$src_lv" "$boot_lv" + +set_sysroot "$boot_lv" + +exit 0 -- 2.23.0

2 2

[PATCH compass-ci 2/5] container/dracut-initrd: add handling of pre-mount stage into setup-dracut.sh
by Xu Xijian 10 Mar '21

10 Mar '21

Add the content of set-local-sysroot.sh into the tail of dracut/modules.d/98dracut-systemd/dracut-pre-mount.sh. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 849480d..d447251 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -18,3 +18,7 @@ cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh sed -i "/install() {/a\ inst /usr/bin/awk" /usr/lib/dracut/modules.d/40network/module-setup.sh sed -i "/install() {/a\ inst /sbin/mke2fs" /usr/lib/dracut/modules.d/98dracut-systemd/module-setup.sh +pre_mount_file="/usr/lib/dracut/modules.d/98dracut-systemd/dracut-pre-mount.sh" +pre_mount_last_line="$(eval 'sed -n \$p $pre_mount_file')" +[ "$pre_mount_last_line" = "exit 0" ] && sed -i '$d' "$pre_mount_file" +cat set-local-sysroot.sh >> "$pre_mount_file" -- 2.23.0

2 2

[PATCH compass-ci 5/5] src/lib/job.cr: add keywords src_lv_suffix and boot_lv_suffix
by Xu Xijian 10 Mar '21

10 Mar '21

1. src_lv_suffix: to combine into use_root_partition, which specifies a logical volume to be a local source rootfs. 2. boot_lv_suffix: to combine into save_root_partition, whose rootfs is generated from use_root_partition. This rootfs will be mounted when start an os and saved locally after shutdown. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0b4bee4..424dd31 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -89,6 +89,8 @@ class Job docker_image kernel_version linux_vmlinuz_path + src_lv_suffix + boot_lv_suffix ) macro method_missing(call) -- 2.23.0

2 1

[PATCH compass-ci 2/2] container/srv-http: add /srv/initrd/rpmbuild-pkg
by Li Ping 10 Mar '21

10 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/srv-http/start-initrd | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start-initrd b/container/srv-http/start-initrd index 7c73ea1..495cc25 100755 --- a/container/srv-http/start-initrd +++ b/container/srv-http/start-initrd @@ -12,6 +12,7 @@ cmd=( -p 11303:11300 -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro -v /srv/initrd/dockerimage:/srv/initrd/dockerimage:ro + -v /srv/initrd/rpmbuild-pkg:/srv/initrd/rpmbuild-pkg:ro -v /etc/localtime:/etc/localtime:ro -d srv-http -- 2.23.0

1 0