- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] container/dracut-initrd: add --fix-missing when apt-get install
by Xu Xijian 04 Feb '21

04 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 7e2bf51..75148a1 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -5,9 +5,9 @@ export DEBIAN_FRONTEND=noninteractive apt-get update -apt-get install -y --no-install-recommends apt-utils >/dev/null 2>&1 -apt-get install -y nfs-common netbase cifs-utils kmod rsync -apt-get install -y dracut dracut-network dracut-config-generic +apt-get install -y --fix-missing --no-install-recommends apt-utils >/dev/null 2>&1 +apt-get install -y --fix-missing nfs-common netbase cifs-utils kmod rsync +apt-get install -y --fix-missing dracut dracut-network dracut-config-generic apt-get clean rm -rf /var/lib/apt/lists/* -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/lib: introduce domain.rb action.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

function: based on response of boot.libvirt generate a domain.xml refer-to: commit 457644c0486e0786ed8b563c6e4928445eea0207 if you want to test with your domain.xml, define vt field in job.yaml vt: domain: xxx/xxx.xml if not will use default options to generate a domain.xml TODO: case 1: Impact of single factors on tasks: vt: cpu: xxx/cpu.xml case 2: Impact of multiple factors on tasks: vt: cpu: xxx/cpu.xml disk: xxx/disk.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/action.rb | 75 +++++++++++++++++++++++++++++++++++++++++ providers/lib/domain.rb | 75 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) create mode 100644 providers/lib/action.rb create mode 100644 providers/lib/domain.rb diff --git a/providers/lib/action.rb b/providers/lib/action.rb new file mode 100644 index 0000000..526d84d --- /dev/null +++ b/providers/lib/action.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# expand each element of the domain.xml +module Action + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + def expand_erb(file) + @context.expand_erb(File.read(file)) + end + + def domain + @doc = Nokogiri::XML(expand_erb(@domain_option)) + end + + def name + @doc.xpath('//domain/name').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/name.xml") + end + + def os + @doc.xpath('//domain/os').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/os.xml") + end + + def emulator + @doc.xpath('//domain/devices/emulator').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/emulator.xml") + end + + def cpu + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/cpu.xml") + end + + def memory + @doc.xpath('//domain/memory').remove + @doc.xpath('//domain/currentMemory').remove + @doc.xpath('//domain/maxMemory').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/memory.xml") + end + + def devices + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/devices.xml") + end + + def disk + @context.info['disk'] ||= nil + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/disk.xml") + end + + def serial + @doc.xpath('//domain/devices/serial').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/serial.xml") + end + + def interface + @doc.xpath('//domain/devices/interface').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/interface.xml") + end + + def active + @doc.xpath('//domain/on_poweroff').remove + @doc.xpath('//domain/on_reboot').remove + @doc.xpath('//domain/on_crash').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/active.xml") + end + + def clock + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/clock.xml") + end + + def seclabel + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/seclabel.xml") + end +end diff --git a/providers/lib/domain.rb b/providers/lib/domain.rb new file mode 100644 index 0000000..8df9c06 --- /dev/null +++ b/providers/lib/domain.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'set' +require 'nokogiri' +require 'yaml' +require_relative 'action' +require_relative "#{ENV['LKP_SRC']}/lib/hashugar" + +# generate a domain.xml +class Domain + include Action + + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + OPTION_FILE = "#{TEMPLATE_DIR}/options.yaml" + def initialize(context, logger) + @doc = nil + @context = context + @logger = logger + @options = Hashugar.new(YAML.safe_load(File.read(OPTION_FILE))) + end + + def generate + domain_option + if user_domain? + default_option + return + end + replaceable_option + default_option + end + + def save_to(filename) + File.open(filename, 'w') do |f| + f.puts @doc.to_xml + end + File.realpath(filename) + end + + private + + def user_domain? + @context.info['vt'].key?('domain') + end + + def load_xml_file(filepath) + host = @context.info['SRV_HTTP_HOST'] + port = @context.info['SRV_HTTP_PORT'] + system "wget --timestamping --progress=bar:force http://#{host}:#{port}/cci/libvirt-xml/#{filepath}" + %x(basename #{filepath}).chomp + end + + def domain_option + @domain_option = "#{TEMPLATE_DIR}/#{(a)options.domain}.xml" + if user_domain? + @domain_option = load_xml_file((a)context.info['vt']['domain']) + end + @logger.info("Domain option: #{@domain_option}") + domain + end + + def default_option + @options.default.each do |one| + instance_eval one + end + end + + def replaceable_option + @options.replaceable.each do |one| + instance_eval one + end + end +end -- 2.23.0

1 0

[PATCH v5 compass-ci 1/4] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..5ea0fcc --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +RUN echo 'deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n \ +\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n' \ +> /etc/apt/sources.list + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /root + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

2 2

[PATCH compass-ci] doc/write-PKGBUILD: fix documentation errors
by Lin Jiaxin 03 Feb '21

03 Feb '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- doc/manual/write-PKGBUILD.en.md | 5 +++-- doc/manual/write-PKGBUILD.zh.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/manual/write-PKGBUILD.en.md b/doc/manual/write-PKGBUILD.en.md index c0c25e9..c259399 100644 --- a/doc/manual/write-PKGBUILD.en.md +++ b/doc/manual/write-PKGBUILD.en.md @@ -13,8 +13,9 @@ PKGBUILD includes variables and functions. - pkgrel: Mandatory. It indicates the release number of a software package. - arch: Mandatory. It indicates the architecture sequence of a software package. - depends: Optional. It indicates the name of the dependency package required for running a software test. -- makedepends: Optional. It indicates the list of files required for building a software package. -- source: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. +- makedepends: Optional. It indicates the name of the dependency package required for building a software package. +- source: Optional. It indicates the list of files required for building a software package. +- md5sums: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. ## Defining a Function diff --git a/doc/manual/write-PKGBUILD.zh.md b/doc/manual/write-PKGBUILD.zh.md index 414bfdc..ff53f0f 100644 --- a/doc/manual/write-PKGBUILD.zh.md +++ b/doc/manual/write-PKGBUILD.zh.md @@ -13,8 +13,9 @@ PKGBUILD�� - pkgrel: �� - arch: �� - depends: �� -- makedepends: �� -- source: ��MD5�� +- makedepends: �� +- source: �� +- md5sums: ��MD5�� ## �� -- 2.23.0

1 0

[PATCH lkp-tests] distro/archlinux: fix pacman search range
by Wang Yong 03 Feb '21

03 Feb '21

[before] $ pacman -Qqs gzip gzip zlib [after] $ pacman -Qq gzip gzip we need list local package accurately -s, --search <regexp> Search each locally-installed package for names or descriptions that match regexp. When including multiple search terms, only packages with descriptions matching ALL of those terms are returned. ref: https://archlinux.org/pacman/pacman.8.html Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/archlinux b/distro/archlinux index 3942080a6..d7b3d8d78 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -88,7 +88,7 @@ distro_install_depends() for pkg in $packages do - pacman -Qqs --noconfirm $pkg || pacman -Sy --noconfirm $pkg + pacman -Qq --noconfirm $pkg || pacman -Sy --noconfirm $pkg done done } -- 2.23.0

1 0

[PATCH v5 compass-ci 3/3] container/defconfig: load config yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

Load "/etc/compass-ci/register/*.yaml", get register account info from cci defaults config. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index fd13312..f9b0824 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -9,6 +9,7 @@ def cci_defaults hash = {} Dir.glob(['/etc/compass-ci/defaults/*.yaml', '/etc/compass-ci/accounts/*.yaml', + '/etc/compass-ci/register/*.yaml', "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| hash.update YAML.load_file(file) || {} end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] container/auto-submit: bind mount volume
by Liu Yinsi 03 Feb '21

03 Feb '21

mount /etc/compass-ci/register, to get auto-submit account info(my_name, my_email) from config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/auto-submit/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/auto-submit/start b/container/auto-submit/start index 4632ab0..79463f6 100755 --- a/container/auto-submit/start +++ b/container/auto-submit/start @@ -21,6 +21,7 @@ cmd = %W[ -v #{ENV['LKP_SRC']}:#{DOCKER_LKP} -v /etc/localtime:/etc/localtime:ro -v /etc/compass-ci/defaults:/etc/compass-ci/defaults:ro + -v /etc/compass-ci/register:/etc/compass-ci/register:ro -v /srv/git:/srv/git -w /c/compass-ci/sbin alpine:auto-submit -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] sbin/auto-submit: config account yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

[how] before: define my_name, my_email, lab, then store into ES and config yaml after: find my_name, my_email from /etc/compass-ci/register/*.yaml, then config yaml [why] store account info into ES, this part already done by container service register-accounts, so when start auto-submit service, no need to store data, only config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/auto_submit | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 5648d59..c59deb7 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -6,7 +6,9 @@ require 'bunny' require 'json' require 'yaml' +require 'set' require_relative '../lib/build_my_info_client' +require "#{ENV['CCI_SRC']}/container/defconfig" # receive message and auto submit job class AutoSubmit @@ -86,15 +88,23 @@ class AutoSubmit end end -option = { - my_name: 'Auto Submit Robot', - my_email: 'autosubmit@localhost', - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp -} +def my_email + names = Set.new %w[auto-submit] + + defaults = relevant_defaults(names) + defaults['auto-submit']['my_email'] +end + +def config_yaml + build_my_info = BuildMyInfo.new(my_email) + + my_info = build_my_info.search_my_info + build_my_info.config_default_yaml(my_info) + build_my_info.config_lab_yaml(my_info) +end # fix verification problems when submit jobs by auto-submit service -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +config_yaml auto_submitter = AutoSubmit.new auto_submitter.listen -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] sparrow/1-storage: optimise format
by Liu Yinsi 03 Feb '21

03 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 47 ++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 0e8d722..0713d7e 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -1,26 +1,37 @@ #! /bin/bash -chown lkp:lkp /srv/result -chown lkp:lkp /srv/initrd -chown -R lkp:lkp /srv/es -chown -R mailer:team /srv/cci/Maildir -chown lkp:lkp /srv/cache/netdata_cache -chown lkp:lkp /srv/cache/netdata_lib -chown lkp:lkp /srv/cci/serial/fluentd-pos -chown lkp:lkp /srv/cci/libvirt-xml +chmod_dir=( + /srv/result + /srv/es/logging-es + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/logs + /srv/cci/libvirt-xml + /srv/cci/serial/fluentd-pos + /srv/cci/Maildir +) + +chmod 775 "${chmod_dir[@]}" + +chown_dir=( + /srv/result + /srv/initrd + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml +) + +chown lkp:lkp "${chown_dir[@]}" + chown lkp:committer /srv/git -chgrp team /srv/cci/serial/logs -chmod 775 /srv/result -chmod 775 /srv/es/logging-es -chmod 775 /srv/es -chmod 775 /srv/cache/netdata_cache -chmod 775 /srv/cache/netdata_lib -chmod 775 /srv/cci/serial/logs -chmod 775 /srv/cci/libvirt-xml -chmod 775 /srv/cci/serial/fluentd-pos -chmod -R 775 /srv/cci/Maildir +chown mailer:team /srv/cci/Maildir +chgrp team /srv/cci/serial/logs chgrp team /srv/dc + chmod g+ws /srv/dc find /tftpboot -type d | xargs chmod 775 -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

1 0