February 2021 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] distro/archlinux: fix pacman search range
by Wang Yong 03 Feb '21

03 Feb '21

[before] $ pacman -Qqs gzip gzip zlib [after] $ pacman -Qq gzip gzip we need list local package accurately -s, --search <regexp> Search each locally-installed package for names or descriptions that match regexp. When including multiple search terms, only packages with descriptions matching ALL of those terms are returned. ref: https://archlinux.org/pacman/pacman.8.html Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/archlinux b/distro/archlinux index 3942080a6..d7b3d8d78 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -88,7 +88,7 @@ distro_install_depends() for pkg in $packages do - pacman -Qqs --noconfirm $pkg || pacman -Sy --noconfirm $pkg + pacman -Qq --noconfirm $pkg || pacman -Sy --noconfirm $pkg done done } -- 2.23.0

1 0

[PATCH v5 compass-ci 3/3] container/defconfig: load config yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

Load "/etc/compass-ci/register/*.yaml", get register account info from cci defaults config. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index fd13312..f9b0824 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -9,6 +9,7 @@ def cci_defaults hash = {} Dir.glob(['/etc/compass-ci/defaults/*.yaml', '/etc/compass-ci/accounts/*.yaml', + '/etc/compass-ci/register/*.yaml', "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| hash.update YAML.load_file(file) || {} end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] container/auto-submit: bind mount volume
by Liu Yinsi 03 Feb '21

03 Feb '21

mount /etc/compass-ci/register, to get auto-submit account info(my_name, my_email) from config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/auto-submit/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/auto-submit/start b/container/auto-submit/start index 4632ab0..79463f6 100755 --- a/container/auto-submit/start +++ b/container/auto-submit/start @@ -21,6 +21,7 @@ cmd = %W[ -v #{ENV['LKP_SRC']}:#{DOCKER_LKP} -v /etc/localtime:/etc/localtime:ro -v /etc/compass-ci/defaults:/etc/compass-ci/defaults:ro + -v /etc/compass-ci/register:/etc/compass-ci/register:ro -v /srv/git:/srv/git -w /c/compass-ci/sbin alpine:auto-submit -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] sbin/auto-submit: config account yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

[how] before: define my_name, my_email, lab, then store into ES and config yaml after: find my_name, my_email from /etc/compass-ci/register/*.yaml, then config yaml [why] store account info into ES, this part already done by container service register-accounts, so when start auto-submit service, no need to store data, only config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/auto_submit | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 5648d59..c59deb7 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -6,7 +6,9 @@ require 'bunny' require 'json' require 'yaml' +require 'set' require_relative '../lib/build_my_info_client' +require "#{ENV['CCI_SRC']}/container/defconfig" # receive message and auto submit job class AutoSubmit @@ -86,15 +88,23 @@ class AutoSubmit end end -option = { - my_name: 'Auto Submit Robot', - my_email: 'autosubmit@localhost', - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp -} +def my_email + names = Set.new %w[auto-submit] + + defaults = relevant_defaults(names) + defaults['auto-submit']['my_email'] +end + +def config_yaml + build_my_info = BuildMyInfo.new(my_email) + + my_info = build_my_info.search_my_info + build_my_info.config_default_yaml(my_info) + build_my_info.config_lab_yaml(my_info) +end # fix verification problems when submit jobs by auto-submit service -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +config_yaml auto_submitter = AutoSubmit.new auto_submitter.listen -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] sparrow/1-storage: optimise format
by Liu Yinsi 03 Feb '21

03 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 47 ++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 0e8d722..0713d7e 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -1,26 +1,37 @@ #! /bin/bash -chown lkp:lkp /srv/result -chown lkp:lkp /srv/initrd -chown -R lkp:lkp /srv/es -chown -R mailer:team /srv/cci/Maildir -chown lkp:lkp /srv/cache/netdata_cache -chown lkp:lkp /srv/cache/netdata_lib -chown lkp:lkp /srv/cci/serial/fluentd-pos -chown lkp:lkp /srv/cci/libvirt-xml +chmod_dir=( + /srv/result + /srv/es/logging-es + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/logs + /srv/cci/libvirt-xml + /srv/cci/serial/fluentd-pos + /srv/cci/Maildir +) + +chmod 775 "${chmod_dir[@]}" + +chown_dir=( + /srv/result + /srv/initrd + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml +) + +chown lkp:lkp "${chown_dir[@]}" + chown lkp:committer /srv/git -chgrp team /srv/cci/serial/logs -chmod 775 /srv/result -chmod 775 /srv/es/logging-es -chmod 775 /srv/es -chmod 775 /srv/cache/netdata_cache -chmod 775 /srv/cache/netdata_lib -chmod 775 /srv/cci/serial/logs -chmod 775 /srv/cci/libvirt-xml -chmod 775 /srv/cci/serial/fluentd-pos -chmod -R 775 /srv/cci/Maildir +chown mailer:team /srv/cci/Maildir +chgrp team /srv/cci/serial/logs chgrp team /srv/dc + chmod g+ws /srv/dc find /tftpboot -type d | xargs chmod 775 -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: optimize load config
by Wei Jihui 03 Feb '21

03 Feb '21

[why] current load config order can not load physical machine [how] optimize load config for: 1. load /etc/compass-ci/defaults/*.yaml load ~/.config/compass-ci/defaults/*.yaml --> get lab 2. load ~/.config/compass-ci/include/lab/$lab.yaml -->get my_token 3. now user get testbox, load load hosts config 4. load other file: like lkp-tests/include/* Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 477a8a12..9174cadc 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -367,26 +367,15 @@ class Job return @job.include?('tbox_group') end - def get_lab_hosts_file - lab_hosts_file_name = @job['testbox'] - lab_hosts_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{lab_hosts_file_name}" - - if File.file?(lab_hosts_file) - lab_hosts_file - else - nil - end - end - def get_hosts_file - lab_hosts_file = get_lab_hosts_file - return lab_hosts_file if lab_hosts_file + testbox_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{@job['testbox']}" + return testbox_file if File.file?(testbox_file) - hosts_file_name = @job['tbox_group'].split(/\.|--/)[0] - hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" - return hosts_file if File.file?(hosts_file) + tbox_group = @job['tbox_group'].split(/\.|--/)[0] + tbox_group_file = "#{LKP_SRC}/hosts/#{tbox_group}" + return tbox_group_file if File.file?(tbox_group_file) - raise ArgumentError, "hosts file not exist: #{hosts_file_name}, maybe need check testbox field" + raise ArgumentError, "hosts file not exist: #{tbox_group}, maybe need check testbox field" end def include_files @@ -438,6 +427,9 @@ class Job lab_yaml = File.join(self_config_path, 'include/lab', "#{@defaults['lab']}.yaml") load_one_defaults(lab_yaml, @job) + + merge_defaults + load_hosts_config end def load_defaults(first_time = true) @@ -749,7 +741,6 @@ class Job each_job_init job = deepcopy(@job) load_defaults - load_hosts_config each_job_init each_job(&block) @jobs.each do |hash| @@ -757,7 +748,6 @@ class Job @job = deepcopy(job) @job.merge!(hash) load_defaults - load_hosts_config each_job_init each_job(&block) end -- 2.23.0

1 0

[PATCH v3 compass-ci] delimiter/utils.rb: fix get the wrong job id when submit job failed
by Cao Xueliang 03 Feb '21

03 Feb '21

If we submit job failed, we get the response "xxx job id=0, error: xxx". We should match the response at first and return nil, otherwise we will get the wrong job id. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41b65de..3120ca6 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -69,6 +69,7 @@ module Utils save_job_to_yaml(job, PROCESS_JOB_YAML) response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" + return nil if response =~ /job id=0/ return $1 if response =~ /job id=(.*)/ return nil -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

2 2

[PATCH v2 lkp-tests] installer/oracle: add shebang
by Wang Yong 03 Feb '21

03 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/oracle | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/installer/oracle b/distro/installer/oracle index b09abc945..11df716d6 100755 --- a/distro/installer/oracle +++ b/distro/installer/oracle @@ -1,3 +1,5 @@ +#!/bin/sh + if ! rpm -q epel-release >/dev/null; then yum install -y epel-release yum makecache fast -- 2.23.0

1 0