February 2021 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] distro/archlinux: redirect pacman output
by Wang Yong 04 Feb '21

04 Feb '21

[before] $ pacman -Qq --noconfirm wget wget $ echo $? 0 [after] $ pacman -Qq --noconfirm wget > /dev/null $ echo $? 0 Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/archlinux b/distro/archlinux index 43c3aa13a..59c8da1d6 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -88,7 +88,7 @@ distro_install_depends() for pkg in $packages do - pacman -Qq --noconfirm $pkg || pacman -Sy --noconfirm $pkg + pacman -Qq --noconfirm $pkg > /dev/null || pacman -Sy --noconfirm $pkg done done } -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/lib: introduce libvirt.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

gem install ruby-libvirt ruby-libvirt providers Ruby binding for libvirt. you can called it based on a domain.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/libvirt.rb | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 providers/lib/libvirt.rb diff --git a/providers/lib/libvirt.rb b/providers/lib/libvirt.rb new file mode 100644 index 0000000..e4e1240 --- /dev/null +++ b/providers/lib/libvirt.rb @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'libvirt' + +# connect libvirt daemon +class LibvirtConnect + def initialize + @conn = Libvirt.open('qemu:///system') + end + + def define(xml) + @dom = @conn.define_domain_xml(File.read(xml)) + end + + def create + @dom.create + end + + def wait + loop do + sleep 10 + break unless @dom.active? + end + end + + def close + @conn.close + end +end -- 2.23.0

1 0

[PATCH compass-ci] providers/lib: introduce libvirt.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

gem install ruby-libvirt ruby-libvirt providers Ruby binding for libvirt. you can called it based on a domain.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/libvirt.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 providers/lib/libvirt.rb diff --git a/providers/lib/libvirt.rb b/providers/lib/libvirt.rb new file mode 100644 index 0000000..6f39651 --- /dev/null +++ b/providers/lib/libvirt.rb @@ -0,0 +1,33 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'libvirt' + +# connect libvirt daemon +class LibvirtConnect + def initialize + @conn = Libvirt.open('qemu:///system') + end + + def define(xml) + @dom = @conn.define_domain_xml(File.read(xml)) + end + + def create + @dom.create + end + + def wait + loop do + sleep 10 + unless @dom.active? + break + end + end + end + + def close + @conn.close + end +end -- 2.23.0

1 0

[PATCH lkp-tests] distro/*: fix output information
by Wang Yong 04 Feb '21

04 Feb '21

[before] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench.cgz [after] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench-cpu.cgz Thanks, Yong --- distro/aliyun | 2 +- distro/archlinux | 2 +- distro/clear | 2 +- distro/debian | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/distro/aliyun b/distro/aliyun index 598243d8e..83727ce32 100755 --- a/distro/aliyun +++ b/distro/aliyun @@ -94,7 +94,7 @@ yum_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.rpm > $pack_to/.${BM_NAME}.packages diff --git a/distro/archlinux b/distro/archlinux index d7b3d8d78..43c3aa13a 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -64,7 +64,7 @@ pacman_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.pkg.tar* > $pack_to/.${BM_NAME}.packages diff --git a/distro/clear b/distro/clear index f21de89c3..5d6fb0e46 100755 --- a/distro/clear +++ b/distro/clear @@ -135,7 +135,7 @@ pack_bundles_files() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } # 1. install all un-installed bundles and record it to uninstalled_bundles diff --git a/distro/debian b/distro/debian index 313f73e35..21dafe21b 100755 --- a/distro/debian +++ b/distro/debian @@ -120,7 +120,7 @@ pack() chown .lkp $pack_to/${BM_NAME}_$date.cgz \ $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } resolve_depends() -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 04 Feb '21

04 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..d4d70c3 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) + +[ ! -n $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 04 Feb '21

04 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..e0197b9 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,12 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin/auto_submit: don't use auto_submit.yaml any more
by Li Yuanchao 04 Feb '21

04 Feb '21

Now DEFAULTS files are added to upstream-repos to keep submit commands. And these info would be taken by messages which the container git-mirror send. So auto_submit parses the messages and get the submit commands. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 14 +++++--------- sbin/auto_submit.yaml | 25 ------------------------- 2 files changed, 5 insertions(+), 34 deletions(-) delete mode 100644 sbin/auto_submit.yaml diff --git a/sbin/auto_submit b/sbin/auto_submit index c59deb7..410445d 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -17,7 +17,6 @@ class AutoSubmit connection.start channel = connection.create_channel @queue = channel.queue('new_refs') - @repo2job = YAML.load_file("#{$PROGRAM_NAME}.yaml") end def get_pkgbuild_repo(repo_array) @@ -57,20 +56,17 @@ class AutoSubmit return unless pkgbuild_repo submit_argv.push("pkgbuild_repo=#{pkgbuild_repo}") - repo2job_key = @repo2job[git_repo] ? git_repo : 'archlinux' - else - return unless @repo2job[git_repo] - - repo2job_key = git_repo end - [submit_argv, repo2job_key] + submit_argv end def submit_job(newrefs_info) - submit_argv, repo2job_key = get_argvs(newrefs_info) + submit_argv = get_argvs(newrefs_info) return unless submit_argv - @repo2job[repo2job_key].each do |argv_config| + return unless newrefs_info['submit'] + + newrefs_info['submit'].each do |argv_config| argvs = Array.new(submit_argv) argvs.push(argv_config) submit(newrefs_info, argvs) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml deleted file mode 100644 index c368ad6..0000000 --- a/sbin/auto_submit.yaml +++ /dev/null @@ -1,25 +0,0 @@ -a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml -c/compass-ci/compass-ci: -- testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -l/linux/linux: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/linux-next: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/openeuler-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/debian-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/archlinux-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=dc-8g os=openeuler os_version=20.03-pre os_mount=container os_arch=aarch64 docker_image=openeuler:20.03-pre queue=dc-build-pkg build-pkg.yaml -- testbox=dc-8g os=archlinux os_version=2021-01-06-pre os_mount=container os_arch=aarch64 docker_image=archlinux:2021-01-06-pre queue=dc-build-pkg build-pkg.yaml -- 2.23.0

1 0

[PATCH compass-ci 1/2] git_mirror.rb: optimize the use of hash.merge
by Li Yuanchao 04 Feb '21

04 Feb '21

1. hash_a.merge(hash_b), this means if the two hashes have some same keys, the last one will cover the first one. I change it so that if a repo file has the same key with DEFAULTS file, its own will be kept. 2. fix that if there's no DEFAULTS file, the merge_defaults method will return nil. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index fa2e0cc..8898a44 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -453,10 +453,12 @@ class MirrorMain def merge_defaults(git_repo) defaults_key = File.dirname(git_repo) while defaults_key != '.' - return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + return @defaults[defaults_key].merge(@git_info[git_repo]) if @defaults[defaults_key] defaults_key = File.dirname(defaults_key) end - return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + return @defaults['default'].merge(@git_info[git_repo]) if @defaults['default'] + + return @git_info[git_repo] end end -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix no my_token
by Luan Shengde 04 Feb '21

04 Feb '21

the lib/build_my_info_client will not generate my_token automatically. and lib/build_my_info_client do not support update if account exists. in the ES. use lib/build-my-info, it will gererate my_token if it's not in my_info. fix rubocop warning: register-accounts:11:18: W: Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used. content.each do |key, value| ^^^ Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index 519f289..e4d39ff 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,16 +5,15 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" +require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" content = YAML.load_file('/etc/compass-ci/register/register.yaml') -content.each do |key, value| +content.each do |_key, value| my_info = {} my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] - build_my_info = BuildMyInfo.new(my_info['my_email']) - build_my_info.store_account_info(my_info) + build_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..749f3667b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,12 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + case line.chomp! + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0