- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 3/3] libvirt/lib: request scheduler to consume job
by Xiao Shenwei 23 Jan '21

23 Jan '21

request url: http://#{@sched_host}:#{@sched_port}/boot.libvirt/mac/#{@info['mac']} Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/consumer.rb | 85 +++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 providers/libvirt/lib/consumer.rb diff --git a/providers/libvirt/lib/consumer.rb b/providers/libvirt/lib/consumer.rb new file mode 100644 index 0000000..f1477fe --- /dev/null +++ b/providers/libvirt/lib/consumer.rb @@ -0,0 +1,85 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'yaml' +require 'json' +require 'rest-client' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +# register hostname mac queues +# request scheduler +class Consumer + attr_reader :info + + def initialize(hostname, queues, logger) + @logger = logger + @info = { 'hostname' => hostname, 'queues' => queues } + host_exists + parse_mac_from_hostname + defconfig + register_info + end + + def connect + res = RestClient.get("http://#{@sched_host}:#{@sched_port}/boot.libvirt/mac/#{@info['mac']}") + if res.empty? + @logger.error('Can not connect scheduler') + raise 'Can not connect scheduler' + end + JSON.parse(res) + end + + def clean_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host_mac?mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host2queues?host=#{@info['hostname']}", {} + ) + end + + private + + def defconfig + names = Set.new %w[ + SCHED_HOST + SCHED_PORT + ] + defaults = relevant_defaults(names) + @sched_host = defaults['SCHED_HOST'] || '172.17.0.1' + @sched_port = defaults['SCHED_PORT'] || 3000 + @logger.info("SCHED_HOST: #{@sched_host}") + @logger.info("SCHED_PORT: #{@sched_port}") + end + + def parse_mac_from_hostname + # TODO + # use digest/md5 replace + cmd = %(echo #{@info['hostname']} | + md5sum | + sed "s/^\$..\$\$..\$\$..\$\$..\$\$..\$.*$/0a-\\1-\\2-\\3-\\4-\\5/" + ) + @info['mac'] = %x(#{cmd}).chomp + @logger.info("Mac address: #{@info['mac']}") + end + + def register_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host_mac?hostname=#{@info['hostname']}&mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host2queues?host=#{@info['hostname']}&queues=#{@info['queues']}", {} + ) + end + + def host_exists + @info['host'] = @info['hostname'].split('.')[0] + host_file = "#{ENV['LKP_SRC']}/hosts/#{@info['host']}" + return if FileTest.exists?(host_file) + + @logger.error("The file does't exist: #{host_file}") + raise "The file does't exist: #{host_file}" + end +end -- 2.23.0

1 0

[PATCH v3 compass-ci 1/3] providers/libvirt: add a libvirt provider
by Xiao Shenwei 23 Jan '21

23 Jan '21

libvirt is a vm management tool, it based on a domain.xml to run vm each element of xml can be parsed qemu-kvm parameter. so we can configure xml file to run different type of vm Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/client.rb | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 providers/libvirt/client.rb diff --git a/providers/libvirt/client.rb b/providers/libvirt/client.rb new file mode 100755 index 0000000..29cdaf3 --- /dev/null +++ b/providers/libvirt/client.rb @@ -0,0 +1,23 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative 'lib/mylog' +require_relative 'lib/consumer' + +def main(hostname, queues) + puts "cat #{hostname}.log" + logger = Mylog.new("#{hostname}.log") + consumer = Consumer.new(hostname, queues, logger) + response = consumer.connect + if response.nil? || response['job_id'].empty? + logger.warn('no job now') + puts '----------' + puts 'no job now' + puts '----------' + return + end + ## clean env + consumer.clean_info +end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/README.md: reformatting the document
by Li Ping 23 Jan '21

23 Jan '21

[why] In markdown documents, the newline effect is not shown, content not placed in ``` ''' would be displayed as a single line, readability would be reduced. Signed-off-by: Li Ping <1477412247(a)qq.com> --- sparrow/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sparrow/README.md b/sparrow/README.md index 794e433..3d7cdd0 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -72,8 +72,10 @@ ``` 执行上述命令正常情况下会提示信息如下: + ``` submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory + ``` compass-ci搭建完毕，执行步骤4下载所需要的rootfs文件就可以开始进行测试了。 4. 下载rootfs文件（根据所需要的rootfs在[该目录](http://124.90.34.227:11300/os/)下获取对应版本的cgz文件） -- 2.23.0

3 2

[PATCH v2 lkp-tests 3/3] distro/depends: add nginx-server-dev
by Wei Jihui 23 Jan '21

23 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/nginx-server-dev | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 distro/depends/nginx-server-dev diff --git a/distro/depends/nginx-server-dev b/distro/depends/nginx-server-dev new file mode 100644 index 00000000..170af747 --- /dev/null +++ b/distro/depends/nginx-server-dev @@ -0,0 +1,2 @@ +pcre-devel +libpcre3-dev -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/3] pkg/nginx-server: add ngixn conf
by Wei Jihui 23 Jan '21

23 Jan '21

nginx-server test need forbidden nf_conntrack when os setup Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/nginx-server/PKGBUILD | 10 ++++++++-- pkg/nginx-server/blacklist.conf | 10 ++++++++++ 2 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 pkg/nginx-server/blacklist.conf diff --git a/pkg/nginx-server/PKGBUILD b/pkg/nginx-server/PKGBUILD index 141722e1..ac458776 100644 --- a/pkg/nginx-server/PKGBUILD +++ b/pkg/nginx-server/PKGBUILD @@ -5,8 +5,8 @@ arch=('i386' 'x86_64') url="https://nginx.org" license=('BSD') nginx_name=("nginx-${pkgver}.${pkgrel}") -source=("https://nginx.org/download/${nginx_name}.tar.gz") -md5sums=('SKIP') +source=("https://nginx.org/download/${nginx_name}.tar.gz" "blacklist.conf") +md5sums=('SKIP' 'SKIP') build() { @@ -19,6 +19,12 @@ build() package() { + mkdir -p "$pkgdir/rootfs/addon/etc/modprobe.d/" + cp blacklist.conf "$pkgdir/rootfs/addon/etc/modprobe.d/" + + mkdir -p "$pkgdir/lkp/benchmarks/${pkgname}/" + cp -r "$LKP_SRC/pkg/${pkgname}/nginx_conf" "$pkgdir/lkp/benchmarks/${pkgname}/" + cd "${srcdir}/${nginx_name}" make install DESTDIR="${pkgdir}" } diff --git a/pkg/nginx-server/blacklist.conf b/pkg/nginx-server/blacklist.conf new file mode 100644 index 00000000..e8798cc7 --- /dev/null +++ b/pkg/nginx-server/blacklist.conf @@ -0,0 +1,10 @@ +install nf_conntrack /bin/false +blacklist nf_conntrack +blacklist nf_conntrack_ipv6 +blacklist xt_conntrack +blacklist nf_conntrack_ftp +blacklist xt_state +blacklist iptable_nat +blacklist ipt_REDIRECT +blacklist nf_nat +blacklist nf_conntrack_ipv4 -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/3] pkg: rename nginx to nginx-server
by Wei Jihui 23 Jan '21

23 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/{nginx => nginx-server}/PKGBUILD | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) rename pkg/{nginx => nginx-server}/PKGBUILD (63%) diff --git a/pkg/nginx/PKGBUILD b/pkg/nginx-server/PKGBUILD similarity index 63% rename from pkg/nginx/PKGBUILD rename to pkg/nginx-server/PKGBUILD index 989676c4..141722e1 100644 --- a/pkg/nginx/PKGBUILD +++ b/pkg/nginx-server/PKGBUILD @@ -1,23 +1,24 @@ -pkgname=nginx +pkgname=nginx-server pkgver=1.14 pkgrel=2 arch=('i386' 'x86_64') url="https://nginx.org" license=('BSD') -source=("https://nginx.org/download/nginx-${pkgver}.${pkgrel}.tar.gz") +nginx_name=("nginx-${pkgver}.${pkgrel}") +source=("https://nginx.org/download/${nginx_name}.tar.gz") md5sums=('SKIP') build() { benchmark_path="/lkp/benchmarks/${pkgname}" mkdir -p "${benchmark_path}" - cd "${srcdir}/${pkgname}-${pkgver}.${pkgrel}" + cd "${srcdir}/${nginx_name}" ./configure --with-http_ssl_module --prefix="${benchmark_path}" make -j4 } package() { - cd "${srcdir}/${pkgname}-${pkgver}.${pkgrel}" + cd "${srcdir}/${nginx_name}" make install DESTDIR="${pkgdir}" } -- 2.23.0

1 0

[PATCH lkp-tests] bin/lkp-setup-rootfs: refactor the chmod_root_ssh()
by Xu Xijian 22 Jan '21

22 Jan '21

Add blank line after return cmd. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- bin/lkp-setup-rootfs | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index cf48c8f6b..bc0f04b2a 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -245,6 +245,7 @@ chmod_root_ssh() { [ -d "/root/.ssh" ] || return [ -f "/root/.ssh/id_rsa" ] || return + chmod 700 /root/.ssh/ chmod 600 /root/.ssh/id_rsa } -- 2.23.0

2 1

[PATCH compass-ci] sparrow/1-storage: remove mount os rootfs
by Liu Yinsi 22 Jan '21

22 Jan '21

[why] for locally deploy Compass CI with one click, we open directory /srv/os to download os rootfs, rather than open shared directory to mount. so remove it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/tiny | 3 --- 1 file changed, 3 deletions(-) diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index eb7270d..e589937 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -26,6 +26,3 @@ dirs=( ) mkdir -p "${dirs[@]}" - -mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr -mount -t cifs //$LKP_SERVER/initrd /srv/initrd -o port=446,guest,ro,hard,vers=1.0,noacl,nouser_xattr -- 2.23.0

1 0

[PATCH lkp-tests 2/2] tests/deploy-cci: mount os rootfs
by Liu Yinsi 22 Jan '21

22 Jan '21

[why] mount our server's os rootfs directory is a dedicated way for $LKP_SRC/test/deploy-cci, it is not a general way for deploy Compass CI with one click, put it in tests/deploy-cci more reasonable, not in $CCI_SRC/sparrow script. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- tests/deploy-cci | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/deploy-cci b/tests/deploy-cci index 8e6acbac4..e2a7bd320 100755 --- a/tests/deploy-cci +++ b/tests/deploy-cci @@ -35,5 +35,13 @@ deploy() ./install-tiny } +mount_rootfs() +{ + mkdir -p /srv/{os,initrd} + mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr + mount -t cifs //$LKP_SERVER/initrd /srv/initrd -o port=446,guest,ro,hard,vers=1.0,noacl,nouser_xattr +} + export_env +mount_rootfs deploy -- 2.23.0

1 0

[PATCH lkp-tests 1/2] distro/depends: fix failed to mount
by Liu Yinsi 22 Jan '21

22 Jan '21

[error] root@test-lys /c/compass-ci/container/os-cifs# mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr mount: /srv: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program. [why] missing depends cifs-utils causes mount error Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/deploy-cci | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/deploy-cci b/distro/depends/deploy-cci index 5664e303b..c0d6a5310 100644 --- a/distro/depends/deploy-cci +++ b/distro/depends/deploy-cci @@ -1 +1,2 @@ git +cifs-utils -- 2.23.0

1 0