- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 1/3] container/assist-cci: Dockerfile
by Ren Wen 15 Jan '21

15 Jan '21

Signed-off-by: Ren Wen <15991987063(a)163.com> --- container/assist-cci/Dockerfile | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 container/assist-cci/Dockerfile diff --git a/container/assist-cci/Dockerfile b/container/assist-cci/Dockerfile new file mode 100644 index 0000000..49007f7 --- /dev/null +++ b/container/assist-cci/Dockerfile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM alpine:3.11 + +MAINTAINER Ren Wen <15991987063(a)163.com> + +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ + apk update && \ + apk add --no-cache 'ruby-dev' 'ruby-etc' \ + 'g++' 'gcc' 'pcre' 'libevent' 'make' 'git' 'bash' 'grep' 'coreutils' 'curl' 'util-linux' + +RUN umask 002 && \ + echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ + gem install activesupport json yaml sinatra puma -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: add package
by Liu Yinsi 15 Jan '21

15 Jan '21

[why] 1. providers/qemu/kvm.sh needs wget, curl, qemu command, so add it. 2. current debian 'qemu' package makes no sense anymore, and it is very unlikely one will need all of them together, so delete it, install qemu-system-ARCH packages, can see:https://packages.debian.org/sid/qemu Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 2 ++ sparrow/0-package/os/debian | 6 +++++- sparrow/0-package/os/openEuler | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 8df36c0..b25dade 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -22,6 +22,8 @@ pkgs=( busybox iproute hostname + wget + curl ) yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index c45e8c2..4d13413 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -13,13 +13,17 @@ pkgs=( docker.io ruby-full ruby-dev - qemu dmidecode lockfile-progs bridge-utils busybox iproute2 hostname + wget + curl + qemu-system-aarch64 + qemu-system-x86 + qemu-system-riscv64 ) apt-get update diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 37c74d0..04996f5 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -22,6 +22,7 @@ pkgs=( iproute hostname libvirt + curl ) yum install -y --skip-broken "${pkgs[@]}" -- 2.23.0

2 2

[PATCH compass-ci 1/2] lib/error_messages.rb: obtain_error_messages_by_error_id function add keyword argument
by Lin Jiaxin 15 Jan '21

15 Jan '21

Determine whether to return the error line or error messages by the value of eturn_error_line Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/error_messages.rb | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/error_messages.rb b/lib/error_messages.rb index 3e1564b..81e0034 100644 --- a/lib/error_messages.rb +++ b/lib/error_messages.rb @@ -40,12 +40,16 @@ class ErrorMessages return @error_messages end - def obtain_error_messages_by_error_id(error_id) + def obtain_error_messages_by_error_id(error_id, return_error_line = false) error_messages_by_error_id = [] error_messages = obtain_error_messages error_messages.each do |k, v| if ('build-pkg.' + build_pkg_error_id(k)) == error_id - error_messages_by_error_id += v.to_a + if return_error_line + error_messages_by_error_id << k + else + error_messages_by_error_id += v.to_a + end end end error_messages_by_error_id -- 2.23.0

2 1

[PATCH lkp-tests] lib/yaml.rb: load_yaml_merge add arg
by Wei Jihui 15 Jan '21

15 Jan '21

add template_context for load_yaml_merge, it is used by load_yaml. when template_context is nil, there is not impact for load_yaml_merge. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/yaml.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/yaml.rb b/lib/yaml.rb index 9c9b8088..4290d594 100755 --- a/lib/yaml.rb +++ b/lib/yaml.rb @@ -52,13 +52,13 @@ def load_yaml_with_flock(file, timeout = nil) end end -def load_yaml_merge(files) +def load_yaml_merge(files, template_context = nil) all = {} files.each do |file| next unless File.size? file begin - yaml = load_yaml(file) + yaml = load_yaml(file, template_context) all.update(yaml) rescue StandardError => e warn "#{e.class.name}: #{e.message.split("\n").first}: #{file}" -- 2.23.0

1 0

[PATCH v1 compass-ci] lib/job.cr: fix failed to upload package
by Cao Xueliang 15 Jan '21

15 Jan '21

[why] If the JSON::Any type pkg_style value is nil, we don't set the default upload path. [how] If the JSON::Any type pkg_style value is nil, we set it to empty hash that will get a default upload path. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 1d25236..07bfda0 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -256,7 +256,9 @@ class Job pkg_style = @hash[item]? break if pkg_style end - return nil unless pkg_style && pkg_style != nil + return nil unless pkg_style + + pkg_style = JSON.parse("{}") if pkg_style == nil tmp_os = pkg_style["os"]? == nil ? "#{os}" : pkg_style["os"] tmp_os_arch = pkg_style["os_arch"]? == nil ? "#{os_arch}" : pkg_style["os_arch"] -- 2.23.0

3 3

[PATCH compass-ci 2/2] lib/compare_error_messages.rb: add the credibility check of job
by Lin Jiaxin 15 Jan '21

15 Jan '21

defect: may match the wrong result error_id="build-pkg.pixz.c:warning:'subsuf'defined-but-not-used[-Wunused-function]" filename="pixz.c" correct result: gcc -DHAVE_CONFIG_H -I. -I.. -D_FORTIFY_SOURCE=2 -pthread -Wall -Wno-unknown-pragmas -march=native -O2 -pipe -fstack-protector-strong -fno-plt -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -MT pixz-pixz.o -MD -MP -MF .deps/pixz-pixz.Tpo -c -o pixz-pixz.o `test -f 'pixz.c' || echo './'`pixz.c ^^^^^^ wrong result: mv -f .deps/pixz.cpu.Tpo .deps/pixz.cpu.Po ^^^^^^ Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 0b51fc3..a959c61 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -8,6 +8,39 @@ require_relative 'es_query' require_relative 'error_messages' require_relative "#{ENV['LKP_SRC']}/lib/common" +# previous_job_id's error_ids include error_id +# If later_job_id's output include error filename that extract from error_id, +# the later_job_id is credible. +def credible?(previous_job_id, later_job_id, error_id) + es = ESQuery.new + + previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') + later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') + + return false if filenames_check(previous_result_file, later_result_file, error_id).value?(false) + + return true +end + +def filenames_check(previous_result_file, later_result_file, error_id) + filenames = [] + filenames_check = Hash.new { |h, k| h[k] = false } + + error_lines = ErrorMessages.new(previous_result_file).obtain_error_messages_by_error_id(error_id, true) + error_lines.each do |error_line| + filenames << $1 if error_line =~ /(.*)(:.*){2}: (error|warning):/ + end + + File.open(later_result_file).each_line do |line| + filenames.each do |filename| + filenames_check[filename] + filenames_check[filename] = true if line.include?(filename) + end + end + + return filenames_check +end + def get_compare_result(previous_job_id, later_job_id) es = ESQuery.new -- 2.23.0

2 1

[PATCH lab-z9] add a117 host info
by Bai Jing 15 Jan '21

15 Jan '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- hosts/taishan200-2280-2s64p-256g--a117 | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a117 diff --git a/hosts/taishan200-2280-2s64p-256g--a117 b/hosts/taishan200-2280-2s64p-256g--a117 new file mode 100644 index 0000000..0f10495 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a117 @@ -0,0 +1,19 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886b57d1 + - /dev/disk/by-id/scsi-350000399886b720d +mac_addr: + - 44:67:47:85:f4:0b + - 44:67:47:85:f4:0c + - 44:67:47:85:f4:0d + - 44:67:47:85:f4:0e + - cc:05:77:ee:b3:ff + - cc:05:77:ee:b4:00 + - cc:05:77:ee:b4:01 + - cc:05:77:ee:b4:02 +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.11.10 -- 2.23.0

1 0

[PATCH v2 compass-ci 2/3] container - assist-cci: build & start script
by Ren Wen 15 Jan '21

15 Jan '21

Mount '/srv/cci' and upload config to dir: 'libvirt-xml'. Use port 'ASSIST_CCI_PORT' or '11392' by default. Signed-off-by: Ren Wen <15991987063(a)163.com> --- container/assist-cci/build | 5 ++++ container/assist-cci/start | 47 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) create mode 100755 container/assist-cci/build create mode 100755 container/assist-cci/start diff --git a/container/assist-cci/build b/container/assist-cci/build new file mode 100755 index 0000000..f82de7f --- /dev/null +++ b/container/assist-cci/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t assist-cci . diff --git a/container/assist-cci/start b/container/assist-cci/start new file mode 100755 index 0000000..b65f814 --- /dev/null +++ b/container/assist-cci/start @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +names = Set.new %w[ + ASSIST_CCI_HOST + ASSIST_CCI_PORT +] + +defaults = relevant_defaults(names) +env = docker_env(defaults) + +DEFAULT_LKP = '/c/lkp-tests' +DEFAULT_CCI = '/c/compass-ci' +DEFAULT_CONFIG_DIR = '/etc/compass-ci/defaults' +ASSIST_CCI_PORT = defaults['ASSIST_CCI_PORT'] || 11392 +SRV_CCI_PATH = '/srv/cci' +LIBVIRT_XML_PATH = "#{SRV_CCI_PATH}/libvirt-xml" +docker_rm 'assist-cci' + +cmd = %w[ + docker run + --name assist-cci + --restart=always + -d +] + env + %W[ + -e LIBVIRT_XML_PATH=#{LIBVIRT_XML_PATH} + -e ASSIST_CCI_PORT=#{ASSIST_CCI_PORT} + -e LKP_SRC=#{DEFAULT_LKP} + -e CCI_SRC=#{DEFAULT_CCI} + -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} + -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} + -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro + -p #{ASSIST_CCI_PORT}:#{ASSIST_CCI_PORT} + -v /etc/localtime:/etc/localtime:ro + -v #{SRV_CCI_PATH}:#{SRV_CCI_PATH} + -w #{DEFAULT_CCI}/container/assist-cci/ + assist-cci +] + +cmd += ['sh', '-c', 'umask 002 && ruby ./routes.rb'] + +system(*cmd) -- 2.23.0

2 2

[PATCH v2 compass-ci 1/3] container - assist-cci: Dockerfile
by Ren Wen 15 Jan '21

15 Jan '21

Signed-off-by: Ren Wen <15991987063(a)163.com> --- container/assist-cci/Dockerfile | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 container/assist-cci/Dockerfile diff --git a/container/assist-cci/Dockerfile b/container/assist-cci/Dockerfile new file mode 100644 index 0000000..16bba06 --- /dev/null +++ b/container/assist-cci/Dockerfile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM alpine:3.11 + +MAINTAINER 15991987063(a)163.com + +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ + apk update && \ + apk add --no-cache 'ruby-dev' 'ruby-etc' \ + 'g++' 'gcc' 'pcre' 'libevent' 'make' 'git' 'bash' 'grep' 'coreutils' 'curl' 'util-linux' + +RUN umask 002 && \ + echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ + gem install activesupport json yaml sinatra puma -- 2.23.0

3 4

[PATCH compass-ci] auto_submit: don't submit job with too old commit
by Li Yuanchao 15 Jan '21

15 Jan '21

If the last commit of a repository is too old, such as 'two months ago', that means the repository is not active. We don't care such repositories. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sbin/auto_submit b/sbin/auto_submit index ecc4d70..5648d59 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -31,6 +31,9 @@ class AutoSubmit def submit(newrefs_info, submit_argv) newrefs_info['new_refs']['heads'].each do |branch, commit_id| + inactive_time = %x(git -C /srv/git/#{newrefs_info['git_repo']}.git log --pretty=format:"%cr" -1 #{commit_id}) + next if inactive_time =~ /(month|year)/ + real_argvs = Array.new(submit_argv) real_argvs.push("upstream_branch=#{branch.delete_prefix('refs/heads/')}") real_argvs.push("upstream_commit=#{commit_id}") -- 2.23.0

1 0