- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] sparrow/8-testbox: add multi-qemu
by Liu Yinsi 29 Apr '21

29 Apr '21

run qemu testboxes for user locally deploy ci env, if the host is virtual, not run qemu. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/8-testbox/multi-qemu | 7 +++++++ sparrow/install-tiny | 1 + 2 files changed, 8 insertions(+) create mode 100755 sparrow/8-testbox/multi-qemu diff --git a/sparrow/8-testbox/multi-qemu b/sparrow/8-testbox/multi-qemu new file mode 100755 index 00000000..8a995cdb --- /dev/null +++ b/sparrow/8-testbox/multi-qemu @@ -0,0 +1,7 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +dmidecode -s system-product-name | grep -iq "virtual" && exit + +nohup $CCI_SRC/providers/multi-qemu -n vm-2p8g -c 10 -q vm-2p8g.$(arch) & diff --git a/sparrow/install-tiny b/sparrow/install-tiny index 40358a4c..a81f2d5c 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -24,3 +24,4 @@ cd $CCI_SRC/sparrow || exit 5-build/register-account 7-systemd/systemd-setup 8-testbox/multi-docker +8-testbox/multi-qemu -- 2.23.0

2 2

[PATCH lkp-tests] jobs: add borrow-30d.yaml
by Liu Yinsi 29 Apr '21

29 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- jobs/borrow-30d.yaml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 jobs/borrow-30d.yaml diff --git a/jobs/borrow-30d.yaml b/jobs/borrow-30d.yaml new file mode 100644 index 000000000..e24fd6480 --- /dev/null +++ b/jobs/borrow-30d.yaml @@ -0,0 +1,14 @@ +suite: borrow +testcase: borrow + +runtime: 30d +ssh_pub_key: <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> +sshd: +# sleep at the bottom +sleep: -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: add gem dep nokogiri for libvirt
by Xu Xijian 29 Apr '21

29 Apr '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/0-package/common | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 82c09ff..a827a65 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -8,7 +8,8 @@ public_network_ok && { gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ - gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch bunny + gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch bunny \ + nokogiri } grep -q '^lkp:' /etc/passwd || useradd -u 1090 lkp -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/0-package: add deps libxml2-devel libxslt-devel
by Xu Xijian 29 Apr '21

29 Apr '21

These 2 deps are for nokogiri, and nokogiri is dep of libvirt. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/0-package/os/centos | 2 ++ sparrow/0-package/os/openEuler | 2 ++ 2 files changed, 4 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index d4d1045..3f7dcac 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -26,6 +26,8 @@ pkgs=( curl sudo iputils + libxml2-devel + libxslt-devel ) yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5829a12..5153239 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -29,6 +29,8 @@ pkgs=( curl sudo iputils + libxml2-devel + libxslt-devel ) public_network_ok && { -- 2.23.0

1 0

[PATCH compass-ci 4/4] lib/mail_job_result.rb: fix incorrect logic
by Lu Weitao 29 Apr '21

29 Apr '21

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/mail_job_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_job_result.rb b/lib/mail_job_result.rb index 389ad26a..744b7d5c 100644 --- a/lib/mail_job_result.rb +++ b/lib/mail_job_result.rb @@ -77,7 +77,7 @@ def get_compare_result(job) commit_id = job['upstream_commit'] return nil unless base_commit && commit_id - condition_list = [{'base_commit' => base_commit}, {'upstream_commit' => commit_id}] + condition_list = [{'upstream_commit' => base_commit}, {'upstream_commit' => commit_id}] options = { :min_samples => min_samples, :no_print => true} matrices_list, suite_list = create_matrices_list(condition_list, options[:min_samples]) -- 2.23.0

1 0

[PATCH compass-ci 3/4] lib/es_query.rb: support unmatch fields
by Lu Weitao 29 Apr '21

29 Apr '21

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/es_query.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 8c5a28fe..9d596bb2 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -15,17 +15,20 @@ class ESQuery end # Example @items: { key1 => value1, key2 => [value2, value3, ..], ...} + # Example @unmatch_items: { key1 => value1, key2 => [value2, value3, ..], ...} # means to query: key1 == value1 && (key2 in [value2, value3, ..]) - def multi_field_query(items, size: 10_000, desc_keyword: nil) + def multi_field_query(items, unmatch_items: {}, size: 10_000, desc_keyword: nil) unless items warn 'empty filter!' exit end query_fields = build_multi_field_subquery_body items + unmatch_fields = build_multi_field_subquery_body unmatch_items query = { query: { bool: { - must: query_fields + must: query_fields, + must_not: unmatch_fields } }, size: size } -- 2.23.0

1 0

[PATCH compass-ci 2/4] lib/compare_matrixes.rb cancel remove 0 changed metric
by Lu Weitao 29 Apr '21

29 Apr '21

[Why] some user are still need check 0 changed metric Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index da2c2b92..cff73cbe 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -191,7 +191,7 @@ def remove_unchanged_field(matrixes_values, suite_list) # matrixes_values.each_key do |success| matrixes_values[success].delete_if do |field| - if suite_list.any? { |suite| field.start_with?(suite) } + if success matrixes_values[success][field][:changed] = true end -- 2.23.0

1 0

[PATCH compass-ci 1/4] sbin/compare: cancel fill 0 to matrix
by Lu Weitao 29 Apr '21

29 Apr '21

[Why] matrix => { iperf.xxx => [100, 80, 0], ... } like above, if a stat_metric fill 0, the avg and stddev will be incorrect Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 13 +++++++++---- lib/matrix2.rb | 13 +------------ 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index fef5714a..da2c2b92 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -129,10 +129,15 @@ def get_values_by_field(matrixes_list, field, matrixes_size, success, options) # values = {} matrixes_list.length.times do |index| - value_list = fill_missing_with_zeros( - matrixes_list[index][field], - matrixes_size[index] - ) + value_list = matrixes_list[index][field] + unless value_list + if success + value_list = [0] unless value_list + else + value_list = [1] unless value_list + end + end + values[index] = get_values(value_list, success) next if index.zero? diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 2fd63243..82ed9a7a 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -62,16 +62,6 @@ def create_stats(result_root) # stats end -def samples_fill_missing_zeros(value, size) - value.concat([0] * (size - value.size)) -end - -def matrix_fill_miss_zeros(matrix, col_size) - matrix.each_value do |value| - samples_fill_missing_zeros(value, col_size) - end -end - # input: job_list # return: matrix of Hash(String, Array(Number)) # Eg: matrix: { @@ -94,8 +84,7 @@ def create_matrix(job_list) matrix[key] << value end end - col_size = job_list.size - matrix_fill_miss_zeros(matrix, col_size) + return matrix, suites end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: add deps libxml2-devel libxslt-devel
by Xu Xijian 29 Apr '21

29 Apr '21

These 2 deps are for nokogiri, and nokogiri is dep of libvirt. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/0-package/os/openEuler | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5829a12..5153239 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -29,6 +29,8 @@ pkgs=( curl sudo iputils + libxml2-devel + libxslt-devel ) public_network_ok && { -- 2.23.0

1 0

[PATCH v2 compass-ci 3/3] sbin/dashboard: add a new script to operate kibana dashboard
by Wu Zhende 29 Apr '21

29 Apr '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/dashboard | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100755 sbin/dashboard diff --git a/sbin/dashboard b/sbin/dashboard new file mode 100755 index 0000000..c15bd75 --- /dev/null +++ b/sbin/dashboard @@ -0,0 +1,67 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'optparse' +require_relative '../lib/constants.rb' +require_relative '../lib/kibana_dashboard.rb' + +opt_type = nil +opt_host = 'kibana' +opt_hash = {} + +options = OptionParser.new do |opts| + opts.banner = 'Usage: dashboard [options] file/id file/id ...' + opts.separator ' importing or exporting kibana dashboards' + opts.separator ' use "dashboard -e" to export kibana dashboards' + opts.separator ' use "dashboard -e -l" to export logging-kibana dashboards' + opts.separator ' use "dashboard -i" to import kibana dashboards' + opts.separator ' use "dashboard -e -h xx.xx.xx.xx -p xx" to assign kibana' + opts.separator '' + opts.separator 'options:' + + opts.on('-e', '--export', 'export kibana dashboards') do + opt_type = 'export' + end + + opts.on('-i', '--import', 'import kibana dashboards') do + opt_type = 'import' + end + + opts.on('-l', '--logging', 'sends requests to logging-kibana, the default value is kibana') do + opt_host = 'logging-kibana' + end + + opts.on('-h host', '--host host', 'set host') do |host| + opt_hash['host'] = host + end + + opts.on('-p port', '--port port', 'set port') do |port| + opt_hash['port'] = port + end +end + +options.parse!(ARGV) + +unless opt_type + puts(options) + exit +end + +parameters = ARGV + +hash = {} +hash['host'] = opt_host == 'kibana' ? KIBANA_HOST : LOGGING_KIBANA_HOST +hash['port'] = opt_host == 'kibana' ? KIBANA_PORT : LOGGING_KIBANA_PORT +hash['user'] = opt_host == 'kibana' ? ES_USER : LOGGING_ES_USER +hash['password'] = opt_host == 'kibana' ? ES_PASSWORD : LOGGING_ES_PASSWORD +hash.merge!(opt_hash) + +kibana_dashboard = KibanaDashboard.new(hash['host'], hash['port'], hash['user'], hash['password']) + +if opt_type == 'export' + kibana_dashboard.export(parameters) +elsif opt_type == 'import' + kibana_dashboard.import(parameters) +end -- 2.23.0

1 0