- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/3] providers/qemu: check qemu command exist
by Liu Yinsi 13 Jan '21

13 Jan '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 9d15f7d..8979da6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -102,7 +102,7 @@ set_qemu() for qemu in "${qemus[@]}" do - [ -n "$(command -v ${qemu})" ] && break + command -v "$qemu" > /dev/null && break done # debian has both qemu-system-x86_64 and qemu-system-riscv64 command -- 2.23.0

1 0

[PATCH compass-ci 1/3] providers/qemu: set qemu command
by Liu Yinsi 13 Jan '21

13 Jan '21

before: set qemu command by iterate qemus after: set qemu command by arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 39a1dc1..9d15f7d 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -96,10 +96,8 @@ set_netdev() set_qemu() { qemus=( - qemu-system-aarch64 + qemu-system-$(arch) qemu-kvm - qemu-system-x86_64 - qemu-system-riscv64 ) for qemu in "${qemus[@]}" -- 2.23.0

1 0

[PATCH compass-ci] provider/qemu/*.sh: add param to configure the RTC time of vm
by Yu Chuan 13 Jan '21

13 Jan '21

[Why] The RTC time of the virtual machine is not consistent with the time of the host physical machine. [Info] Before: [root@localhost ~]# timedatectl Local time: Wed 2021-01-13 15:55:01 CST Universal time: Wed 2021-01-13 07:55:01 UTC RTC time: Wed 2021-01-13 07:55:01 Time zone: Asia/Shanghai (CST, +0800) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a After: [root@localhost ~]# timedatectl Local time: Wed 2021-01-13 15:55:55 CST Universal time: Wed 2021-01-13 07:55:55 UTC RTC time: Wed 2021-01-13 15:55:55 Time zone: Asia/Shanghai (CST, +0800) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a Signed-off-by: Yu Chuan <13186087857(a)163.com> --- providers/qemu/kvm.sh | 1 + providers/qemu/pxe.sh | 1 + 2 files changed, 2 insertions(+) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 39a1dc19f112..064fd711c298 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -130,6 +130,7 @@ public_option() -initrd $initrd -smp $nr_cpu -m $memory + -rtc base=localtime -k en-us -no-reboot -nographic diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index 8cf198daa639..fa688331964a 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -28,6 +28,7 @@ kvm=( -cpu Kunpeng-920 -device virtio-gpu-pci -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd + -rtc base=localtime -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} -k en-us -no-reboot -- 2.23.0

1 0

[PATCH v2 compass-ci] tool/es-jobs: a new tool for summarizing jobs information
by Wu Zhende 13 Jan '21

13 Jan '21

Use es-jobs to summarizes information about a group of related jobs. [Example] es-jobs submit_id=xxx -f 'job_state,dmesg.timestamp:last' -r '-1' -s 'on_fail' means: 1. query the job whose submit_id is xxx 2. I want to get job_state,dmesg.timestamp:last 3. don't refine jobs 4. -s 'on_fail' to get data in stats which contain 'on_fail' [Output] { "job_state": [ "crystal.605716.failed" ], "dmesg.timestamp:last": [ 657.181408 ], "stats_filter_result": { "crystal.605716.openeuler_docker.\u001b[0mwordpress_build_on_fail": 1 }, "stats.count": { "stats.has_error": 1, "stats.has_error_jobs": [ "crystal.605716" ] } } Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/es_jobs.rb | 139 +++++++++++++++++++++++++++++++++++++++++++++++++ sbin/es-jobs | 58 +++++++++++++++++++++ 2 files changed, 197 insertions(+) create mode 100644 lib/es_jobs.rb create mode 100755 sbin/es-jobs diff --git a/lib/es_jobs.rb b/lib/es_jobs.rb new file mode 100644 index 0000000..a90b1ba --- /dev/null +++ b/lib/es_jobs.rb @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: GPL-2.0-only +# frozen_string_literal: true + +LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' + +require "#{LKP_SRC}/lib/stats" +require_relative './es_query' + +# deal jobs search from es +class ESJobs + def initialize(es_query, my_refine = [], fields = [], stats_filter = []) + @es_query = es_query + @es = ESQuery.new(ES_HOST, ES_PORT) + @refine = my_refine + @fields = fields + @stats_filter = stats_filter + @stats_filter_result = {} + @refine_jobs = [] + @jobs = {} + @stats_level = { + 0 => 'stats.success', + 1 => 'stats.unknown', + 2 => 'stats.warning', + 3 => 'stats.has_error' + } + set_defaults + deal_jobs + end + + def set_defaults + query_result = @es.multi_field_query(@es_query) + query_result['hits']['hits'].each do |job| + @jobs[job['_id']] = job['_source'] + end + + @stats = { + 'stats.count' => Hash.new(0), + 'stats.sum' => Hash.new(0), + 'stats.avg' => Hash.new(0) + } + @result = {} + @fields.each do |field| + @result[field] = [] + end + end + + def add_result_fields(job, level) + return unless @refine.include?(level) || @refine.include?(-1) + + @refine_jobs << job['id'] + @fields.each do |field| + value = job[field] + if value + value = job['id'] + '.' + value if field == 'job_state' + @result[field] << value + end + + next unless job['stats'] + + @result[field] << job['stats'][field] if job['stats'][field] + end + end + + def deal_jobs + stats_count = Hash.new(0) + stats_jobs = {} + + @jobs.each do |job_id, job| + level = deal_stats(job) + add_result_fields(job, level) + + stat_key = @stats_level[level] + stat_jobs_key = stat_key + '_jobs' + + stats_count[stat_key] += 1 + stats_jobs[stat_jobs_key] ||= [] + stats_jobs[stat_jobs_key] << job_id + end + + @stats['stats.count'].merge!(stats_count) + @stats['stats.count'].merge!(stats_jobs) + end + + def deal_stats(job, level = 0) + return 1 unless job['stats'] + + job['stats'].each do |key, value| + match_stats_filter(key, value, job['id']) + calculate_stat(key, value) + level = get_stat_level(key, level) + end + return level + end + + def match_stats_filter(key, value, job_id) + @stats_filter.each do |filter| + next unless key.include?(filter) + + key = job_id + '.' + key + @stats_filter_result[key] = value + + break + end + end + + def calculate_stat(key, value) + if function_stat?(key) + return unless @fields.include?('stats.sum') + + @stats['stats.sum'][key] += value + else + return unless @fields.include?('stats.avg') + + @stats['stats.avg'][key] = (@stats['stats.avg'][key] + value) / 2 + end + end + + def get_stat_level(stat, level) + return level if level >= 3 + return 3 if stat.match(/error|fail/i) + return 2 if stat.match(/warn/i) + + return 0 + end + + def output + result = { + 'stats.count' => @stats['stats.count'] + } + + @stats.each do |key, value| + result[key] = value if @fields.include?(key) + end + + @result['stats_filter_result'] = @stats_filter_result unless @stats_filter.empty? + @result.merge!(result) + puts JSON.pretty_generate(@result) + end +end diff --git a/sbin/es-jobs b/sbin/es-jobs new file mode 100755 index 0000000..25cbe9c --- /dev/null +++ b/sbin/es-jobs @@ -0,0 +1,58 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'optparse' +require 'json' + +require_relative '../lib/es_jobs' + +def parse_argv + items = {} + ARGV.each do |item| + key, value = item.split('=') + if key && value + value_list = value.split(',') + items[key] = value_list.length > 1 ? value_list : value + end + end + items +end +opt_refine = [-1] +opt_fields = [] +opt_stats_filter = [] +opt_parser = OptionParser.new do |opts| + opts.banner = 'Usage: es-jobs [options] search_key1=val1[,val2..] ..' + + opts.separator 'search_key can be id, suite, os, etc.' + opts.separator 'how to use -r' + opts.separator 'like es-jobs submit_id=xxx -r "-1,0,1,2,3"' + opts.separator '-1 means not refine' + opts.separator '{ + -1 => null, + 0 => stats.succes, + 1 => stats.unknow, + 2 => stats.warning, + 3 => stats.has_error +}' + + opts.on('-r fields', '--refine fields', 'refine jobs') do |fields| + opt_refine = fields.split(',').map(&:to_i) + end + + opts.on('-f fields', '--fields fields', 'fields you want to see') do |fields| + opt_fields = fields.split(',') + end + + opts.on('-s fields', '--stats-filter fields', 'return data contains fields') do |fields| + opt_stats_filter = fields.split(',') + end +end + +opt_parser.parse!(ARGV) +items = parse_argv +raise 'Please enter a query' if items.empty? + +es_jobs = ESJobs.new(items, opt_refine, opt_fields, opt_stats_filter) +es_jobs.output -- 2.23.0

2 1

[PATCH v8 compass-ci] providers/qemu: adapt for x86_64 and riscv64
by Liu Yinsi 13 Jan '21

13 Jan '21

before: support kvm for aarch64 after: support kvm for aarch64, x86_64, riscv64 test: /srv/result/host-info/2021-01-12/vm-2p8g/debian-sid-aarch64/z9.318111 /srv/result/host-info/2021-01-12/vm-2p8g/debian-sid-x86_64/z9.318142 /srv/result/host-info/2021-01-12/vm-2p8g/debian-sid-riscv64/z9.319551 [how] put public options like $kernel, $initrd, $smp together, set individual options like $machine, $bios, $nic according to different qemu command, then merge all options to run qemu. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 121 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 106 insertions(+), 15 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 577bca1..a61acab 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -18,12 +18,6 @@ check_logfile() } } -qemu_command() -{ - qemu=qemu-system-aarch64 - command -v $qemu >/dev/null || qemu=qemu-kvm -} - write_logfile() { ipxe_script=ipxe_script @@ -66,12 +60,57 @@ check_option_value() [ -n "$initrds" ] || exit } -get_initrd() +set_initrd() { initrd=initrd cat $initrds > $initrd } +set_bios() +{ + bios=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd + [ -f "$bios" ] || bios=/usr/share/ovmf/OVMF.fd +} + +set_helper() +{ + helper=/usr/libexec/qemu-bridge-helper + [ -f "$helper" ] || helper=/usr/lib/qemu/qemu-bridge-helper +} + +set_nic() +{ + nic="tap,model=virtio-net-pci,helper=$helper,br=br0,mac=${mac}" +} + +set_device() +{ + device="virtio-net-device,netdev=net0,mac=${mac}" +} + +set_netdev() +{ + netdev="bridge,br=br0,id=net0,helper=${helper}" +} + +set_qemu() +{ + qemus=( + qemu-system-aarch64 + qemu-kvm + qemu-system-x86_64 + qemu-system-riscv64 + ) + + for qemu in "${qemus[@]}" + do + [ -n "$(command -v ${qemu})" ] && break + done + + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 +} + print_message() { echo $SCHED_PORT @@ -83,36 +122,88 @@ print_message() sleep 5 } -run_qemu() +public_option() { kvm=( $qemu - -machine virt-4.0,accel=kvm,gic-version=3 -kernel $kernel -initrd $initrd -smp $nr_cpu -m $memory - -cpu Kunpeng-920 - -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd - -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} -k en-us -no-reboot -nographic -serial file:${log_file} -monitor null ) +} + +individual_option() +{ + case "$qemu" in + qemu-system-aarch64) + arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + ;; + qemu-kvm) + [ "$(arch)" == "aarch64" ] && arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + [ "$(arch)" == "x86_64" ] && arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-x86_64) + arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-riscv64) + arch_option=( + -machine virt + -device $device + -netdev $netdev + ) + ;; + esac +} - "${kvm[@]}" --append "${append}" +run_qemu() +{ + "${kvm[@]}" "${arch_option[@]}" --append "${append}" +} + +set_options() +{ + set_initrd + set_bios + set_helper + set_nic + set_device + set_netdev + set_qemu } check_logfile write_logfile -qemu_command parse_ipxe_script check_option_value -get_initrd + +set_options print_message +public_option +individual_option + run_qemu -- 2.23.0

2 2

[PATCH v2 lkp-tests] tests/*: add makepkg -s operation in archlinux
by Wang Yong 13 Jan '21

13 Jan '21

use makepkg -s to auto install depends in archlinux ref: https://git.archlinux.org/pacman.git/plain/scripts/makepkg.sh.in Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/build-pkg | 11 +++++++++-- tests/cci-makepkg | 14 +++++++++++--- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 31762384d..db4007b20 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -92,7 +92,9 @@ create_cgz_name() cgz_path="$PKG_MNT/${pack_to}/${pkgname}" cgz_name="${cgz_path}/${upstream_commit}.cgz" + pkg_args="-A --check --skippgpcheck" + [ "${os}" == "archlinux" ] && pkg_args="${pkg_args} -s --needed --noconfirm" [ -n "$git_tag" ] && { cgz_name="${cgz_path}/${git_tag}-${pkgrel}.cgz" @@ -153,8 +155,13 @@ build_source_pkg() create_cgz_name get_config - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + if [ "${os}" == "archlinux" ]; then + BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + else + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + fi local build_state="$?" diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 7d1e5cb15..e4bab41eb 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -133,10 +133,18 @@ update_softlink() build_source_pkg() { - local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" - echo "$makepkg" + if [ "${DISTRO}" == "archlinux" ]; then + local makepkg="$LKP_SRC/sbin/makepkg -A -s --needed --noconfirm --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" + echo "$makepkg" - PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + + else + local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" + echo "$makepkg" + + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + fi [ "$?" == 0 ] || exit 1 update_softlink -- 2.23.0

2 2

[PATCH v3 lkp-tests] tests/send_email: refactor send_email
by Wei Jihui 13 Jan '21

13 Jan '21

It is convenient for other tasks reuse to send email [usage] job.yaml test_task: on_fail: send_email: job_failed sleep: 1h This will send email to user when job is failed. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/email.sh | 26 ++++++++++++++++++++++++++ tests/email | 41 ----------------------------------------- tests/send_email | 20 ++++++++++++++++++++ 3 files changed, 46 insertions(+), 41 deletions(-) create mode 100755 lib/email.sh delete mode 100755 tests/email create mode 100755 tests/send_email diff --git a/lib/email.sh b/lib/email.sh new file mode 100755 index 00000000..d7c2c44a --- /dev/null +++ b/lib/email.sh @@ -0,0 +1,26 @@ +#!/bin/sh + +job_failed_content() +{ + email_content="To: $my_email +Subject: [NOTIFY Compass-CI] Test job failed: $id + +Dear $my_name: + + Sorry to inform you that your test job is failed, you can click the following link to view details. + http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT:-11300}$result_root + +Regards +Compass-CI +" +} + +send_email() +{ + local subject=$1 + local email_content + + ${subject}_content + + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$email_content" +} diff --git a/tests/email b/tests/email deleted file mode 100755 index a5f87d42..00000000 --- a/tests/email +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/sh -# - my_email -# - info - -: ${info:=failure} - -SCRIPT_DIR=$(dirname $(realpath $0)) -PROJECT_DIR=$(dirname $SCRIPT_DIR) - -. $PROJECT_DIR/lib/http.sh -. $PROJECT_DIR/lib/env.sh - -data_failure() -{ - data="To: $my_email -Subject: [NOTIFY Compass-ci] Test job failed - -Dear $my_username: - - Sorry to inform you that your test job is failed. you can click the following link to view details. - http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root - -Regards -Compass-Ci -" -} - -compose_email() -{ - [ "$info" == "failure" ] && data_failure -} - -run_email() -{ - [ -n "$my_email" ] || return 0 - my_username=$(echo "$my_email" | awk -F '@' '{print $1}') - compose_email - curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" -} - -run_email diff --git a/tests/send_email b/tests/send_email new file mode 100755 index 00000000..0132c3c8 --- /dev/null +++ b/tests/send_email @@ -0,0 +1,20 @@ +#!/bin/sh +# - subject +# my_email + +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/email.sh + +[ -n "$my_email" ] || { + echo "Can not get my_email, send email failed!" + exit 0 +} + +[ -n "$subject" ] || { + echo "Can not get email subject, send email failed!" + exit 0 +} + +send_email "$subject" -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] reset-tbox to reboot machine testbox
by Zhang Dewan 13 Jan '21

13 Jan '21

[why] sumbit a job to reboot machine testbox is better. [how] cp ~/lkp-test/jobs/reset-tbox-yaml . submit reset-tbox.yaml host_name="full-name-of-machine-testbox" Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- jobs/reset-tbox.yaml | 10 ++++++++++ tests/reset-tbox | 17 +++++++++++++++++ tests/reset-tbox.yaml | 12 ++++++++++++ 3 files changed, 39 insertions(+) create mode 100644 jobs/reset-tbox.yaml create mode 100755 tests/reset-tbox create mode 100644 tests/reset-tbox.yaml diff --git a/jobs/reset-tbox.yaml b/jobs/reset-tbox.yaml new file mode 100644 index 00000000..9c3f4f01 --- /dev/null +++ b/jobs/reset-tbox.yaml @@ -0,0 +1,10 @@ +suite: reset-tbox +testcase: reset-tbox + +testbox: dc-8g +os_mount: container +docker_image: 172.168.131.2:5001/reset-tbox +queue: reset + +host_name: +reset-tbox: diff --git a/tests/reset-tbox b/tests/reset-tbox new file mode 100755 index 00000000..b0f645d7 --- /dev/null +++ b/tests/reset-tbox @@ -0,0 +1,17 @@ +#!/bin/bash +# host_name + +source "/etc/lab/.account.info" +hosts_file=/etc/lab/hosts_file + +# require $host_name +host_ibmc_ip=$(grep -w $host_name $hosts_file | awk '{print $1}') + +if [[ "$host_ibmc_ip" =~ ^[0-9]+.* ]]; then + echo ">>> exec ipmitool commamd!" + ipmitool -I lanplus -H $host_ibmc_ip -U $iBMC_user -P $iBMC_passwd power reset + [ $? -eq 0 ] && echo "succeed!" +else + echo "specify host cannot found0!" + exit +fi diff --git a/tests/reset-tbox.yaml b/tests/reset-tbox.yaml new file mode 100644 index 00000000..f557085f --- /dev/null +++ b/tests/reset-tbox.yaml @@ -0,0 +1,12 @@ +short_description: > + To reboot machine testbox like taishan200-2280-2s64p-256g--aN etc. + +description: | + reset-tbox executed by a special docker which running on ibmc-mgnt, where you can access all server's iBMC-os by ipmitool, and then reboot the host. + +homepage: + +parameters: + host_name: see it on the first line about 'latbox aN' + +results: -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: install libvirt
by Xiao Shenwei 13 Jan '21

13 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- sparrow/0-package/common | 1 + sparrow/0-package/os/openEuler | 1 + 2 files changed, 2 insertions(+) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 4a9da60..2d7c9e3 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -51,3 +51,4 @@ sed '/^Environment=/a\TimeoutSec=0' /usr/lib/systemd/system/docker.service systemctl daemon-reload systemctl restart docker +systemctl start libvirtd diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index a63964f..37c74d0 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -21,6 +21,7 @@ pkgs=( busybox iproute hostname + libvirt ) yum install -y --skip-broken "${pkgs[@]}" -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code: set default values
by Liu Yinsi 13 Jan '21

13 Jan '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f2366c2..ca91da5 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -20,6 +20,8 @@ server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} : ${ASSIST_RESULT_PORT:=8102} : ${RESULT_WEBDAV_PORT:=3080} +: ${ES_HOST:=$server_ip} +: ${ES_PORT:=9200} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF @@ -45,6 +47,9 @@ LOCAL_ROBOT_EMAIL_ADDRESS: $LOCAL_ROBOT_EMAIL_ADDRESS MAILDIR: /srv/cci/Maildir/.compass-ci ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +lab: nolab +ES_HOST: $ES_HOST +ES_PORT: $ES_PORT EOF mkdir -p /etc/compass-ci/account -- 2.23.0

1 0

[PATCH lkp-tests] tests/send_email: refactor send_email
by Wei Jihui 13 Jan '21

13 Jan '21

It is convenient for other tasks reuse to send email [usage] job.yaml test_task: on_fail: send_email: job_failed sleep: 1h This will send email to user when job is failed. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/email.sh | 26 ++++++++++++++++++++++++++ tests/email | 41 ----------------------------------------- tests/send_email | 21 +++++++++++++++++++++ 3 files changed, 47 insertions(+), 41 deletions(-) create mode 100755 lib/email.sh delete mode 100755 tests/email create mode 100755 tests/send_email diff --git a/lib/email.sh b/lib/email.sh new file mode 100755 index 00000000..a407ad40 --- /dev/null +++ b/lib/email.sh @@ -0,0 +1,26 @@ +#!/bin/sh + +job_failed_content() +{ + email_content="To: $my_email +Subject: [NOTIFY Compass-CI] Test job failed: $id + +Dear $my_name: + + Sorry to inform you that your test job is failed. you can click the following link to view details. + http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT:-11300}$result_root + +Regards +Compass-CI +" +} + +send_email() +{ + local subject=$1 + local email_content + + ${subject}_content + + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$email_content" +} diff --git a/tests/email b/tests/email deleted file mode 100755 index a5f87d42..00000000 --- a/tests/email +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/sh -# - my_email -# - info - -: ${info:=failure} - -SCRIPT_DIR=$(dirname $(realpath $0)) -PROJECT_DIR=$(dirname $SCRIPT_DIR) - -. $PROJECT_DIR/lib/http.sh -. $PROJECT_DIR/lib/env.sh - -data_failure() -{ - data="To: $my_email -Subject: [NOTIFY Compass-ci] Test job failed - -Dear $my_username: - - Sorry to inform you that your test job is failed. you can click the following link to view details. - http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root - -Regards -Compass-Ci -" -} - -compose_email() -{ - [ "$info" == "failure" ] && data_failure -} - -run_email() -{ - [ -n "$my_email" ] || return 0 - my_username=$(echo "$my_email" | awk -F '@' '{print $1}') - compose_email - curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" -} - -run_email diff --git a/tests/send_email b/tests/send_email new file mode 100755 index 00000000..2911d3b8 --- /dev/null +++ b/tests/send_email @@ -0,0 +1,21 @@ +#!/bin/sh +# - subject +# my_email + +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/email.sh + +[ -n "$my_email" ] || { + echo "Can not get my_email, send email failed!" + exit 0 +} + +[ -n "$subject" ] || { + echo "Can not get email subject, send email failed!" + exit 0 +} + + +send_email $subject -- 2.23.0

3 2

[PATCH v2 lkp-tests 1/3] stats/crystal/memory--stat.cr: create memory--stat.cr with memory--stat
by Lu Kaiyi 13 Jan '21

13 Jan '21

create memory--stat.cr as same content with memory--stat Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/memory--stat.cr | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 stats/crystal/memory--stat.cr diff --git a/stats/crystal/memory--stat.cr b/stats/crystal/memory--stat.cr new file mode 100644 index 000000000..1a39e4075 --- /dev/null +++ b/stats/crystal/memory--stat.cr @@ -0,0 +1,22 @@ +#!/usr/bin/env ruby + +# time: 123456 +# cgroup: pmbench.1 +# cache 0 + +def parse + cg = 'nocg' + $stdin.each_line do |line| + case line + when /^time:/ + puts line + when /^cgroup: (\S+)/ + cg = $1 + when /(\S+)\s*(\S+)/ + k, v, = line.split + puts "#{cg}.#{k}: #{v}" + end + end +end + +parse -- 2.23.0

2 1

[PATCH v8 compass-ci 2/2] lib/es_query.rb: parse es_result to a Hash result
by Lu Kaiyi 13 Jan '21

13 Jan '21

[why] make es result more easy access. [example] input: es_result = [{"key"=>"build-pkg", "doc_count"=>354526, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>5708}, {"key"=>"finished", "doc_count"=>1033}, {"key"=>"incomplete", "doc_count"=>204}, {"key"=>"submit", "doc_count"=>136}, ... output: result_hash = {"build-pkg"=> {"failed"=>5708, "finished"=>1033, "incomplete"=>204, "submit"=>136, "OOM"=>11, "post_run"=>2}, "cci-depends"=> {"finished"=>1785, "failed"=>675, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 91aa6ab..928bd68 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -128,7 +128,7 @@ class ESQuery result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] return nil if result.empty? - result + parse_fields(result) end end @@ -216,3 +216,47 @@ def build_aggs_from_fields(fields) aggs_hash['aggs']["all_#{field}"].merge!(sub_aggs) if sub_aggs aggs_hash end + +# input: +# es_result = +# [{"key"=>"build-pkg", +# "doc_count"=>354526, +# "all_job_state"=> +# {"doc_count_error_upper_bound"=>0, +# "sum_other_doc_count"=>0, +# "buckets"=> +# [{"key"=>"failed", "doc_count"=>5708}, +# {"key"=>"finished", "doc_count"=>1033}, +# {"key"=>"incomplete", "doc_count"=>204}, +# {"key"=>"submit", "doc_count"=>136}, +# ... +# +# output: +# result_hash = +# {"build-pkg"=> +# {"failed"=>5708, +# "finished"=>1033, +# "incomplete"=>204, +# "submit"=>136, +# "OOM"=>11, +# "post_run"=>2}, +# "cci-depends"=> +# {"finished"=>1785, +# "failed"=>675, +# ... +def parse_fields(es_result) + result_hash = {} + es_result.each do |result| + key = result['key'] + sub_field = result.keys.detect { |field| field.start_with?('all_') } + + if sub_field + all_field = result[sub_field]['buckets'] + result_hash[key] = parse_fields(all_field) + else + result_hash[key] = result['doc_count'] + end + end + + result_hash +end -- 2.23.0

3 3

[PATCH compass-ci] lib: remove file executable permissions
by Lu Kaiyi 13 Jan '21

13 Jan '21

[why] the files in lib dir don't need executable permissions. [how] chmod a-x assign_account_client.rb chmod a-x build_my_info_client.rb chmod a-x redis_client.rb Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/assign_account_client.rb | 0 lib/build_my_info_client.rb | 0 lib/redis_client.rb | 0 3 files changed, 0 insertions(+), 0 deletions(-) mode change 100755 => 100644 lib/assign_account_client.rb mode change 100755 => 100644 lib/build_my_info_client.rb mode change 100755 => 100644 lib/redis_client.rb diff --git a/lib/assign_account_client.rb b/lib/assign_account_client.rb old mode 100755 new mode 100644 diff --git a/lib/build_my_info_client.rb b/lib/build_my_info_client.rb old mode 100755 new mode 100644 diff --git a/lib/redis_client.rb b/lib/redis_client.rb old mode 100755 new mode 100644 -- 2.23.0

1 0

[PATCH compass-ci] sbin/compare remove change of failure metric
by Lu Weitao 13 Jan '21

13 Jan '21

[Why] We only display the failure metrics that belong the latest job, aimed to express the latest job exists failure. so the change rate of failure metric is useless like: centos openeuler metric -------------------- ------------------------------ ------------------------------ fails:runs change fails:runs | | | 1:12 -8.3% 0:24 stderr./lkp/lkp/src/lib/env.sh:line#:/proc/sys/kernel/nmi_watchdog:Permission_denied [How] centos openeuler metric -------------------- ------------------------------ ------------------------------ fails:runs fails:runs | | 0:12 20:24 stderr.which:no_time_in(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/lkp/lkp/src/bin) centos openeuler metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 2.638681e+10 ± 23% -0.1% 2.636343e+10 ± 22% iperf.tcp.receiver.bps 2.638853e+10 ± 23% +0.1% 2.641691e+10 ± 22% iperf.tcp.sender.bps 6.80 ± 141% -50.4% 3.37 ± 223% iperf.time.elapsed_time Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 58 ++++++++--------------------------------- 1 file changed, 11 insertions(+), 47 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index fa578f6..5bb9d26 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -83,15 +83,12 @@ def get_values(value_list, success) end def get_compare_value(base_value_average, value_average, success) - # get compare value(change or reproduction) + # get compare value(change) # - if success - return if base_value_average.zero? + return unless success + return if base_value_average.zero? - (100 * value_average / base_value_average - 100).round(1) - else - (100 * (value_average - base_value_average)).round(1) - end + (100 * value_average / base_value_average - 100).round(1) end def field_changed?(base_values, values, success, field, options) @@ -264,7 +261,7 @@ def get_html_header(matrixes_titles, success) compare_title = CHANGE_STR else common_title = FAILS_RUNS_STR - compare_title = REPRODUCTION_STR + compare_title = '' end header = get_html_index(matrixes_titles) @@ -550,35 +547,6 @@ def format_fails_runs(fails, runs) fails_str + ':' + runs_str end -def get_reproduction_index_str(reproduction, compare_index) - reproduction_str = format('%+.1f%%', reproduction) - reproduction_index = compare_index - reproduction_str.index('.') - if reproduction_index.negative? - reproduction_str = format('%+.1e%%', reproduction).sub('e+0', 'e+').sub('e-0', 'e-') - reproduction_index = compare_index - reproduction_str.index('e') - 1 - end - return reproduction_index, reproduction_str -end - -def get_reprodcution(reproduction, compare_index) - if reproduction - reproduction_index, reproduction_str = get_reproduction_index_str(reproduction, compare_index) - space_str = ' ' * (SUB_SHORT_COLUMN_WIDTH - reproduction_str.length) - reproduction_str = space_str.insert(reproduction_index, reproduction_str) - else - reproduction_str = format("%-#{SUB_SHORT_COLUMN_WIDTH}s", '0') - end - reproduction_str -end - -def format_reproduction(reproduction, theme, compare_index) - color = get_compare_value_color(reproduction, theme) - colorize( - color, - get_reprodcution(reproduction, compare_index) - ) -end - def get_change_index_str(change, compare_index) change_str = format('%+.1f%%', change) change_index = compare_index - change_str.index('.') @@ -693,7 +661,7 @@ def get_title_name(success) compare_title = CHANGE_STR else common_title = FAILS_RUNS_STR - compare_title = REPRODUCTION_STR + compare_title = '' end return common_title, compare_title end @@ -718,14 +686,14 @@ def get_base_matrix_title_symbol(common_index, success) title_symbol end -def get_other_matrixes_title_symbol(_compare_title, matrixes_number, common_index, success) +def get_other_matrixes_title_symbol(compare_title, matrixes_number, common_index, success) title_symbol = ' ' * ( (INTERVAL_WIDTH + COLUMN_WIDTH) * matrixes_number ) start_point = 0 common_symbol = success ? '\\' : '|' - compare_symbol = '|' + compare_symbol = compare_title.empty? ? '' : '|' compare_index = SUB_SHORT_COLUMN_WIDTH / 2 (matrixes_number - 1).times do |_| @@ -786,12 +754,8 @@ def get_success_str(values, index, theme, compare_index) (change_str || '') + stddev_str end -def get_failure_str(values, index, theme, compare_index) - unless index.zero? - reproduction_str = format_reproduction( - values[:reproduction], theme, compare_index - ) - end +def get_failure_str(values, index) + reproduction_str = format("%-#{SUB_SHORT_COLUMN_WIDTH}s", '') unless index.zero? fails_runs_str = format_fails_runs( values[:fails], @@ -810,7 +774,7 @@ def get_values_str(matrixes, success, theme) ) + INTERVAL_BLANK else get_failure_str( - values, index, theme, compare_index + values, index ) + INTERVAL_BLANK end end -- 2.23.0

1 0

[PATCH v8 compass-ci 1/2] lib/es_query.rb: get multi-field aggregation count result
by Lu Kaiyi 13 Jan '21

13 Jan '21

[why] get multi-field aggregation count for overview of report page. [input]: fields, query_items (optional for query_items, default no scope limitation) [example] input: fields = ['suite'] query_items = { 'os' => 'openeuler' } output: [{"key"=>"build-pkg", "doc_count"=>186214}, {"key"=>"cci-depends", "doc_count"=>1230}, {"key"=>"iperf", "doc_count"=>1103}, {"key"=>"spinlock", "doc_count"=>1000}, {"key"=>"cci-makepkg", "doc_count"=>949}, {"key"=>"iperf-walk-os-test", "doc_count"=>474}, {"key"=>"bisect", "doc_count"=>381}, ... input: fields = ['suite', 'job_state'] query_items = { 'os' => 'openeuler' } output: [{"key"=>"build-pkg", "doc_count"=>186227, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>3830}, {"key"=>"finished", "doc_count"=>803}, {"key"=>"incomplete", "doc_count"=>196}, {"key"=>"submit", "doc_count"=>136}, ... input: fields = ['os', 'os_version', 'job_state'] query_items = {} output: [{"key"=>"openeuler", "doc_count"=>193287, "all_os_version"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"20.03", "doc_count"=>191789, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>3909}, {"key"=>"finished", "doc_count"=>2401}, {"key"=>"incomplete", "doc_count"=>300}, {"key"=>"submit", "doc_count"=>136}, {"key"=>"OOM", "doc_count"=>23}, {"key"=>"post_run", "doc_count"=>1}]}}, {"key"=>"20.03-SP1", "doc_count"=>904, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>641}, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 70 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index 88a420f..91aa6ab 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -87,6 +87,49 @@ class ESQuery result end + + # input: + # fields, query_items + # + # fields => fields: Array(keyword) + # eg: + # ['suite', 'job_state'] + # + # query_items => query_items: Hash(keyword, value) + # eg: + # { + # 'os' => 'openeuler', + # ... + # } + # (optional for query_items, default no scope limitation) + # output: + # [{"key"=>"build-pkg", + # "doc_count"=>186175, + # "all_job_state"=> + # {"doc_count_error_upper_bound"=>0, + # "sum_other_doc_count"=>0, + # "buckets"=> + # [{"key"=>"failed", "doc_count"=>3830}, + # {"key"=>"finished", "doc_count"=>803}, + # {"key"=>"incomplete", "doc_count"=>196}, + # ... + def query_fields(fields, query_items = {}) + field1 = fields.first + aggs_hash = build_aggs_from_fields(fields) + query = { + query: { + bool: { + must: build_multi_field_subquery_body(query_items) + } + }, + aggs: aggs_hash['aggs'], + size: 0 + } + result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] + return nil if result.empty? + + result + end end # Range Query Example: @@ -146,3 +189,30 @@ def assign_desc_body(keyword) }] } end + +# input: +# fields = ['os', 'os_version', 'job_state'] +# output: +# aggs_hash = { +# "aggs"=> +# {"all_os"=> +# {"terms"=>{:field=>"os", :size=>1000}, +# "aggs"=> +# {"all_os_version"=> +# {"terms"=>{:field=>"os_version", :size=>1000}, +# "aggs"=> +# {"all_job_state"=>{"terms"=>{:field=>"job_state", :size=>1000}}}}}}} +# } +def build_aggs_from_fields(fields) + aggs_hash = {} + return if fields.empty? + + field = fields.shift + aggs_hash['aggs'] ||= {} + aggs_hash['aggs']["all_#{field}"] = { + 'terms' => { field: field, size: 1000 } + } + sub_aggs = build_aggs_from_fields(fields) + aggs_hash['aggs']["all_#{field}"].merge!(sub_aggs) if sub_aggs + aggs_hash +end -- 2.23.0

1 0

[PATCH v2 lkp-tests 3/3] stats/crystal/memory--stat.cr: fix some errors manually
by Lu Kaiyi 13 Jan '21

13 Jan '21

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/memory--stat.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/stats/crystal/memory--stat.cr b/stats/crystal/memory--stat.cr index da7ac0d86..f45ab5fb1 100644 --- a/stats/crystal/memory--stat.cr +++ b/stats/crystal/memory--stat.cr @@ -6,14 +6,14 @@ def parse cg = "nocg" - $stdin.each_line do |line| + STDIN.each_line do |line| case line when /^time:/ puts line when /^cgroup: (\S+)/ cg = $1 when /(\S+)\s*(\S+)/ - k, v, = line.split + k, v = line.split puts "#{cg}.#{k}: #{v}" end end -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/3] stats/crystal/memory--stat.cr: do first conversion by ruby2crystal tools
by Lu Kaiyi 13 Jan '21

13 Jan '21

use the ruby2crystal tools do first conversion like below: /c/crystal/ruby_crystal_codemod/exe/ruby_crystal_codemod memory--stat.cr /c/lkp-tests/tools/ruby2crystal.sh memory--stat.cr Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/memory--stat.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/stats/crystal/memory--stat.cr b/stats/crystal/memory--stat.cr index 1a39e4075..da7ac0d86 100644 --- a/stats/crystal/memory--stat.cr +++ b/stats/crystal/memory--stat.cr @@ -1,11 +1,11 @@ -#!/usr/bin/env ruby +#!/usr/bin/env crystal # time: 123456 # cgroup: pmbench.1 # cache 0 def parse - cg = 'nocg' + cg = "nocg" $stdin.each_line do |line| case line when /^time:/ -- 2.23.0

1 0

[PATCH compass-ci] doc/send-mail.md: format markdown file
by Wei Jihui 13 Jan '21

13 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- doc/send-mail.md | 51 +++++++++++++++++++++++------------------------- 1 file changed, 24 insertions(+), 27 deletions(-) diff --git a/doc/send-mail.md b/doc/send-mail.md index f503a08..5af8d17 100644 --- a/doc/send-mail.md +++ b/doc/send-mail.md @@ -1,13 +1,5 @@ - # send mail - - - - - - - ## purpose The service is used to send mails with formatted mail data @@ -23,38 +15,42 @@ The service is used to send mails with formatted mail data ## data format - send_mail_yaml - - data format - data='{ - "subject": "email subject", - "to": "email_to_addr", - "body": "email message" - }' + data='{ + "subject": "email subject", + "to": "email_to_addr", + "body": "email message" + }' or - data=" - subject: email subject - to: email_to_addr - body: email message - " + data=" + subject: email subject + to: email_to_addr + body: email message + " - send_mail_text - - data format - data=" - To: email_to_addr - Subject: email_subject + data=" + To: email_to_addr + Subject: email_subject - mail_msg_line1 - mail_msg_line2 - ... - " + mail_msg_line1 + mail_msg_line2 + ... + " ## usage: - send_mail_yaml + ```shell curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_yaml' -d "$data" + ``` + - send_mail_text + ```shell curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "$data" + ``` ## example + ```ruby data = " To: test_email(a)163.com Subject: test mail 10 @@ -63,3 +59,4 @@ The service is used to send mails with formatted mail data " %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") + ``` -- 2.23.0

1 0

[PATCH v2 lkp-tests] sbin/submit: modify job_id to job id
by Hu Xuejiao 13 Jan '21

13 Jan '21

[why] We use "es-find (id=xxx)" to search id-information,but when submit, it is (job_id=xxx). so it is recommended that the value be the same as the id=xxx. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/submit | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sbin/submit b/sbin/submit index 5273896b..39cdcf57 100755 --- a/sbin/submit +++ b/sbin/submit @@ -168,10 +168,10 @@ ARGV.each do |jobfile| JSON.parse(messages).each do |msg| if msg['message'].empty? job_ids << msg['job_id'].to_s - puts("submit #{jobfile}, got job_id=#{msg['job_id']}") + puts("submit #{jobfile}, got job id=#{msg['job_id']}") else opt_monitor = false - puts("submit #{jobfile} failed, got job_id=#{msg['job_id']}, error: #{msg['message']}") + puts("submit #{jobfile} failed, got job id=#{msg['job_id']}, error: #{msg['message']}") end end end -- 2.23.0

2 1

[PATCH compass-ci] doc: add a doc about catalogs under /srv/git
by Li Yuanchao 13 Jan '21

13 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- doc/about_srv_git.md | 182 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 doc/about_srv_git.md diff --git a/doc/about_srv_git.md b/doc/about_srv_git.md new file mode 100644 index 0000000..3374828 --- /dev/null +++ b/doc/about_srv_git.md @@ -0,0 +1,182 @@ + +# ��`/srv/git`�� + +``` +delimiter: + ��git-bisect,��good commit�� + +auto-submit�� + ��job��rabbitmq��git-mirror��job�� + +git-daemon�� + ��git��/srv/git��git clone/git push�� + �� "git clone git://localhost/$project" ��git-daemon�� + +git-mirror�� + ��clone�� + +remote-git�� + ��/srv/git��curl��/srv/git��git�� +``` + + +# /srv/git�� + + +��a-z��git-mirror�� +``` +drwxrwsr-x. 55 lkp committer 4.0K 2020-11-20 03:54 u +drwxrwsr-x. 78 lkp committer 4.0K 2020-12-04 11:24 v +drwxrwsr-x. 75 lkp committer 4.0K 2020-12-04 11:25 i +drwxrwsr-x. 986 lkp committer 36K 2020-12-04 11:37 d +drwxrwsr-x. 1612 lkp committer 60K 2020-12-05 03:50 g +drwxrwsr-x. 827 lkp committer 28K 2020-12-05 04:00 b +drwxrwsr-x. 66 lkp committer 4.0K 2020-12-05 04:05 w +drwxrwsr-x. 32 lkp committer 4.0K 2020-12-05 04:17 y +drwxrwsr-x. 706 lkp committer 28K 2020-12-05 05:49 e +drwxrwsr-x. 837 lkp committer 28K 2020-12-08 22:05 f +drwxrwsr-x. 679 lkp committer 24K 2020-12-10 18:18 k +drwxrwsr-x. 948 lkp committer 36K 2020-12-11 02:31 h +drwxrwsr-x. 384 lkp committer 12K 2020-12-11 03:12 j +drwxrwsr-x. 927 lkp committer 36K 2020-12-15 03:11 n +drwxrwsr-x. 1757 lkp committer 56K 2020-12-15 11:09 l +drwxrwsr-x. 54 lkp committer 4.0K 2020-12-15 14:29 t +drwxrwsr-x. 1018 lkp committer 36K 2020-12-15 15:57 a +drwxrwsr-x. 1344 lkp committer 40K 2020-12-15 22:28 m +drwxrwsr-x. 809 lkp committer 28K 2020-12-16 05:21 o +drwxrwsr-x. 1342 lkp committer 36K 2020-12-21 19:19 c +drwxrwsr-x. 5800 lkp committer 212K 2020-12-22 03:08 p +drwxrwsr-x. 395 lkp committer 12K 2020-12-22 22:16 q +drwxrwsr-x. 1632 lkp committer 64K 2020-12-28 21:25 r +drwxrwsr-x. 445 lkp committer 20K 2020-12-28 23:16 x +drwxrwsr-x. 1796 lkp committer 56K 2021-01-05 09:20 s +drwxrwsr-x. 39 lkp committer 4.0K 2021-01-08 10:48 z +``` + +��lkp:committer��git-mirror�� +``` +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-16 10:57 git.code.sf.net +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-16 11:22 code.reversed.top +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-20 03:56 source.ffmpeg.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-21 02:46 git.kernel.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-22 04:05 gitlab.univ-nantes.fr +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-02 18:14 git.musl-libc.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-05 05:56 gitlab.dkrz.de +drwxrwsr-x. 8 lkp committer 4.0K 2020-12-05 06:17 git.sv.gnu.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-05 06:17 git.ageinghacker.net +drwxrwsr-x. 11 lkp committer 4.0K 2020-12-05 06:39 git.qemu-project.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-05 06:44 git.sigxcpu.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-05 06:56 git.osmocom.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-05 07:38 gitlab.linphone.org +drwxrwsr-x. 8 lkp committer 4.0K 2020-12-05 07:52 gitlab.gnome.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-10 14:40 webrtc.googlesource.com +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-11 03:33 www.kismetwireless.net +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-11 03:34 src.fedoraproject.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-16 05:53 git.tuxfamily.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 17:14 chriso(a)github.com +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 17:55 nedko.arnaudov.name +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 17:59 git.gnome.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 18:01 android.googlesource.com +drwxrwsr-x. 5 lkp committer 4.0K 2020-12-17 18:40 gopkg.in +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 19:53 git.cryptomilk.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 20:05 git.dpkg.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-17 20:09 kernel.googlesource.com +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-19 14:15 aur(a)archlinux.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-21 05:57 boringssl.googlesource.com +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 04:44 hub.spigotmc.org +drwxrwsr-x. 14 lkp committer 4.0K 2020-12-22 04:49 repo.or.cz +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 04:52 luajit.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 05:02 git.assembla.com +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-22 05:07 git.purrdata.net +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 05:09 brettc(a)github.com +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 05:11 git.teknik.io +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-22 14:40 code.googlesource.com +drwxrwsr-x. 7 lkp committer 4.0K 2020-12-23 01:50 git.qemu.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-23 01:51 git.merproject.org +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-23 01:52 code.qt.io +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-23 01:57 git.tartarus.org +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-27 22:17 gist.github.com +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-29 00:28 invent.kde.org +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-29 00:55 git.sv.nongnu.org +drwxrwsr-x. 6 lkp committer 4.0K 2020-12-29 02:57 git.savannah.gnu.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-29 02:57 git.xfce.org +drwxrwsr-x. 7 lkp committer 4.0K 2020-12-29 02:58 anongit.freedesktop.org +drwxrwsr-x. 3 lkp committer 4.0K 2020-12-29 02:59 git.ejiek.com +drwxrwsr-x. 32 lkp committer 4.0K 2020-12-29 03:00 gitlab.com +drwxrwsr-x. 5 lkp committer 4.0K 2020-12-30 22:00 git.torproject.org +drwxrwsr-x. 3 lkp committer 4.0K 2021-01-02 05:58 skia.googlesource.com +drwxrwsr-x. 7 lkp committer 4.0K 2021-01-03 05:57 aur(a)aur4.archlinux.org +drwxrwsr-x. 7 lkp committer 4.0K 2021-01-05 10:56 gitlab.freedesktop.org +drwxrwsr-x. 3 lkp committer 4.0K 2021-01-05 10:59 gitlab.xiph.org +drwxrwsr-x. 3 lkp committer 4.0K 2021-01-05 10:59 git.savannah.nongnu.org +drwxrwsr-x. 4 lkp committer 4.0K 2021-01-05 11:01 git.tukaani.org +drwxrwsr-x. 13 lkp committer 4.0K 2021-01-05 11:04 aur.archlinux.org +drwxrwsr-x. 6 lkp committer 4.0K 2021-01-05 11:07 bitbucket.org +drwxrwsr-x. 1664 lkp committer 40K 2021-01-07 09:08 github.com +drwxrwsr-x. 5 lkp committer 4.0K 2021-01-08 05:03 go.googlesource.com +drwxrwsr-x. 7 lkp committer 4.0K 2021-01-08 05:24 chromium.googlesource.com +drwxrwsr-x. 14 lkp committer 4.0K 2021-01-08 06:00 aur(a)aur.archlinux.org +``` + +��git-mirror�� +``` +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:30 coccinelle +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:45 oec-hardware +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:45 libbpf +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:46 qemu +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:51 smatch +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-17 11:51 gtk +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-21 11:02 pixz +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-31 11:26 sparse +drwxrwsr-x. 3 lkp committer 4.0K 2020-08-31 11:27 pahole +drwxrwsr-x. 3 lkp committer 4.0K 2020-09-22 15:10 upstream-repos +drwxrwsr-x. 3 lkp committer 4.0K 2020-11-09 19:31 linux +drwxrwsr-x. 24 lkp committer 4.0K 2020-11-10 16:40 submodules +drwxrwsr-x. 4 lkp committer 4.0K 2020-12-02 10:00 compass-ci +``` + +��pkgbuild��make�� +``` +drwxrwsr-x. 43 lkp team 200K 2020-11-11 15:17 archlinux +drwxrwsr-x. 82 root team 4.0K 2020-12-15 18:12 pkg +drwxrwsr-x. 13 wfg team 4.0K 2020-12-16 12:55 make +``` + +��rpm��spec�� +``` +drwxrwsr-x. 3 root committer 4.0K 2020-12-08 11:15 rpmbuild-pkg +``` + +��clone�� +``` +drwxrwsr-x. 3 zhy committer 4.0K 2020-06-16 11:25 mysql-server +drwxrwsr-x. 3 zhy committer 4.0K 2020-07-10 14:28 atomic +drwxrwsr-x. 3 zhy committer 4.0K 2020-07-10 14:28 spinlock +drwxrwsr-x. 3 zhy committer 4.0K 2020-07-10 14:29 mutex +drwxrwsr-x. 4 zhy committer 4.0K 2020-08-24 11:52 avx2neon +drwxrwsr-x. 5 liping committer 4.0K 2020-08-24 11:52 AvxToNeon +``` + +��compass-ci/lkp-tests/.../upstream-repos�� +``` +drwxrwsr-x. 3 wfg committer 4.0K 2020-09-14 15:26 gitee.com +``` + +�� +``` +drwxrwxrwx. 3 weijihui weijihui 4.0K 2020-07-27 15:26 intrintesttool +``` + +ansible�� +``` +drwxrwsr-x. 17 root committer 4.0K 2020-12-29 10:33 ansible-examples +``` + +�� +``` +drwxrwsr-x. 2 root committer 16K 2020-03-27 10:45 lost+found +drwxrwsr-x. 2 wfg committer 4.0K 2020-07-24 16:58 gzip +drwxrwsr-x. 5 wfg committer 4.0K 2020-08-01 11:15 cci +drwxrwsr-x. 5 wfg committer 4.0K 2020-08-04 16:27 lkp-tests +drwxrwsr-x. 4 wfg committer 4.0K 2020-09-19 14:15 crystal-ci +``` -- 2.23.0

2 1

[PATCH compass-ci] tool/es-jobs: add a new tool
by Wu Zhende 12 Jan '21

12 Jan '21

Use es-jobs to summarizes information about a group of related jobs. [Example] es-jobs submit_id=xxx -f 'id,job_state,dmesg.timestamp:last,stats.sum,stats.avg' -r '0' means: 1.query the job whose submit_id is xxx 2.I want to get id,job_state,dmesg.timestamp:last,stats.sum,stats.avg 3.refine stats.success jobs [Output] { "id": [ "crystal.597090", "crystal.597093" ], "job_state": [ "finished", "finished" ], "dmesg.timestamp:last": [ 75.256233, 48.990905 ], "stats.sum": { "kmsg.timestamp:last": 124.24713799999999, "dmesg.timestamp:last": 124.24713799999999 }, "stats.avg": { "iperf.tcp.receiver.bps": 17008962077.755077, "iperf.tcp.sender.bps": 17036843967.505566, "iperf.udp.bps": 524286.6541701214 }, "stats.count": { "stats.success": 2, "stats.success_jobs": [ "crystal.597090", "crystal.597093" ] } } Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/es_jobs.rb | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++ sbin/es-jobs | 53 +++++++++++++++++++++++++++ 2 files changed, 152 insertions(+) create mode 100644 lib/es_jobs.rb create mode 100755 sbin/es-jobs diff --git a/lib/es_jobs.rb b/lib/es_jobs.rb new file mode 100644 index 0000000..78db8f0 --- /dev/null +++ b/lib/es_jobs.rb @@ -0,0 +1,99 @@ +# SPDX-License-Identifier: GPL-2.0-only +# frozen_string_literal: true + +LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' + +require "#{LKP_SRC}/lib/stats" +require_relative './es_query' + +# deal jobs search from es +class ESJobs + def initialize(es_query, my_refine = [-1], fields = []) + @es = ESQuery.new(ES_HOST, ES_PORT) + @refine = my_refine + @fields = fields + @refine_jobs = [] + @query_result = @es.multi_field_query(es_query) + @jobs = {} + @stats_count = Hash.new(0) + @stats_sum = Hash.new(0) + @stats_avg = Hash.new(0) + @stats_level = { + 0 => 'stats.success', + 1 => 'stats.unknow', + 2 => 'stats.warning', + 3 => 'stats.has_error' + } + set_defaults + deal_jobs + end + + def set_defaults + @query_result['hits']['hits'].each do |job| + @jobs[job['_id']] = job['_source'] + end + + @result = {} + @fields.each do |field| + @result[field] = [] + end + end + + def add_result_fields(job, level) + return unless @refine.include?(level) || @refine.include?(-1) + + @refine_jobs << job['id'] + @fields.each do |field| + @result[field] << job[field] if job[field] + + next unless job['stats'] + + @result[field] << job['stats'][field] if job['stats'][field] + end + end + + def deal_jobs + @jobs.each do |job_id, job| + level = deal_stats(job) + add_result_fields(job, level) + + @stats_count[@stats_level[level]] += 1 + stat_key = @stats_level[level] + '_jobs' + @stats_count[stat_key] = [] unless @stats_count.key?(stat_key) + @stats_count[stat_key] << job_id + end + end + + def deal_stats(job, level = 0) + return 1 unless job['stats'] + + job['stats'].each do |key, value| + if function_stat?(key) + @stats_sum[key] += value + else + @stats_avg[key] = (@stats_avg[key] + value) / 2 + end + + level = get_stat_level(key, level) + end + return level + end + + def get_stat_level(stat, level) + return level if level >= 3 + return 3 if stat.match(/error|fail/i) + return 2 if stat.match(/warn/i) + + return 0 + end + + def output + result = { + 'stats.count' => @stats_count + } + result['stats.sum'] = @stats_sum if @fields.include?('stats.sum') + result['stats.avg'] = @stats_avg if @fields.include?('stats.avg') + @result.merge!(result) + puts JSON.pretty_generate(@result) + end +end diff --git a/sbin/es-jobs b/sbin/es-jobs new file mode 100755 index 0000000..f075ef2 --- /dev/null +++ b/sbin/es-jobs @@ -0,0 +1,53 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'optparse' +require 'json' + +require_relative '../lib/es_jobs' + +def parse_argv + items = {} + ARGV.each do |item| + key, value = item.split('=') + if key && value + value_list = value.split(',') + items[key] = value_list.length > 1 ? value_list : value + end + end + items +end +opt_refine = [-1] +opt_fields = [] +opt_parser = OptionParser.new do |opts| + opts.banner = 'Usage: es-jobs [options] search_key1=val1[,val2..] ..' + + opts.separator 'search_key can be id, suite, os, etc.' + opts.separator 'How to use -r' + opts.separator 'Like es-jobs submit_id=xxx -r "-1,0,1,2,3"' + opts.separator '-1 means not refine' + opts.separator '{ + -1 => null, + 0 => stats.succes, + 1 => stats.unknow, + 2 => stats.warning, + 3 => stats.has_error +}' + + opts.on('-r fields', '--refine fields', 'refine jobs') do |fields| + opt_refine = fields.split(',').map(&:to_i) + end + + opts.on('-f fields', '--fields fields', 'fields you want to see') do |fields| + opt_fields = fields.split(',') + end +end + +opt_parser.parse!(ARGV) +items = parse_argv +raise 'Please enter a query' if items.empty? + +es_jobs = ESJobs.new(items, opt_refine, opt_fields) +es_jobs.output -- 2.23.0

3 2

[PATCH v2 lkp-tests] tests/send_email: refactor send_email
by Wei Jihui 12 Jan '21

12 Jan '21

It is convenient for other tasks reuse to send email [usage] job.yaml test_task: on_fail: send_email: job_failed sleep: 1h This will send email to user when job is failed. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/email.sh | 26 ++++++++++++++++++++++++++ tests/email | 41 ----------------------------------------- tests/send_email | 21 +++++++++++++++++++++ 3 files changed, 47 insertions(+), 41 deletions(-) create mode 100755 lib/email.sh delete mode 100755 tests/email create mode 100755 tests/send_email diff --git a/lib/email.sh b/lib/email.sh new file mode 100755 index 00000000..d7c2c44a --- /dev/null +++ b/lib/email.sh @@ -0,0 +1,26 @@ +#!/bin/sh + +job_failed_content() +{ + email_content="To: $my_email +Subject: [NOTIFY Compass-CI] Test job failed: $id + +Dear $my_name: + + Sorry to inform you that your test job is failed, you can click the following link to view details. + http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT:-11300}$result_root + +Regards +Compass-CI +" +} + +send_email() +{ + local subject=$1 + local email_content + + ${subject}_content + + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$email_content" +} diff --git a/tests/email b/tests/email deleted file mode 100755 index a5f87d42..00000000 --- a/tests/email +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/sh -# - my_email -# - info - -: ${info:=failure} - -SCRIPT_DIR=$(dirname $(realpath $0)) -PROJECT_DIR=$(dirname $SCRIPT_DIR) - -. $PROJECT_DIR/lib/http.sh -. $PROJECT_DIR/lib/env.sh - -data_failure() -{ - data="To: $my_email -Subject: [NOTIFY Compass-ci] Test job failed - -Dear $my_username: - - Sorry to inform you that your test job is failed. you can click the following link to view details. - http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root - -Regards -Compass-Ci -" -} - -compose_email() -{ - [ "$info" == "failure" ] && data_failure -} - -run_email() -{ - [ -n "$my_email" ] || return 0 - my_username=$(echo "$my_email" | awk -F '@' '{print $1}') - compose_email - curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" -} - -run_email diff --git a/tests/send_email b/tests/send_email new file mode 100755 index 00000000..2911d3b8 --- /dev/null +++ b/tests/send_email @@ -0,0 +1,21 @@ +#!/bin/sh +# - subject +# my_email + +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/email.sh + +[ -n "$my_email" ] || { + echo "Can not get my_email, send email failed!" + exit 0 +} + +[ -n "$subject" ] || { + echo "Can not get email subject, send email failed!" + exit 0 +} + + +send_email $subject -- 2.23.0

1 0

[PATCH v2 compass-ci] lib/job.cr: set the upload path of package based on os attribute of top-level package style
by Cao Xueliang 12 Jan '21

12 Jan '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 218e2e1..6642703 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -249,11 +249,30 @@ class Job set_upload_dirs() end - def get_package_dir - if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? - mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup - common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + def get_pkg_common_dir + pkg_style = nil + ["cci-makepkg", "cci-depends", "build-pkg"].each do |item| + pkg_style = @hash[item]? + break if pkg_style end + return nil unless pkg_style && pkg_style != nil + + tmp_os = pkg_style["os"]? == nil ? "#{os}" : pkg_style["os"] + tmp_os_arch = pkg_style["os_arch"]? == nil ? "#{os_arch}" : pkg_style["os_arch"] + tmp_os_version = pkg_style["os_version"]? == nil ? "#{os_version}" : pkg_style["os_version"] + + tmp_os_mount = pkg_style["os_mount"]? == nil ? "#{os_mount}" : pkg_style["os_mount"] + mount_type = tmp_os_mount == "cifs" ? "nfs" : tmp_os_mount.dup + + common_dir = "#{mount_type}/#{tmp_os}/#{tmp_os_arch}/#{tmp_os_version}" + + return common_dir + end + + def get_package_dir + package_dir = "" + common_dir = get_pkg_common_dir + return package_dir unless common_dir if @hash["cci-makepkg"]? package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" @@ -265,10 +284,9 @@ class Job else package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" package_dir += ",/cci/build-config" if @hash["config"]? - else - package_dir = "" end return package_dir -- 2.23.0

1 0

[PATCH lkp-tests] tests/*: add makepkg -s operation in archlinux
by Wang Yong 12 Jan '21

12 Jan '21

use makepkg -s to auto install depends in archlinux ref: https://git.archlinux.org/pacman.git/plain/scripts/makepkg.sh.in Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/build-pkg | 16 +++++++++++++--- tests/cci-makepkg | 14 +++++++++++--- 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 31762384d..5a473a00c 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -92,7 +92,12 @@ create_cgz_name() cgz_path="$PKG_MNT/${pack_to}/${pkgname}" cgz_name="${cgz_path}/${upstream_commit}.cgz" - pkg_args="-A --check --skippgpcheck" + + if [ "${os}" == "archlinux" ]; then + pkg_args="-A --check --skippgpcheck -s --needed --noconfirm" + else + pkg_args="-A --check --skippgpcheck" + fi [ -n "$git_tag" ] && { cgz_name="${cgz_path}/${git_tag}-${pkgrel}.cgz" @@ -153,8 +158,13 @@ build_source_pkg() create_cgz_name get_config - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + if [ "${os}" == "archlinux" ]; then + BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + else + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf + fi local build_state="$?" diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 7d1e5cb15..e4bab41eb 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -133,10 +133,18 @@ update_softlink() build_source_pkg() { - local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" - echo "$makepkg" + if [ "${DISTRO}" == "archlinux" ]; then + local makepkg="$LKP_SRC/sbin/makepkg -A -s --needed --noconfirm --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" + echo "$makepkg" - PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + + else + local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" + echo "$makepkg" + + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + fi [ "$?" == 0 ] || exit 1 update_softlink -- 2.23.0

1 0