- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 2/2] assign-account: add 'lab' to default yaml
by Luan Shengde 08 Jan '21

08 Jan '21

the default yaml file will contain: my_email my_name lab lab is used to read the lab yaml according to its value. example: if the value of lab is 'z9', the sched will read token from file: z9.yaml Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index af08d91..7e0a5a7 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -173,6 +173,7 @@ class AccountStorage # they are added along with 'my_ssh_pubkey' when sending assign account request account_yaml['my_email'] = @data['my_email'] account_yaml['my_name'] = @data['my_name'] + account_yaml['lab'] = @data['lab'] File.open(default_yaml, 'w') do |f| f.puts account_yaml.to_yaml -- 2.23.0

2 1

[PATCH lkp-tests 1/2] lib/job.rb: add fault_reproduction
by Wei Jihui 08 Jan '21

08 Jan '21

do fault reproduction when submit a failed job Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index 8346db87..631ebfc9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -255,6 +255,7 @@ class Job hash.delete('override') end hash.merge!(@overrides) + hash.merge!(fault_reproduction) if hash.has_key?("id") @jobs.concat(multi_args(hash)) # return [hash] or [h1,h2] end rescue StandardError => e @@ -285,6 +286,10 @@ class Job @jobfile = jobfile end + def fault_reproduction + wait = load_yaml("#{lkp_src}/jobs/fault_reproduction.yaml", {}) + end + def multi_args(hash) jobs_array = [] hash.each { |key, value| -- 2.23.0

2 1

[PATCH lkp-tests] bootstrap: don't try to install ntpdate
by Wu Fengguang 08 Jan '21

08 Jan '21

It's not necessary for every boot. Let's not slow down every test run. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- lib/bootstrap.sh | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index d4b41f3b..b04ea3dd 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -237,20 +237,6 @@ run_ntpdate() [ -n "$LKP_SERVER" ] || return is_clearlinux && clearlinux_timesync && return - has_cmd ntpdate || { - detect_system - local DISTRO=$_system_name_lowercase - [ "$DISTRO" == "unknown" ] && { - echo "[WARNING] Current os cannot be recognized by lkp-test, so skip run_ntpdate." - return - } - has_cmd ruby || { - echo "[WARNING] Cannot find ruby in $DISTRO, so skip run_ntpdate." - return - } - . $LKP_SRC/distro/installer/$DISTRO $(echo ntpdate | $LKP_SRC/sbin/adapt-packages $DISTRO) - } - has_cmd ntpdate || return local hour="$(date +%H)" -- 2.23.0

1 0

[PATCH v3 lkp-tests] lib/bootstrap.sh: 'run-ipconfig' step should be before the check step
by Yu Chuan 08 Jan '21

08 Jan '21

[Why] The `run-ipconfig` step is the IP configuration step for initramfs; The `[ -z "$net_devices"]` step is check whether the network is normal. Now the `check normal` step is before 'ip configuration' step, so we encounter the following error: + export NO_NETWORK=1 + echo export NO_NETWORK=1 due to no net devices + return [Debug Msg] + [ ! -e /usr/share/initramfs-tools/scripts/functions ] + get_net_devices + local net_devices + local i + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + [ /sys/class/net/enp0s1/ != /sys/class/net/enp0s1/ ] + [ p0s1/ != /sys/class/net/enp0s1/ ] + basename /sys/class/net/enp0s1/ + net_devices= enp0s1 + [ != /sys/class/net/lo/ ] + continue + echo enp0s1 + local net_devices= enp0s1 + [ -z ] + warn_no_eth0 + [ -f /proc/config.gz ] + return + echo ls /sys/class/net + ls /sys/class/net + is_virt + [ -n ] + [ -n ] + has_cmd virt-what + command -v virt-what + grep -q -w hypervisor /proc/cpuinfo + return 0 + export NO_NETWORK=1 + echo export NO_NETWORK=1 due to no net devices + return Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 7d15bb370fa9..770d550724f1 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -160,6 +160,9 @@ setup_network() return } + $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig + network_ok && return + local net_devices=$(get_net_devices) if [ -z "$net_devices" ]; then @@ -181,9 +184,6 @@ setup_network() fi fi - $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-ipconfig - network_ok && return - local err_msg='IP-Config: Auto-configuration of network failed' dmesg | grep -q -F "$err_msg" || { # Include $err_msg in the error message so that it matches -- 2.23.0

2 3

[PATCH v3 lkp-tests] lib/bootstrap.sh: fix error when os is not integrated into lkp-tests
by Yu Chuan 08 Jan '21

08 Jan '21

[Why] If the current os has not been integrated into lkp-tests, run method `detect_system`, then var `$_system_name_lowercase` will be 'unknown'. So we should adapt this. [Error] + local DISTRO=unknown ++ echo ntpdate ++ /home/yuchuan/lkp-tests/sbin/adapt-packages unknown Traceback (most recent call last): 3: from /home/yuchuan/lkp-tests/sbin/adapt-packages:11:in `<main>' 2: from /usr/share/ruby/psych.rb:497:in `load_file' 1: from /usr/share/ruby/psych.rb:497:in `open' /usr/share/ruby/psych.rb:497:in `initialize': No such file or directory @ rb_sysopen - /home/yuchuan/lkp-tests/distro/adaptation/unknown (Errno::ENOENT) [How] If the current os has not been integrated into lkp-tests, then return. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 7d15bb370fa9..912a76bb6687 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -240,6 +240,10 @@ run_ntpdate() has_cmd ntpdate || { detect_system local DISTRO=$_system_name_lowercase + [ "$DISTRO" == "unknown" ] && { + echo "[WARNING] Current os cannot be recognized by lkp-test, so skip run_ntpdate." + return + } . $LKP_SRC/distro/installer/$DISTRO $(echo ntpdate | $LKP_SRC/sbin/adapt-packages $DISTRO) } -- 2.23.0

2 3

[PATCH v5 compass-ci] providers/qemu: adapt for x86_64 and riscv64
by Liu Yinsi 08 Jan '21

08 Jan '21

before: support kvm for aarch64 after: support kvm for aarch64, x86_64, riscv64 test: /srv/result/host-info/2021-01-07/vm-8p64g/debian-sid-riscv64/z9.288315 /srv/result/host-info/2021-01-07/vm-8p64g/debian-sid-x86_64/z9.288332 /srv/result/host-info/2021-01-07/vm-8p64g/debian-sid-aarch64/z9.288335 [how] put public options like $kernel, $initrd, $smp together, list different options like $machine, $bios, $nic according to different qemu command, then merge all options. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 110 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 95 insertions(+), 15 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index c926d3d..a8a2b33 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -18,12 +18,6 @@ check_logfile() } } -qemu_command() -{ - qemu=qemu-system-aarch64 - command -v $qemu >/dev/null || qemu=qemu-kvm -} - write_logfile() { ipxe_script=ipxe_script @@ -66,12 +60,56 @@ check_option_value() [ -n "$initrds" ] || exit } -get_initrd() +set_initrd() { initrd=initrd cat $initrds > $initrd } +set_bios() +{ + bios=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd + [ -f "$bios" ] || bios=/usr/share/ovmf/OVMF.fd +} + +set_helper() +{ + helper=/usr/libexec/qemu-bridge-helper + [ -f "$helper" ] || helper=/usr/lib/qemu/qemu-bridge-helper +} + +set_nic() +{ + nic="tap,model=virtio-net-pci,helper=$helper,br=br0,mac=${mac}" +} + +set_device() +{ + device="virtio-net-device,netdev=net0,mac=${mac}" +} + +set_netdev() +{ + netdev="bridge,br=br0,id=net0,helper=${helper}" +} + +set_qemu() +{ + qemu=( + qemu-system-aarch64 + qemu-kvm + qemu-system-x86_64 + ) + + for qemu in "${qemu[@]}" + do + [ -n "$(command -v ${qemu})" ] && break + done + + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 +} + print_message() { echo $SCHED_PORT @@ -83,36 +121,78 @@ print_message() sleep 5 } -run_qemu() +public_option() { kvm=( $qemu - -machine virt-4.0,accel=kvm,gic-version=3 -kernel $kernel -initrd $initrd -smp $nr_cpu -m $memory - -cpu Kunpeng-920 - -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd - -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} -k en-us -no-reboot -nographic -serial file:${log_file} -monitor null ) +} - "${kvm[@]}" --append "${append}" +run_qemu() +{ + case "$qemu" in + qemu-system-aarch64) + arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + ;; + qemu-kvm) + [ "$(arch)" == "aarch64" ] && arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + [ "$(arch)" == "x86_64" ] && arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-x86_64) + arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-riscv64) + arch_option=( + -machine virt + -device virtio-net-device,netdev=net0,mac=${mac} + -netdev bridge,br=br0,id=net0,helper=${helper} + ) + ;; + esac + + "${kvm[@]}" "${arch_option[@]}" --append "${append}" } check_logfile write_logfile -qemu_command parse_ipxe_script check_option_value -get_initrd + +set_initrd +set_bios +set_helper +set_nic +set_device +set_netdev +set_qemu print_message +public_option run_qemu -- 2.23.0

3 4

[PATCH lkp-tests] lib/bootstrap.sh: avoid the problem caused by the lack of ruby pkg
by Yu Chuan 08 Jan '21

08 Jan '21

[Why] We use $LKP_SRC/sbin/adapt-packages to find the ntpdate pkg suitable for different systems. But we are not sure that ruby is pre-installed in each system. So add a check, if ruby is not pre-installed, return. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 30e4af828bcd..d4b41f3bd0d5 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -244,6 +244,10 @@ run_ntpdate() echo "[WARNING] Current os cannot be recognized by lkp-test, so skip run_ntpdate." return } + has_cmd ruby || { + echo "[WARNING] Cannot find ruby in $DISTRO, so skip run_ntpdate." + return + } . $LKP_SRC/distro/installer/$DISTRO $(echo ntpdate | $LKP_SRC/sbin/adapt-packages $DISTRO) } -- 2.23.0

2 1

[PATCH lkp-tests] distro/depends: add vim in initramfs-common
by Zhang Yale 08 Jan '21

08 Jan '21

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- distro/depends/initramfs-common | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/initramfs-common b/distro/depends/initramfs-common index 5f766da6..a2ccf411 100644 --- a/distro/depends/initramfs-common +++ b/distro/depends/initramfs-common @@ -2,3 +2,4 @@ time wget perf which +vim -- 2.23.0

1 0

[PATCH lkp-tests] tests: add email
by Wei Jihui 08 Jan '21

08 Jan '21

send email to user when test job failed Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/email | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100755 tests/email diff --git a/tests/email b/tests/email new file mode 100755 index 00000000..fe870471 --- /dev/null +++ b/tests/email @@ -0,0 +1,41 @@ +#!/bin/sh +# - my_email +# - info + +: ${info:=failure} + +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/http.sh +. $PROJECT_DIR/lib/env.sh + +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Test job failed + +Dear $my_username: + + Sorry to inform you that your test job is failed. you can click the following link to view details. + https://compass-ci.openeuler.org/$result_root + +Regards +Compass-Ci +" +} + +compose_email() +{ + [ "$info" == "failure" ] && data_failure +} + +run_email() +{ + [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') + compose_email + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" +} + +run_email -- 2.23.0

2 1

[PATCH v3 compass-ci] scheduler: libvirt: a new boot type of libvirt
by Ren Wen 08 Jan '21

08 Jan '21

Requested by: curl -X GET 'http://<host>:<port>/boot.libvirt/mac/<mac_value>' Response :: some job fields in json format: - jod_id - kernel_uri - initrds_uri - kernel_params - result - LKP_SERVER - templates : this field is for libvirt Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index b2f0802..50ebe97 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -7,7 +7,7 @@ class Sched boot_type = @env.params.url["boot_type"] case boot_type - when "ipxe" + when "ipxe", "libvirt" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) when "grub" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) @@ -135,6 +135,28 @@ class Sched return response.to_json end + private def get_boot_libvirt(job : Job) + response = {} of String => String | Nil | Array(String) | JSON::Any + + response["job_id"] = job.id + response["kernel_uri"] = job.kernel_uri.lchop("kernel ") + + initrds_uri = job["initrds_uri"]? + initrds_uri = initrds_uri.as_a.map(&.as_s.lchop("initrd ")) if initrds_uri + response["initrds_uri"] = initrds_uri + + kernel_params = job["kernel_params"]? + kernel_params = kernel_params.as_a.map(&.to_s).reject(&.starts_with?("initrd=")).join(" ") if kernel_params + response["kernel_params"] = kernel_params + + response["result_root"] = job.result_root + response["LKP_SERVER"] = job["LKP_SERVER"]? + # *templates* is used by libvirt + response["templates"] = job["templates"]? + + return response.to_json + end + private def get_boot_grub(job : Job) initrd_lkp_cgz = "lkp-#{job.os_arch}.cgz" @@ -176,6 +198,8 @@ class Sched case boot_type when "ipxe" return job ? get_boot_ipxe(job) : ipxe_msg("No job now") + when "libvirt" + return job ? get_boot_libvirt(job) : {"job_id" => ""}.to_json when "grub" return job ? get_boot_grub(job) : grub_msg("No job now") when "container" -- 2.23.0

3 4

Re: [PATCH lkp-tests] update api-avx2neon
by Wu Fengguang 08 Jan '21

08 Jan '21

On Thu, Jan 07, 2021 at 12:28:52PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > tests/api-avx2neon | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > >diff --git a/tests/api-avx2neon b/tests/api-avx2neon >index 51097d6a8..9a64f89bf 100755 >--- a/tests/api-avx2neon >+++ b/tests/api-avx2neon >@@ -1,6 +1,17 @@ > #!/bin/bash > >-cd $BENCHMARK_ROOT/api-avx2neon/tests || exit >+ >+script=$(dirname $(realpath $0)) Name can be improved. It's SCRIPT_DIR not script. >+cd $script >+yum install -y git >+yum install -y make The 2 yum install can be merged. >+ >+git clone https://github.com/Siguyi/AvxToNeon >+ >+cd AvxToNeon >+cd tests The 2 cd can be merged. >+make > > echo "api-avx2neon running" > ./test >-- >2.23.0 >

1 1

[PATCH v3 lkp-tests 3/3] stats/crystal/kernbench.cr: fix some errors manually
by Lu Kaiyi 08 Jan '21

08 Jan '21

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr index 9d0661281..9777a11ca 100644 --- a/stats/crystal/kernbench.cr +++ b/stats/crystal/kernbench.cr @@ -1,10 +1,8 @@ #!/usr/bin/env crystal -LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.realpath(PROGRAM_NAME))) +require "../../lib/statistics" -require "#{LKP_SRC}/lib/statistics" - -$stdin.each_line do |line| +STDIN.each_line do |line| case line when /Elapsed Time/ puts "elapsed_time: " + line.split[2] -- 2.23.0

1 0

[PATCH v3 lkp-tests 2/3] stats/crystal/kernbench.cr: do first step conversion by ruby2crystal tools
by Lu Kaiyi 08 Jan '21

08 Jan '21

use the ruby2crystal tools do first conversion like below: /c/crystal/ruby_crystal_codemod/exe/ruby_crystal_codemod kernbench.cr /c/lkp-tests/tools/ruby2crystal.sh kernbench.cr Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr index 0da37fe88..9d0661281 100644 --- a/stats/crystal/kernbench.cr +++ b/stats/crystal/kernbench.cr @@ -1,28 +1,28 @@ -#!/usr/bin/env ruby +#!/usr/bin/env crystal -LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) +LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.realpath(PROGRAM_NAME))) require "#{LKP_SRC}/lib/statistics" $stdin.each_line do |line| case line when /Elapsed Time/ - puts 'elapsed_time: ' + line.split[2] - puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "elapsed_time: " + line.split[2] + puts "elapsed_time_stddev: " + line.split[3].gsub(/[()]/, "") when /User Time/ - puts 'user_time: ' + line.split[2] - puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "user_time: " + line.split[2] + puts "user_time_stddev: " + line.split[3].gsub(/[()]/, "") when /System Time/ - puts 'system_time: ' + line.split[2] - puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "system_time: " + line.split[2] + puts "system_time_stddev: " + line.split[3].gsub(/[()]/, "") when /Percent CPU/ - puts 'percent_cpu: ' + line.split[2] - puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "percent_cpu: " + line.split[2] + puts "percent_cpu_stddev: " + line.split[3].gsub(/[()]/, "") when /Context Switches/ - puts 'context_switches: ' + line.split[2] - puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "context_switches: " + line.split[2] + puts "context_switches_stddev: " + line.split[3].gsub(/[()]/, "") when /Sleeps/ - puts 'sleeps: ' + line.split[1] - puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + puts "sleeps: " + line.split[1] + puts "sleeps_stddev: " + line.split[2].gsub(/[()]/, "") end end -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/3] stats/crystal/kernbench.cr: create kernbench.cr as same content with stats/kernbench
by Lu Kaiyi 08 Jan '21

08 Jan '21

create kernbench.cr in crystal directory from stats/kernbench first, and then, the second patch will know what has changed from original file. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 stats/crystal/kernbench.cr diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr new file mode 100644 index 000000000..0da37fe88 --- /dev/null +++ b/stats/crystal/kernbench.cr @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby + +LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) + +require "#{LKP_SRC}/lib/statistics" + +$stdin.each_line do |line| + case line + when /Elapsed Time/ + puts 'elapsed_time: ' + line.split[2] + puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /User Time/ + puts 'user_time: ' + line.split[2] + puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /System Time/ + puts 'system_time: ' + line.split[2] + puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Percent CPU/ + puts 'percent_cpu: ' + line.split[2] + puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Context Switches/ + puts 'context_switches: ' + line.split[2] + puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Sleeps/ + puts 'sleeps: ' + line.split[1] + puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + end +end -- 2.23.0

1 0

[PATCH compass-ci] sbin/auto_submit.yaml: add/update linux repos
by Lin Jiaxin 08 Jan '21

08 Jan '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/auto_submit.yaml | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index 69b73ff..9ea88df 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -4,8 +4,20 @@ c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml l/linux/linux: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=nfs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=cifs os_arch=aarch64 build-linux.yaml +l/linux/linux-next: +- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=cifs os_arch=aarch64 build-linux.yaml +l/linux/openeuler-kernel: +- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=cifs os_arch=aarch64 build-linux.yaml +l/linux/debian-kernel: +- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=cifs os_arch=aarch64 build-linux.yaml +l/linux/archlinux-kernel: +- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml +- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=cifs os_arch=aarch64 build-linux.yaml archlinux: - testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=nfs os_arch=aarch64 build-pkg.yaml - testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

2 1

[PATCH compass-ci] git_mirror.rb: fix that delete dir not exist
by Li Yuanchao 07 Jan '21

07 Jan '21

As don't mkdir directly before, should not delete dir directly too. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index c7dd019..f5c653b 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -46,7 +46,6 @@ class GitMirror ret = !stderr.include?('fatal') break if ret end - FileUtils.rm_r(mirror_dir) unless ret return ret end -- 2.23.0

1 0

[PATCH v2 compass-ci] container/openresty-proxy-cache: add docker openresty-proxy-cache image
by cuiyili 07 Jan '21

07 Jan '21

When makepkg, find it locally and then get it remotely. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/openresty-proxy-cache/build | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100755 container/openresty-proxy-cache/build diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build new file mode 100755 index 0000000..0846197 --- /dev/null +++ b/container/openresty-proxy-cache/build @@ -0,0 +1,7 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +git clone https://gitee.com/cuiyili/openresty.git || exit 1 + +docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

2 1

[PATCH lkp-tests] depends/pgbench: add dependency package
by Hu Xuejiao 07 Jan '21

07 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- distro/depends/pgbench | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/pgbench b/distro/depends/pgbench index 6b2ea5aa..e8c054dc 100644 --- a/distro/depends/pgbench +++ b/distro/depends/pgbench @@ -1,3 +1,4 @@ postgresql postgresql-contrib sudo +postgresql-server -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/ebizzy: modify the value of iterations
by Hu Xuejiao 07 Jan '21

07 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- tests/ebizzy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/ebizzy b/tests/ebizzy index dcc301871..63c2f732a 100755 --- a/tests/ebizzy +++ b/tests/ebizzy @@ -19,7 +19,7 @@ duration=${duration%%s} [ -n "$mmap" ] && opt_mmap=" -m " [ -n "$linear" ] && opt_linear=" -z " - +[[ "$iterations" =~ ^[0-9]{3}x$ ]] && terations=100 for i in $(seq 1 $iterations) do echo Iteration: $i -- 2.23.0

2 1

[PATCH compass-ci] container/openresty-proxy-cache: add docker openresty-proxy-cache image
by cuiyili 07 Jan '21

07 Jan '21

When makepkg, find it locally and then get it remotely. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/openresty-proxy-cache/build | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100755 container/openresty-proxy-cache/build diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build new file mode 100755 index 0000000..0846197 --- /dev/null +++ b/container/openresty-proxy-cache/build @@ -0,0 +1,7 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +git clone https://gitee.com/cuiyili/openresty.git || exit 1 + +docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin: add email-mapping
by Luan Shengde 07 Jan '21

07 Jan '21

for user to: - add - delete - re-map - search email mapping from stdin or file Usage: email_mapping -n name -e email_address email_mapping -n name -e email_address -r email_mapping -n name -d email_mapping -s name email_mapping -f mapping_file email_mapping -f mapping_file -r|-d options: -n, --name name|email|logo appoint a name|email|logo to add mapping -e, --email email_address appoint a email to be mapped -f, --file filename appoint a mapping file for name/email -r, --re-map do re-mappings -d, --delete delete email mappings -s, --search search email mapping -h, --help show this message the name can be either a name, an email or something like logo the file content consists of names and emails, like: name1: email_address1 name2: email_address2 you can map multi name to a single email address Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/email-mapping.rb | 90 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100755 sbin/email-mapping.rb diff --git a/sbin/email-mapping.rb b/sbin/email-mapping.rb new file mode 100755 index 0000000..09338c8 --- /dev/null +++ b/sbin/email-mapping.rb @@ -0,0 +1,90 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'redis' +require 'json' +require 'yaml' +require 'optparse' + +require "#{ENV['CCI_SRC']}/lib/email_mapping_client" + +r_name = nil +d_email = nil +email_mapping = nil +re_mapping = false +del_map = false +search_map = false + +options = OptionParser.new do |opts| + opts.banner = "Usage: email_mapping -n name -e email_address\n" + opts.banner += " email_mapping -n name -e email_address -r\n" + opts.banner += " email_mapping -n name -d\n" + opts.banner += " email_mapping -s name\n" + opts.banner += " email_mapping -f mapping_file\n" + opts.banner += " email_mapping -f mapping_file -r|-d\n" + + opts.separator '' + opts.separator 'options:' + + opts.on('-n name|email|logo', '--name name|email|logo', 'appoint a name|email|logo to add mapping') do |name| + r_name = name + end + + opts.on('-e email_address', '--email email_address', 'appoint a email to be mapped') do |email| + d_email = email + end + + opts.on('-f filename', '--file filename', 'appoint a mapping file for name/email') do |filename| + mapping_content = YAML.load_file(filename) || nil + email_mapping = mapping_content + end + + opts.on('-r', '--re-map', 'do re-mappings') do + re_mapping = true + end + + opts.on('-d', '--delete', 'delete email mappings') do + del_map = true + end + + opts.on('-s', '--search', 'search email mapping') do + search_map = true + end + + opts.on_tail('-h', '--help', 'show this message') do + puts opts + exit + end +end + +if ARGV.empty? + ARGV << '-h' +elsif (['-n', '-f'] - ARGV).eql? ['-n', '-f'] + ARGV.clear + ARGV << '-h' +end + +options.parse!(ARGV) + +def email_mapping(r_name, d_email, re_mapping, del_map, search_map) + email_mapping = AddEmailMapping.new(r_name, d_email) + if re_mapping + email_mapping.re_map_email + elsif del_map + email_mapping.delete_email_mapping + elsif search_map + puts email_mapping.search_map + else + email_mapping.add_email_mapping + end +end + +if email_mapping.nil? + email_mapping(r_name, d_email, re_mapping, del_map, search_map) +else + email_mapping.each do |name, email| + email_mapping(name, email, re_mapping, del_map, search_map) + end +end -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib: add email_mapping_client
by Luan Shengde 07 Jan '21

07 Jan '21

used to - add - delete - re-map - search email mappings Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/email_mapping_client.rb | 45 +++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 lib/email_mapping_client.rb diff --git a/lib/email_mapping_client.rb b/lib/email_mapping_client.rb new file mode 100755 index 0000000..ec82d93 --- /dev/null +++ b/lib/email_mapping_client.rb @@ -0,0 +1,45 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'redis' +require 'json' + +# used to add/delete/re-map/search email mappings +class AddEmailMapping + REDIS_HOST = ENV['REDIS_HOST'] || 'localhost' + REDIS_PORT = ENV['REDIS_PORT'] || '6379' + + def initialize(r_name, d_email) + @r_name = r_name + @d_email = d_email + @redis = Redis.new('host' => REDIS_HOST, 'port' => REDIS_PORT) + end + + def add_email_mapping + if @redis.hexists 'email_mapping', @r_name + puts "#{@r_name} has already mapped an email." + else + @redis.hset 'email_mapping', @r_name, @d_email + puts "Successfully mapped email for #{@r_name}." + end + end + + def delete_email_mapping + @redis.hdel 'email_mapping', @r_name + end + + def re_map_email + @redis.hset 'email_mapping', @r_name, @d_email + puts "Successfully re-mapped email for #{@r_name}." + end + + def search_map + d_email = @redis.hget 'email_mapping', @r_name + + return d_email if d_email + + return "#{@r_name} has not mapped an email yet." + end +end -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/2] tests: add email
by Wei Jihui 07 Jan '21

07 Jan '21

send email to user when test job failed Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/email | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 tests/email diff --git a/tests/email b/tests/email new file mode 100755 index 00000000..40b48fd9 --- /dev/null +++ b/tests/email @@ -0,0 +1,40 @@ +#!/bin/sh +# - my_email +# - info + +: ${info:=failure} + +SCRIPT_DIR=$(dirname $(realpath $0)) +PROJECT_DIR=$(dirname $SCRIPT_DIR) + +. $PROJECT_DIR/lib/http.sh +. $PROJECT_DIR/lib/env.sh + +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Test job failed + +Dear $my_name: + + Sorry to inform you that your test job is failed. you can click the following link to view details. + http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root + +Regards +Compass-Ci +" +} + +compose_email() +{ + [ "$info" == "failure" ] && data_failure +} + +run_email() +{ + [ -n "$my_email" ] || return 0 + compose_email + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" +} + +run_email -- 2.23.0

2 1

[PATCH v2 lkp-tests] tests/email: get username from my_email
by Wei Jihui 07 Jan '21

07 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/email | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/email b/tests/email index 40b48fd9..a5f87d42 100755 --- a/tests/email +++ b/tests/email @@ -15,7 +15,7 @@ data_failure() data="To: $my_email Subject: [NOTIFY Compass-ci] Test job failed -Dear $my_name: +Dear $my_username: Sorry to inform you that your test job is failed. you can click the following link to view details. http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root @@ -33,6 +33,7 @@ compose_email() run_email() { [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" } -- 2.23.0

3 2

[PATCH v2 lkp-tests 2/2] stats/crystal/kernbench.cr: kernbench to kernbench.cr
by Lu Kaiyi 07 Jan '21

07 Jan '21

change kernbench into kernbench.cr Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr index 0da37fe88..9777a11ca 100644 --- a/stats/crystal/kernbench.cr +++ b/stats/crystal/kernbench.cr @@ -1,28 +1,26 @@ -#!/usr/bin/env ruby +#!/usr/bin/env crystal -LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) +require "../../lib/statistics" -require "#{LKP_SRC}/lib/statistics" - -$stdin.each_line do |line| +STDIN.each_line do |line| case line when /Elapsed Time/ - puts 'elapsed_time: ' + line.split[2] - puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "elapsed_time: " + line.split[2] + puts "elapsed_time_stddev: " + line.split[3].gsub(/[()]/, "") when /User Time/ - puts 'user_time: ' + line.split[2] - puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "user_time: " + line.split[2] + puts "user_time_stddev: " + line.split[3].gsub(/[()]/, "") when /System Time/ - puts 'system_time: ' + line.split[2] - puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "system_time: " + line.split[2] + puts "system_time_stddev: " + line.split[3].gsub(/[()]/, "") when /Percent CPU/ - puts 'percent_cpu: ' + line.split[2] - puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "percent_cpu: " + line.split[2] + puts "percent_cpu_stddev: " + line.split[3].gsub(/[()]/, "") when /Context Switches/ - puts 'context_switches: ' + line.split[2] - puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "context_switches: " + line.split[2] + puts "context_switches_stddev: " + line.split[3].gsub(/[()]/, "") when /Sleeps/ - puts 'sleeps: ' + line.split[1] - puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + puts "sleeps: " + line.split[1] + puts "sleeps_stddev: " + line.split[2].gsub(/[()]/, "") end end -- 2.23.0

2 1

[PATCH v2 lkp-tests] jobs/ebizzy.yaml: modify the value of iterations
by Hu Xuejiao 07 Jan '21

07 Jan '21

[why] seq: invalid floating point argument: 100x Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- jobs/ebizzy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/ebizzy.yaml b/jobs/ebizzy.yaml index 9a3e9a16..cc9994c8 100644 --- a/jobs/ebizzy.yaml +++ b/jobs/ebizzy.yaml @@ -3,7 +3,7 @@ testcase: ebizzy category: benchmark nr_threads: 200% -iterations: 100x +iterations: 100 ebizzy: duration: 10s -- 2.23.0

2 2

[PATCH v3 compass-ci] 3-code/git: modify the git download path
by Hu Xuejiao 07 Jan '21

07 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/3-code/git | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/sparrow/3-code/git b/sparrow/3-code/git index 48741ec..d72b531 100755 --- a/sparrow/3-code/git +++ b/sparrow/3-code/git @@ -5,14 +5,10 @@ # fix files permissions cause container service start failed umask 002 -# download git trees - -cd /c || exit - -# git clone https://gitee.com/openeuler/compass-ci.git +# git clone compass-ci, lkp-tests, upstream-repos # modify and manual run for now: -[ -d "compass-ci/.git" ] || { - git clone https://gitee.com/wu_fengguang/compass-ci.git +[ -d "/c/compass-ci/.git" ] || { + git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci cd /c/compass-ci || exit [ -n "$compass_commit_id" ] && git reset --hard "$compass_commit_id" @@ -20,15 +16,15 @@ cd /c || exit ln -s compass-ci /c/cci } -[ -d "lkp-tests/.git" ] || { - git clone https://gitee.com/wu_fengguang/lkp-tests.git +[ -d "/c/lkp-tests/.git" ] || { + git clone https://gitee.com/wu_fengguang/lkp-tests.git /c/lkp-tests cd /c/lkp-tests || exit [ -n "$lkp_commit_id" ] && git reset --hard "$lkp_commit_id" } -[ -d "upstream-repos/.git" ] || { - git clone https://gitee.com/wu_fengguang/upstream-repos.git +[ -d "/c/upstream-repos/.git" ] || { + git clone https://gitee.com/wu_fengguang/upstream-repos.git /c/upstream-repos cd /c/upstream-repos || exit [ -n "$upstream_commit_id" ] && git reset --hard "$upstream_commit_id" -- 2.23.0

2 1

[PATCH v4 compass-ci 5/7] send-internet-mail: email rate limit
by Luan Shengde 07 Jan '21

07 Jan '21

add email_in_limit to store email address that send count less than limit value add email_out_limit to store email address that send count beyond limit value the email queues email_in_limit/email_out_limit will be reset everyday at 00:00, every time send mail to an email address, the value for the email address in email_in_limit will +1, until the value up to the limited value, the email address will be moved to email_out_limit, and the email send to the address will be throw away. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../send-internet-mail/email_limit_queue.rb | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100755 container/send-internet-mail/email_limit_queue.rb diff --git a/container/send-internet-mail/email_limit_queue.rb b/container/send-internet-mail/email_limit_queue.rb new file mode 100755 index 0000000..9891ae9 --- /dev/null +++ b/container/send-internet-mail/email_limit_queue.rb @@ -0,0 +1,62 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'redis' + +# check if the emails count has beyond the limit +# email_in_limit is used to store email address email count +# less than the limit value +# email_out_limit is used to store email address email count +# beyond the limit +# case the email is the first one for the day, the email will +# be add to email_in_limit with value 1 +# case the email in email_in_limit but email count less than the +# limit value, its value will +1 +# case the email's email count up to the limit value, the email +# will be moved to email_out_limit +# when the email is in email_out_limit, if send mail to the email +# address, the email will be kicked out from the email address bar +class EmailRateLimit + def initialize(mail_info) + @mail_info = mail_info + @redis = Redis.new('host' => REDIS_HOST, 'port' => REDIS_PORT) + end + + def check_email_counts + email_to = @mail_info['to'].clone + email_cc = @mail_info['cc'].clone + @mail_info['to'] = check_emails(email_to) + @mail_info['cc'] = check_emails(email_cc) + + return @mail_info + end + + def check_emails(mail_list) + return if mail_list.nil? || mail_list.empty? + + mail_list.clone.each do |email| + if @redis.hexists 'email_out_limit', email + mail_list -= [email] + next + elsif @redis.hexists 'email_in_limit', email + email_account = @redis.hget 'email_in_limit', email + @redis.hset 'email_in_limit', email, email_account.to_i + 1 + else + @redis.hset 'email_in_limit', email, 1 + end + + change_queue(email) + end + return mail_list + end + + def change_queue(email) + return unless (@redis.hget 'email_in_limit', email).to_i >= 10 + + @redis.hdel 'email_in_limit', email + @redis.hset 'email_out_limit', email, 10 + end +end -- 2.23.0

3 3

[PATCH lkp-tests] jobs/ebizzy.yaml: modify the value of iterations
by Hu Xuejiao 07 Jan '21

07 Jan '21

[why] seq: invalid floating point argument: 100 Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- jobs/ebizzy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/ebizzy.yaml b/jobs/ebizzy.yaml index 9a3e9a16..cc9994c8 100644 --- a/jobs/ebizzy.yaml +++ b/jobs/ebizzy.yaml @@ -3,7 +3,7 @@ testcase: ebizzy category: benchmark nr_threads: 200% -iterations: 100x +iterations: 100 ebizzy: duration: 10s -- 2.23.0

2 1

[PATCH v2 lkp-tests 1/2] stats/crystal/kernbench.cr: create kernbench.cr as same content with stats/kernbench
by Lu Kaiyi 07 Jan '21

07 Jan '21

create kernbench.cr in crystal directory from stats/kernbench first, and then, the second patch will know what has changed from original file. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 stats/crystal/kernbench.cr diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr new file mode 100644 index 000000000..0da37fe88 --- /dev/null +++ b/stats/crystal/kernbench.cr @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby + +LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) + +require "#{LKP_SRC}/lib/statistics" + +$stdin.each_line do |line| + case line + when /Elapsed Time/ + puts 'elapsed_time: ' + line.split[2] + puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /User Time/ + puts 'user_time: ' + line.split[2] + puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /System Time/ + puts 'system_time: ' + line.split[2] + puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Percent CPU/ + puts 'percent_cpu: ' + line.split[2] + puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Context Switches/ + puts 'context_switches: ' + line.split[2] + puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Sleeps/ + puts 'sleeps: ' + line.split[1] + puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + end +end -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add openeuler_docker.yaml
by Bai Jing 07 Jan '21

07 Jan '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/openeuler_docker.yaml | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 jobs/openeuler_docker.yaml diff --git a/jobs/openeuler_docker.yaml b/jobs/openeuler_docker.yaml new file mode 100644 index 000000000..376ece156 --- /dev/null +++ b/jobs/openeuler_docker.yaml @@ -0,0 +1,5 @@ +suite: openeuler_docker +category: benchmark +openeuler_docker: + dockerfile_url: + - CentOS/CentOS-Dockerfiles.git -- 2.23.0

2 4

[PATCH v2 lkp-tests] jobs: add openeuler_docker.yaml
by Bai Jing 07 Jan '21

07 Jan '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/openeuler_docker.yaml | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 jobs/openeuler_docker.yaml diff --git a/jobs/openeuler_docker.yaml b/jobs/openeuler_docker.yaml new file mode 100644 index 000000000..a5db34eef --- /dev/null +++ b/jobs/openeuler_docker.yaml @@ -0,0 +1,6 @@ +suite: openeuler_docker +category: benchmark + +openeuler_docker: + dockerfile_url: + - CentOS/CentOS-Dockerfiles.git -- 2.23.0

1 0

[PATCH lkp-tests] test/email: get username from my_email
by Wei Jihui 07 Jan '21

07 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/email | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/email b/tests/email index 40b48fd9..a5f87d42 100755 --- a/tests/email +++ b/tests/email @@ -15,7 +15,7 @@ data_failure() data="To: $my_email Subject: [NOTIFY Compass-ci] Test job failed -Dear $my_name: +Dear $my_username: Sorry to inform you that your test job is failed. you can click the following link to view details. http://api.compass-ci.openeuler.org:$SRV_HTTP_PORT/$result_root @@ -33,6 +33,7 @@ compose_email() run_email() { [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-49000}/send_mail_text" -d "$data" } -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] tests/host-info: computing vm memory
by Wei Jihui 07 Jan '21

07 Jan '21

[why] it can not compute vm memory when vm memory less than 16G. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/host-info | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index fe55d1ff..2721648c 100755 --- a/tests/host-info +++ b/tests/host-info @@ -4,8 +4,8 @@ get_memory() { memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') - memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') + memory=$(echo $memory_new | awk '{print int($0)+1}')G echo "memory: $memory" } -- 2.23.0

1 0

[PATCH compass-ci] 4-docker/buildall: modify the logic
by Hu Xuejiao 07 Jan '21

07 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index ca4406b..ee257b0 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -33,7 +33,7 @@ do_one() cd $container [ "$container_name" == 'ssh-r' ] && exit docker images | grep -wq "$container_name" - if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then + if [ "$?" != 0 ] && [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

2 1

[PATCH v4 compass-ci] providers/qemu: adapt for x86_64 and riscv64
by Liu Yinsi 07 Jan '21

07 Jan '21

before: support kvm for aarch64 after: support kvm for aarch64, x86_64, riscv64 [how] put public options like $kernel, $initrd, $smp together, list different options like $machine, $bios, $nic according to different qemu command, then merge all options. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 106 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 93 insertions(+), 13 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index c926d3d..8749568 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -18,12 +18,6 @@ check_logfile() } } -qemu_command() -{ - qemu=qemu-system-aarch64 - command -v $qemu >/dev/null || qemu=qemu-kvm -} - write_logfile() { ipxe_script=ipxe_script @@ -72,6 +66,50 @@ get_initrd() cat $initrds > $initrd } +get_bios() +{ + bios=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd + [ -f "$bios" ] || bios=/usr/share/ovmf/OVMF.fd +} + +get_helper() +{ + helper=/usr/libexec/qemu-bridge-helper + [ -f "$helper" ] || helper=/usr/lib/qemu/qemu-bridge-helper +} + +get_nic() +{ + nic="tap,model=virtio-net-pci,helper=$helper,br=br0,mac=${mac}" +} + +get_device() +{ + device="virtio-net-device,netdev=net0,mac=${mac}" +} + +get_netdev() +{ + netdev="bridge,br=br0,id=net0,helper=${helper}" +} + +get_qemu() +{ + qemu=( + qemu-system-aarch64 + qemu-kvm + qemu-system-x86_64 + ) + + for qemu in "${qemu[@]}" + do + [ -n "$(command -v ${qemu})" ] && break + done + + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 +} + print_message() { echo $SCHED_PORT @@ -83,36 +121,78 @@ print_message() sleep 5 } -run_qemu() +public_option() { kvm=( $qemu - -machine virt-4.0,accel=kvm,gic-version=3 -kernel $kernel -initrd $initrd -smp $nr_cpu -m $memory - -cpu Kunpeng-920 - -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd - -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} -k en-us -no-reboot -nographic -serial file:${log_file} -monitor null ) +} - "${kvm[@]}" --append "${append}" +run_qemu() +{ + case "$qemu" in + qemu-system-aarch64) + arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + ;; + qemu-kvm) + [ "$(arch)" == "aarch64" ] && arch_option=( + -machine virt-4.0,accel=kvm,gic-version=3 + -cpu Kunpeng-920 + -bios $bios + -nic $nic + ) + [ "$(arch)" == "x86_64" ] && arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-x86_64) + arch_option=( + -bios $bios + -nic $nic + ) + ;; + qemu-system-riscv64) + arch_option=( + -machine virt + -device virtio-net-device,netdev=net0,mac=${mac} + -netdev bridge,br=br0,id=net0,helper=${helper} + ) + ;; + esac + + "${kvm[@]}" "${arch_option[@]}" --append "${append}" } check_logfile write_logfile -qemu_command parse_ipxe_script check_option_value + get_initrd +get_bios +get_helper +get_nic +get_device +get_netdev +get_qemu print_message +public_option run_qemu -- 2.23.0

2 2

[PATCH v2 compass-ci] sparrow: add setup.yaml
by Liu Yinsi 07 Jan '21

07 Jan '21

[why] before deploy compass-ci, fill configuration setup.yaml firstly, will make the whole process cleaner. fill my_name and my_email, it's necessary to verify account when submit job. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..b1cac8c --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,11 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. +# mkdir -p /etc/compass-ci/ +# cp setup.yaml /etc/compass-ci/setup.yaml +# vim /etc/compass-ci/setup.yaml + +# account name +my_name: xxxxxx + +# email address +my_email: xxx@xxx -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker: fix logical error
by Liu Yinsi 07 Jan '21

07 Jan '21

only when images not exist or execute ./buildall without parameters "reboot", need to execute ./build and ./install, otherwise just execute ./first-run and ./start Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index ca4406b..59a0770 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -33,7 +33,7 @@ do_one() cd $container [ "$container_name" == 'ssh-r' ] && exit docker images | grep -wq "$container_name" - if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then + if [ "$?" != 0 ] || [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

2 2

[PATCH v5 compass-ci] providers/qemu: refactor kvm script
by Liu Yinsi 07 Jan '21

07 Jan '21

encapsulating into functions. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 160 +++++++++++++++++++++++++----------------- 1 file changed, 97 insertions(+), 63 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index fd921dc..c926d3d 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -7,78 +7,112 @@ : ${nr_cpu:=1} : ${memory:=1G} -log_file=/srv/cci/serial/logs/${hostname} -if [ ! -f "$log_file" ]; then - touch $log_file - # fluentd refresh time is 1s - # let fluentd to monitor this file first - sleep 2 -fi +check_logfile() +{ + log_file=/srv/cci/serial/logs/${hostname} + [ -f "$log_file" ] || { + touch $log_file + # fluentd refresh time is 1s + # let fluentd to monitor this file first + sleep 2 + } +} + +qemu_command() +{ + qemu=qemu-system-aarch64 + command -v $qemu >/dev/null || qemu=qemu-kvm +} -qemu=qemu-system-aarch64 -command -v $qemu >/dev/null || qemu=qemu-kvm +write_logfile() +{ + ipxe_script=ipxe_script + curl http://${SCHED_HOST:-172.17.0.1}:${SCHED_PORT:-3000}/boot.ipxe/mac/${mac} > $ipxe_script + cat $ipxe_script >> ${log_file} +} -ipxe_script=ipxe_script -curl http://${SCHED_HOST:-172.17.0.1}:${SCHED_PORT:-3000}/boot.ipxe/mac/${mac} > $ipxe_script -cat $ipxe_script >> ${log_file} +parse_ipxe_script() +{ + append= + initrds= + while read a b c + do + case "$a" in + '#') + ;; + initrd) + file=$(basename "$b") + wget --timestamping -a ${log_file} --progress=bar:force $b + initrds+="$file " + ;; + kernel) + kernel=$(basename "$b") + wget --timestamping -a ${log_file} --progress=bar:force $b + append=$(echo "$c" | sed -r "s/ initrd=[^ ]+//g") + ;; + *) + ;; + esac + done < $ipxe_script +} -append= -initrds= -while read a b c -do - case "$a" in - '#') - ;; - initrd) - file=$(basename "$b") - wget --timestamping -a ${log_file} --progress=bar:force $b - initrds+="$file " - ;; - kernel) - kernel=$(basename "$b") - wget --timestamping -a ${log_file} --progress=bar:force $b - append=$(echo "$c" | sed -r "s/ initrd=[^ ]+//g") - ;; - *) - ;; - esac -done < $ipxe_script +check_option_value() +{ + [ -s "$kernel" ] || { + echo "can't get kernel or kernel size is 0" + exit + } + + [ -n "$initrds" ] || exit +} -[ -s "$kernel" ] || { - echo "can't get kernel or kernel size is 0" - exit +get_initrd() +{ + initrd=initrd + cat $initrds > $initrd } -[ -n "$initrds" ] || { - exit +print_message() +{ + echo $SCHED_PORT + echo kernel: $kernel + echo initrds: $initrds + echo append: $append + echo less $log_file + + sleep 5 } -initrd=initrd -cat $initrds > $initrd +run_qemu() +{ + kvm=( + $qemu + -machine virt-4.0,accel=kvm,gic-version=3 + -kernel $kernel + -initrd $initrd + -smp $nr_cpu + -m $memory + -cpu Kunpeng-920 + -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd + -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} + -k en-us + -no-reboot + -nographic + -serial file:${log_file} + -monitor null + ) + + "${kvm[@]}" --append "${append}" +} -echo $SCHED_PORT -echo kernel: $kernel -echo initrds: $initrds -echo append: $append -echo less $log_file +check_logfile +write_logfile -sleep 5 +qemu_command +parse_ipxe_script +check_option_value +get_initrd -kvm=( - $qemu - -machine virt-4.0,accel=kvm,gic-version=3 - -kernel $kernel - -initrd $initrd - -smp $nr_cpu - -m $memory - -cpu Kunpeng-920 - -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd - -nic tap,model=virtio-net-pci,helper=/usr/libexec/qemu-bridge-helper,br=br0,mac=${mac} - -k en-us - -no-reboot - -nographic - -serial file:${log_file} - -monitor null -) +print_message -"${kvm[@]}" --append "${append}" +run_qemu -- 2.23.0

2 2

[PATCH v6 compass-ci 2/2] lib/es_query.rb: parse es_result to a Hash result
by Lu Kaiyi 07 Jan '21

07 Jan '21

[why] make es result more easy access. [example] input: es_result = [{"key"=>"build-pkg", "doc_count"=>354526, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>5708}, {"key"=>"finished", "doc_count"=>1033}, {"key"=>"incomplete", "doc_count"=>204}, {"key"=>"submit", "doc_count"=>136}, ... output: result_hash = {"build-pkg"=> {"failed"=>5708, "finished"=>1033, "incomplete"=>204, "submit"=>136, "OOM"=>11, "post_run"=>2}, "cci-depends"=> {"finished"=>1785, "failed"=>675, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 91aa6ab..207aaae 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -128,7 +128,7 @@ class ESQuery result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] return nil if result.empty? - result + parse_fields(result) end end @@ -216,3 +216,47 @@ def build_aggs_from_fields(fields) aggs_hash['aggs']["all_#{field}"].merge!(sub_aggs) if sub_aggs aggs_hash end + +# input: +# es_result = +# [{"key"=>"build-pkg", +# "doc_count"=>354526, +# "all_job_state"=> +# {"doc_count_error_upper_bound"=>0, +# "sum_other_doc_count"=>0, +# "buckets"=> +# [{"key"=>"failed", "doc_count"=>5708}, +# {"key"=>"finished", "doc_count"=>1033}, +# {"key"=>"incomplete", "doc_count"=>204}, +# {"key"=>"submit", "doc_count"=>136}, +# ... +# +# output: +# result_hash = +# {"build-pkg"=> +# {"failed"=>5708, +# "finished"=>1033, +# "incomplete"=>204, +# "submit"=>136, +# "OOM"=>11, +# "post_run"=>2}, +# "cci-depends"=> +# {"finished"=>1785, +# "failed"=>675, +# ... +def parse_fields(es_result) + result_hash ||= {} + es_result.each do |result| + key = result['key'] + sub_field = result.keys.detect { |field| field.start_with?('all_') } + + if sub_field + all_field = result[sub_field]['buckets'] + result_hash[key] = parse_fields(all_field) + else + result_hash[key] = result['doc_count'] + end + end + + result_hash +end -- 2.23.0

1 0

[PATCH v6 compass-ci 1/2] lib/es_query.rb: get multi-field aggregation count result
by Lu Kaiyi 07 Jan '21

07 Jan '21

[why] get multi-field aggregation count for overview of report page. [input]: fields, query_items (optional for query_items, default no scope limitation) [example] input: fields = ['suite'] query_items = { 'os' => 'openeuler' } output: [{"key"=>"build-pkg", "doc_count"=>186214}, {"key"=>"cci-depends", "doc_count"=>1230}, {"key"=>"iperf", "doc_count"=>1103}, {"key"=>"spinlock", "doc_count"=>1000}, {"key"=>"cci-makepkg", "doc_count"=>949}, {"key"=>"iperf-walk-os-test", "doc_count"=>474}, {"key"=>"bisect", "doc_count"=>381}, ... input: fields = ['suite', 'job_state'] query_items = { 'os' => 'openeuler' } output: [{"key"=>"build-pkg", "doc_count"=>186227, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>3830}, {"key"=>"finished", "doc_count"=>803}, {"key"=>"incomplete", "doc_count"=>196}, {"key"=>"submit", "doc_count"=>136}, ... input: fields = ['os', 'os_version', 'job_state'] query_items = {} output: [{"key"=>"openeuler", "doc_count"=>193287, "all_os_version"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"20.03", "doc_count"=>191789, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>3909}, {"key"=>"finished", "doc_count"=>2401}, {"key"=>"incomplete", "doc_count"=>300}, {"key"=>"submit", "doc_count"=>136}, {"key"=>"OOM", "doc_count"=>23}, {"key"=>"post_run", "doc_count"=>1}]}}, {"key"=>"20.03-SP1", "doc_count"=>904, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>641}, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 70 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/lib/es_query.rb b/lib/es_query.rb index 88a420f..91aa6ab 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -87,6 +87,49 @@ class ESQuery result end + + # input: + # fields, query_items + # + # fields => fields: Array(keyword) + # eg: + # ['suite', 'job_state'] + # + # query_items => query_items: Hash(keyword, value) + # eg: + # { + # 'os' => 'openeuler', + # ... + # } + # (optional for query_items, default no scope limitation) + # output: + # [{"key"=>"build-pkg", + # "doc_count"=>186175, + # "all_job_state"=> + # {"doc_count_error_upper_bound"=>0, + # "sum_other_doc_count"=>0, + # "buckets"=> + # [{"key"=>"failed", "doc_count"=>3830}, + # {"key"=>"finished", "doc_count"=>803}, + # {"key"=>"incomplete", "doc_count"=>196}, + # ... + def query_fields(fields, query_items = {}) + field1 = fields.first + aggs_hash = build_aggs_from_fields(fields) + query = { + query: { + bool: { + must: build_multi_field_subquery_body(query_items) + } + }, + aggs: aggs_hash['aggs'], + size: 0 + } + result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] + return nil if result.empty? + + result + end end # Range Query Example: @@ -146,3 +189,30 @@ def assign_desc_body(keyword) }] } end + +# input: +# fields = ['os', 'os_version', 'job_state'] +# output: +# aggs_hash = { +# "aggs"=> +# {"all_os"=> +# {"terms"=>{:field=>"os", :size=>1000}, +# "aggs"=> +# {"all_os_version"=> +# {"terms"=>{:field=>"os_version", :size=>1000}, +# "aggs"=> +# {"all_job_state"=>{"terms"=>{:field=>"job_state", :size=>1000}}}}}}} +# } +def build_aggs_from_fields(fields) + aggs_hash = {} + return if fields.empty? + + field = fields.shift + aggs_hash['aggs'] ||= {} + aggs_hash['aggs']["all_#{field}"] = { + 'terms' => { field: field, size: 1000 } + } + sub_aggs = build_aggs_from_fields(fields) + aggs_hash['aggs']["all_#{field}"].merge!(sub_aggs) if sub_aggs + aggs_hash +end -- 2.23.0

1 0

[PATCH v4 compass-ci 7/7] send-internet-mail/start: add key REDIS_PORT
by Luan Shengde 07 Jan '21

07 Jan '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start index 02c6ab6..136435d 100755 --- a/container/send-internet-mail/start +++ b/container/send-internet-mail/start @@ -13,6 +13,7 @@ names = Set.new %w[ ROBOT_EMAIL_PASSWORD SEND_MAIL_PORT SENT_MAILDIR + REDIS_PORT ] defaults = relevant_defaults(names) -- 2.23.0

1 0

[PATCH v4 compass-ci 6/7] lib/mail-post: add email checks
by Luan Shengde 07 Jan '21

07 Jan '21

add check email count only email addresses that email count within the limited value can be sent emails add check email mapping if has set email mapping for the entered address. the address will be converted to the mapping value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail-post.rb | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) mode change 100644 => 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb old mode 100644 new mode 100755 index 9d3e601..803a134 --- a/lib/mail-post.rb +++ b/lib/mail-post.rb @@ -57,14 +57,32 @@ post '/send_mail_encode' do send_mail(mail_info) end +def check_email_limit(mail_info) + email_limit = EmailRateLimit.new(mail_info) + email_limit.check_email_counts +end + +def check_email_mapping(mail_info) + email_mapping = EmailAddrMapping.new(mail_info) + email_mapping.check_email_mapping +end + def check_send_mail(mail_info) raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? + + return mail_info unless ENV['SEND_MAIL_PORT'].to_s == '49000' + + mail_info = check_email_mapping(mail_info.clone) + mail_info = check_email_limit(mail_info.clone) + return mail_info end def send_mail(mail_info) - check_send_mail(mail_info) + mail_info = check_send_mail(mail_info) + return if mail_info['to'].empty? + mail = Mail.new do from ENV['ROBOT_EMAIL_ADDRESS'] subject mail_info['subject'] @@ -79,7 +97,6 @@ def send_mail(mail_info) end def check_to_store_email(mail) - return if ENV['SEND_MAIL_PORT'].to_s != '49000' return if ENV['HOST_SERVER'] != 'z9' -- 2.23.0

1 0

[PATCH v4 compass-ci 4/7] send-internet-mail: add email mapping
by Luan Shengde 07 Jan '21

07 Jan '21

example: name => email_addr logo => email_addr email_addr_r => email_addr_d an email address can be set multi mappings, you can set a name or email to a specified email address when you write either a name or an email address in the send address bar, the email address will be converted to the specified address Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/email_mapping.rb | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100755 container/send-internet-mail/email_mapping.rb diff --git a/container/send-internet-mail/email_mapping.rb b/container/send-internet-mail/email_mapping.rb new file mode 100755 index 0000000..4c068b0 --- /dev/null +++ b/container/send-internet-mail/email_mapping.rb @@ -0,0 +1,49 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'redis' + +# email address translation according to the email_mapping queue +# email_mapping is used to set email mapping, example: +# email_mapping: +# name => email_address_d +# email_address_r1 => email_address_d +# email_address_r2 => email_address_d +# ... +# can set multi keys for an email address, the key can be a name, +# an email, or some logo else +# case add the key is added to email address bar, the key will be +# transferred to the destination email address +class EmailAddrMapping + def initialize(mail_info) + @mail_info = mail_info + @redis = Redis.new('host' => REDIS_HOST, 'port' => REDIS_PORT) + end + + def check_email_mapping + email_to = @mail_info['to'].clone + email_cc = @mail_info['cc'].clone + email_bcc = @mail_info['bcc'].clone + @mail_info['to'] = email_mapping(email_to) + @mail_info['cc'] = email_mapping(email_cc) + @mail_info['bcc'] = email_mapping(email_bcc) + + return @mail_info + end + + def email_mapping(mail_list) + return if mail_list.nil? || mail_list.empty? + + mail_list.clone.each do |email| + next unless @redis.hexists 'email_mapping', email + + mapped_email = @redis.hget 'email_mapping', email + mail_list -= [email] + mail_list << mapped_email unless mail_list.include? mapped_email + end + return mail_list + end +end -- 2.23.0

1 0

[PATCH v4 compass-ci 3/7] send-internet-mail: add timing work
by Luan Shengde 07 Jan '21

07 Jan '21

timing work: reset email rate limit queues everyday Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/email_init.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100755 container/send-internet-mail/email_init.rb diff --git a/container/send-internet-mail/email_init.rb b/container/send-internet-mail/email_init.rb new file mode 100755 index 0000000..62f27ea --- /dev/null +++ b/container/send-internet-mail/email_init.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'rufus-scheduler' +require 'redis' + +redis = Redis.new(host: REDIS_HOST, port: REDIS_PORT) +email_init = Rufus::Scheduler.new + +# Timing work for email counting everyday +# the email counts queues will be reset everyday +# to ensure can send mail in the new day to user +email_init.cron '0 0 * * *' do + redis.del 'email_in_limit' + redis.del 'email_out_limit' +end -- 2.23.0

1 0

[PATCH v4 compass-ci 2/7] send-internet-mail: optimize
by Luan Shengde 07 Jan '21

07 Jan '21

add init redis host/port add invoke code files Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/send-internet-mail.rb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb index 8fdaa0d..7b357f8 100755 --- a/container/send-internet-mail/send-internet-mail.rb +++ b/container/send-internet-mail/send-internet-mail.rb @@ -6,7 +6,13 @@ require 'mail' require 'sinatra' -require "#{ENV['CCI_SRC']}/lib/mail-post" +REDIS_HOST = %x(/sbin/ip route | awk '/default/ {print $3}').chomp +REDIS_PORT = ENV['REDIS_PORT'] + +require_relative 'email_init' +require_relative 'email_limit_queue' +require_relative 'email_mapping' +require_relative '../../lib/mail-post' smtp = { address: 'smtp.qq.com', -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: delete unnecessary mkdir
by Li Yuanchao 07 Jan '21

07 Jan '21

Git clone will mkdir itself, so no need to mkdir before. And if FileUtils.mkdir_p fail, it will cause worker thread die, but 'git clone' will not. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index b153978..c7dd019 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -72,7 +72,6 @@ class GitMirror if File.directory?(mirror_dir) possible_new_refs = git_fetch(mirror_dir) else - FileUtils.mkdir_p(mirror_dir) possible_new_refs = git_clone(fork_info['url'], mirror_dir) end feedback(fork_info['git_repo'], possible_new_refs) -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: libvirt: a new boot type of libvirt
by Ren Wen 07 Jan '21

07 Jan '21

Requested by: curl -X GET 'http://<host>:<port>/boot.libvirt/mac/<mac_value>' Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index b2f0802..02cced6 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -7,7 +7,7 @@ class Sched boot_type = @env.params.url["boot_type"] case boot_type - when "ipxe" + when "ipxe", "libvirt" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) when "grub" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) @@ -135,6 +135,28 @@ class Sched return response.to_json end + private def get_boot_libvirt(job : Job) + response = {} of String => String | Nil | Array(String) | JSON::Any + + response["job_id"] = job.id + response["kernel_uri"] = job.kernel_uri.lchop("kernel ") + + initrds_uri = job["initrds_uri"]? + initrds_uri = initrds_uri.as_a.map(&.as_s.lchop("initrd ")) if initrds_uri + response["initrds_uri"] = initrds_uri + + kernel_params = job["kernel_params"]? + kernel_params = kernel_params.as_a.map(&.to_s).reject(&.starts_with?("initrd=")).join(" ") if kernel_params + response["kernel_params"] = kernel_params + + response["result_root"] = job.result_root + response["LKP_SERVER"] = job["LKP_SERVER"]? + # *templates* is used by libvirt + response["templates"] = job["templates"]? + + return response.to_json + end + private def get_boot_grub(job : Job) initrd_lkp_cgz = "lkp-#{job.os_arch}.cgz" @@ -176,6 +198,8 @@ class Sched case boot_type when "ipxe" return job ? get_boot_ipxe(job) : ipxe_msg("No job now") + when "libvirt" + return job ? get_boot_libvirt(job) : {"info" => "No job now"}.to_json when "grub" return job ? get_boot_grub(job) : grub_msg("No job now") when "container" -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] compare: remove not latest failed metrics
by Lu Weitao 07 Jan '21

07 Jan '21

[Why] User just need the failed metrics belong latest job in compare result [How] 1. query job_list and order by start_time desc 2. auto group job_list - the first job of each dimension is the latest job { group_key => { dim1 => [job1, ...] ... } } 3. get latest job_list for each group 4. in compare result, ignore the failed metric that not belongs the latest job_list Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare.rb | 4 ++-- lib/compare_matrixes.rb | 22 ++++++++++++++-------- lib/matrix2.rb | 24 +++++++++++++++++------- 3 files changed, 33 insertions(+), 17 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 6a2f515..21e24df 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -62,8 +62,8 @@ end def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') - groups_matrices, suites_hash = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices, suites_hash, options) + groups_matrices, suites_hash, latest_jobs_hash = create_groups_matrices_list(conditions, dims) + compare_group_matrices(groups_matrices, suites_hash, latest_jobs_hash, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 21bd5c9..fa578f6 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -159,7 +159,7 @@ def get_matrixes_fields(matrixes_list) matrixes_fields end -def get_matrixes_values(matrixes_list, options) +def get_matrixes_values(matrixes_list, latest_jobs, options) # get all matrixes all field values # matrixes_values = { false => {}, true => {} } @@ -168,6 +168,8 @@ def get_matrixes_values(matrixes_list, options) next if field == 'stats_source' success = successful?(field) + next unless success || latest_failure?(field, latest_jobs) + matrixes_values[success][field] = get_values_by_field( matrixes_list, field, matrixes_size, success, options @@ -176,6 +178,10 @@ def get_matrixes_values(matrixes_list, options) matrixes_values end +def latest_failure?(field, latest_jobs) + latest_jobs.any? { |job| job['stats'][field] } +end + def remove_unchanged_field(matrixes_values, suite_list) # remove unchanged field from matrixes values and remove :changed key # @@ -197,7 +203,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, suite_list, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suite_list, latest_jobs, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -206,7 +212,7 @@ def compare_matrixes(matrixes_list, suite_list, matrixes_titles = nil, group_key return warn('Matrix cannot be empty!') || '' if matrixes_empty?(matrixes_list) options = { 'perf-profile': 5, theme: :none, no_print: false }.merge(options) - matrixes_values = get_matrixes_values(matrixes_list, options) + matrixes_values = get_matrixes_values(matrixes_list, latest_jobs, options) remove_unchanged_field(matrixes_values, suite_list) if matrixes_list.length > 1 no_print = options[:no_print] result_str = group_key ? "\n\n\n\n\n" + group_key : '' @@ -387,28 +393,28 @@ end # input: group matrices # output: pre compare result of each group # the result with more comparison objects first -def compare_group_matrices(group_matrices, suites_hash, options) +def compare_group_matrices(group_matrices, suites_hash, latest_jobs_hash, options) result_str = '' group_matrices_array = sort_by_matrix_size(group_matrices) have_multi_member = group_matrices_array[0][1].size > 1 group_matrices_array.each do |matrice_kv| next if have_multi_member && matrice_kv[1].size < 2 - result_str += get_matrix_str(matrice_kv[0], matrice_kv[1], suites_hash[matrice_kv[0]], options) + result_str += get_matrix_str(matrice_kv[0], matrice_kv[1], suites_hash[matrice_kv[0]], latest_jobs_hash[matrice_kv[0]], options) end result_str end -def get_matrix_str(matrice_key, matrice_value, suite_list, options) +def get_matrix_str(matrice_key, matrice_value, suite_list, latest_jobs, options) m_list = [] m_titles = [] matrice_value.each do |dim, matrix| m_titles << dim m_list << matrix end - return compare_matrixes(m_list, suite_list, m_titles, matrice_key, options: options) if options[:no_print] + return compare_matrixes(m_list, suite_list, latest_jobs, m_titles, matrice_key, options: options) if options[:no_print] - print compare_matrixes(m_list, suite_list, m_titles, matrice_key, options: options) + print compare_matrixes(m_list, suite_list, latest_jobs, m_titles, matrice_key, options: options) return '' end diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 09573ec..e603763 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -115,27 +115,37 @@ end # } def combine_group_query_data(job_list, dims) suites_hash = {} + latest_jobs_hash = {} groups = auto_group(job_list, dims) have_multi_member = multi_member?(groups) one_size_count = 0 groups.each do |group_key, value| if value.size < 2 one_size_count += 1 - if have_multi_member || one_size_count > 3 - groups.delete(group_key) - next - end + next if group_deleted?(groups, group_key, have_multi_member, one_size_count) end - get_groups_matrix(groups, group_key, value, suites_hash) + get_groups_matrix(groups, group_key, value, suites_hash, latest_jobs_hash) end - return groups, suites_hash + return groups, suites_hash, latest_jobs_hash end -def get_groups_matrix(groups, group_key, value, suites_hash) +def group_deleted?(groups, group_key, have_multi_member, one_size_count) + was_deleted = false + if have_multi_member || one_size_count > 3 + groups.delete(group_key) + was_deleted = true + end + + was_deleted +end + +def get_groups_matrix(groups, group_key, value, suites_hash, latest_jobs_hash) suite_list = [] + latest_jobs_hash[group_key] = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) + latest_jobs_hash[group_key] << jobs[0] end suites_hash[group_key] = suite_list end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/compare.rb: order query result by start_time with desc
by Lu Weitao 07 Jan '21

07 Jan '21

[Why] 1. When query result include more than 10,000 jobs, we still need the latest jobs use to compare 2. Support we recognize the latest job that have failure messages from each group-dimension-job_list Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 9090c3b..6a2f515 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -40,7 +40,7 @@ def create_matrices_list(conditions) suite_list = [] es = ESQuery.new(ES_HOST, ES_PORT) conditions.each do |condition| - query_results = es.multi_field_query(condition) + query_results = es.multi_field_query(condition, desc_keyword: 'start_time') matrix, suites = combine_query_data(query_results) matrices_list << matrix suite_list.concat(suites) @@ -68,7 +68,7 @@ end def create_groups_matrices_list(conditions, dims) es = ESQuery.new(ES_HOST, ES_PORT) - query_results = es.multi_field_query(conditions) + query_results = es.multi_field_query(conditions, desc_keyword: 'start_time') combine_group_query_data(query_results['hits']['hits'], dims) end -- 2.23.0

1 0

[PATCH compass-ci] service/monitoring: add license
by Wu Zhende 07 Jan '21

07 Jan '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/monitoring/amqp.cr | 3 +++ src/monitoring/filter.cr | 3 +++ src/monitoring/monitoring.cr | 3 +++ src/monitoring/parse_serial_logs.cr | 3 +++ 4 files changed, 12 insertions(+) diff --git a/src/monitoring/amqp.cr b/src/monitoring/amqp.cr index 27ad3d2..f23113a 100644 --- a/src/monitoring/amqp.cr +++ b/src/monitoring/amqp.cr @@ -1,3 +1,6 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + require "amqp-client" require "./monitoring" diff --git a/src/monitoring/filter.cr b/src/monitoring/filter.cr index 2bd1d1a..39cfbb6 100644 --- a/src/monitoring/filter.cr +++ b/src/monitoring/filter.cr @@ -1,3 +1,6 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + require "set" require "json" diff --git a/src/monitoring/monitoring.cr b/src/monitoring/monitoring.cr index 8d52d46..046302b 100644 --- a/src/monitoring/monitoring.cr +++ b/src/monitoring/monitoring.cr @@ -1,3 +1,6 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + require "kemal" require "json" diff --git a/src/monitoring/parse_serial_logs.cr b/src/monitoring/parse_serial_logs.cr index 6c8a220..5a54849 100644 --- a/src/monitoring/parse_serial_logs.cr +++ b/src/monitoring/parse_serial_logs.cr @@ -1,3 +1,6 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + require "../scheduler/elasticsearch_client" require "set" require "json" -- 2.23.0

1 0

[PATCH v3 compass-ci] providers/docker: copy rootfs/addon/root/. to container
by Xiao Shenwei 07 Jan '21

07 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 4388e19..9848b4f 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -2,7 +2,6 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -mkdir -p /root/.ssh -cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ +cp -r /lkp/lkp/src/rootfs/addon/root/. /root cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap -- 2.23.0

3 3

[PATCH lkp-tests] pkg/turbostat: changed turbostat's source url
by Zhang Yale 07 Jan '21

07 Jan '21

turbostat can not be found in x86_64 Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- pkg/turbostat/PKGBUILD | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkg/turbostat/PKGBUILD b/pkg/turbostat/PKGBUILD index d7e536c7..24adcffe 100644 --- a/pkg/turbostat/PKGBUILD +++ b/pkg/turbostat/PKGBUILD @@ -4,7 +4,7 @@ pkgrel=1 url='https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git' arch=('i386' 'x86_64') license=('GPL') -source=('https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git') +source=("git://$GIT_SERVER/l/linux/linux.git") md5sums=('SKIP') build() -- 2.23.0

2 2

[PATCH lkp-tests] pkg/api-avx2neon: fix x86 version build
by Wang Yong 07 Jan '21

07 Jan '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- pkg/api-avx2neon/PKGBUILD | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/pkg/api-avx2neon/PKGBUILD b/pkg/api-avx2neon/PKGBUILD index fac047add..30967beb7 100644 --- a/pkg/api-avx2neon/PKGBUILD +++ b/pkg/api-avx2neon/PKGBUILD @@ -1,3 +1,5 @@ +# add pkg_arch: x86_64 to build x86 version +# ref: Makefile pkgname=api-avx2neon pkgver=1 pkgrel=1 @@ -9,7 +11,12 @@ md5sums=('SKIP') build() { cd "$srcdir/AvxToNeon/tests" - make ARCH=$(arch) + + if [ "${pkg_arch}" == "aarch64" ]; then + make + elif [ "${pkg_arch}" == "x86_64" ]; then + make ARCH=x86 + fi } package() { -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: fix that delete unused variable
by Li Yuanchao 06 Jan '21

06 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index bd55b33..b153978 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -374,6 +374,6 @@ class MirrorMain submodule = %x(git -C #{mirror_dir} show HEAD:.gitmodules 2>/dev/null) return if submodule.empty? - handle_submodule(submodule, File.basename(git_repo)) + handle_submodule(submodule) end end -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix the name of *.cgz
by Liu Shaofei 06 Jan '21

06 Jan '21

[why] the name of *.cgz packages append the field of 'pkgrel'. The example is as follow: /srv/initrd/build-pkg/nfs/openeuler/aarch64/20.03-SP1/log4cpp/1.1.3-2.cgz Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 436d43840..98341e7ac 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -95,11 +95,11 @@ create_cgz_name() pkg_args="-A --check --skippgpcheck" [ -n "$git_tag" ] && { - cgz_name="${cgz_path}/${git_tag}.cgz" + cgz_name="${cgz_path}/${git_tag}-${pkgrel}.cgz" } [ -n "$pkg_ver" ] && { - cgz_name="${cgz_path}/${pkg_ver}.cgz" + cgz_name="${cgz_path}/${pkg_ver}-${pkgrel}.cgz" pkg_args="${pkg_args} --skipchecksums --skipinteg" } } -- 2.23.0

1 1

[PATCH v2 lkp-tests 2/2] tests/build-pkg: fix the name of *.cgz
by Liu Shaofei 06 Jan '21

06 Jan '21

[why] the name of *.cgz packages append the field of 'pkgrel'. The example is as follow: /srv/initrd/build-pkg/nfs/openeuler/aarch64/20.03-SP1/log4cpp/1.1.3-2.cgz Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 436d43840..98341e7ac 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -95,11 +95,11 @@ create_cgz_name() pkg_args="-A --check --skippgpcheck" [ -n "$git_tag" ] && { - cgz_name="${cgz_path}/${git_tag}.cgz" + cgz_name="${cgz_path}/${git_tag}-${pkgrel}.cgz" } [ -n "$pkg_ver" ] && { - cgz_name="${cgz_path}/${pkg_ver}.cgz" + cgz_name="${cgz_path}/${pkg_ver}-${pkgrel}.cgz" pkg_args="${pkg_args} --skipchecksums --skipinteg" } } -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] tests/cci-makepkg: remove useless variable
by Liu Shaofei 06 Jan '21

06 Jan '21

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/cci-makepkg | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index d48e8495e..e737955b9 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -108,7 +108,6 @@ distro_install_depends ${benchmark}-dev build_depends_pkg -i $benchmark $TMP sync_dest="$PKG_MNT/$pack_to" -date=$(date +"%Y%m%d") pkgver=$(get_pkg_info pkgver) pkgrel=$(get_pkg_info pkgrel) bm_name=$(check_shared_pkg) -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: remove useless variable
by Liu Shaofei 06 Jan '21

06 Jan '21

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/cci-makepkg | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index d48e8495e..e737955b9 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -108,7 +108,6 @@ distro_install_depends ${benchmark}-dev build_depends_pkg -i $benchmark $TMP sync_dest="$PKG_MNT/$pack_to" -date=$(date +"%Y%m%d") pkgver=$(get_pkg_info pkgver) pkgrel=$(get_pkg_info pkgrel) bm_name=$(check_shared_pkg) -- 2.23.0

1 1

[PATCH lkp-tests 2/2] jobs: add jobs/fault_reproduction.yaml
by Wei Jihui 06 Jan '21

06 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- jobs/fault_reproduction.yaml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 jobs/fault_reproduction.yaml diff --git a/jobs/fault_reproduction.yaml b/jobs/fault_reproduction.yaml new file mode 100644 index 00000000..46a1e042 --- /dev/null +++ b/jobs/fault_reproduction.yaml @@ -0,0 +1,14 @@ +ssh_pub_key: + <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> +sshd: +sleep: 6h + +on_fail: + sshd: + sleep: 6h -- 2.23.0

1 0

[PATCH compass-ci] feat: scheduler: add new boot way
by Ren Wen 06 Jan '21

06 Jan '21

A new way to find job, requested by: curl -X GET 'http://<host>:<port>/boot.libvirt/mac/<mac_value>' Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index b2f0802..f6981e7 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -7,7 +7,7 @@ class Sched boot_type = @env.params.url["boot_type"] case boot_type - when "ipxe" + when "ipxe", "libvirt" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) when "grub" host = @redis.hash_get("sched/mac2host", normalize_mac(value)) @@ -135,6 +135,29 @@ class Sched return response.to_json end + private def get_boot_libvirt(job : Job) + response = {} of String => String | Nil | Array(String) | Hash(String, String) + + response["job_id"] = job.id + response["kernel_uri"] = job.kernel_uri.lchop("kernel ") + + initrds_uri = job["initrds_uri"]? + initrds_uri = initrds_uri.as_a.map(&.as_s.lchop("initrd ")) if initrds_uri + response["initrds_uri"] = initrds_uri + + kernel_params = job["kernel_params"]? + kernel_params = kernel_params.as_a.map(&.to_s).reject(&.starts_with?("initrd=")).join(" ") if kernel_params + response["kernel_params"] = kernel_params + + response["result_root"] = job.result_root + response["LKP_SERVER"] = "#{job["LKP_SERVER"]?}" + + # Not implement now + response["templates"] = {} of String => String + + return response.to_json + end + private def get_boot_grub(job : Job) initrd_lkp_cgz = "lkp-#{job.os_arch}.cgz" @@ -176,6 +199,8 @@ class Sched case boot_type when "ipxe" return job ? get_boot_ipxe(job) : ipxe_msg("No job now") + when "libvirt" + return job ? get_boot_libvirt(job) : {"info" => "No job now"}.to_json when "grub" return job ? get_boot_grub(job) : grub_msg("No job now") when "container" -- 2.23.0

1 1

[PATCH lkp-tests] lib/detect-system.sh: add system arch riscv64
by Liu Yinsi 06 Jan '21

06 Jan '21

[why] when run job in riscv64 qemu, _system_arch=unknown, it cause qemu quit, so add it. test: /srv/result/iperf/2021-01-06/vm-8p64g/debian-sid-riscv64/tcp-30/z9.281254 Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/detect-system.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/detect-system.sh b/lib/detect-system.sh index 90006c8f0..1b344b4f0 100755 --- a/lib/detect-system.sh +++ b/lib/detect-system.sh @@ -31,6 +31,8 @@ parse_executable_arch() _system_arch=aarch64;; *ARM) _system_arch=arm;; + *RISC-V) + _system_arch=riscv64;; *) _system_arch=unknown return 1;; -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: add wget in initramfs-common
by Zhang Yale 06 Jan '21

06 Jan '21

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- distro/depends/initramfs-common | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/initramfs-common b/distro/depends/initramfs-common index 5d3ecb76..5f766da6 100644 --- a/distro/depends/initramfs-common +++ b/distro/depends/initramfs-common @@ -1,3 +1,4 @@ time +wget perf which -- 2.23.0

1 0

[PATCH lkp-tests] jobs: change testbox and add queue filed in bisect.yaml
by Cao Xueliang 06 Jan '21

06 Jan '21

testbox: dc-1g-bisect => dc-1g queue: dc-1g-bisect Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 8c2ee8f62..172554a30 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -1,4 +1,5 @@ suite: bisect docker_image: delimiter:latest -testbox: dc-1g-bisect +testbox: dc-1g +queue: dc-1g-bisect bisect: -- 2.23.0

1 0

[PATCH lkp-tests] jobs: dc-1g => dc-1g-bisect in bisect.yaml
by Cao Xueliang 06 Jan '21

06 Jan '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 11ffc7c05..8c2ee8f62 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -1,4 +1,4 @@ suite: bisect docker_image: delimiter:latest -testbox: dc-1g +testbox: dc-1g-bisect bisect: -- 2.23.0

2 3

[PATCH compass-ci] lib/job.cr: set docker job os
by Wu Zhende 06 Jan '21

06 Jan '21

docker job's os/os_version/os_mount don't need to define separately. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 2554db1..6f344fe 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -153,6 +153,7 @@ class Job private def set_defaults append_init_field() + set_docker_os() set_user_lkp_src() set_os_dir() set_submit_date() @@ -170,6 +171,14 @@ class Job set_subqueue() end + private def set_docker_os + return unless is_docker_job? + + os_info = docker_image.split(":") + self["os"] = os_info[0] + self["os_version"] = os_info[1] + end + private def set_kernel return if os_mount == "container" @@ -309,10 +318,23 @@ class Job @hash[key] = JSON::Any.new(value) if value end + private def is_docker_job? + if testbox =~ /^dc/ + return true + else + return false + end + end + # defaults to the 1st value VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container"] private def set_os_mount + if is_docker_job? + self["os_mount"] = "container" + return + end + if @hash["os_mount"]? if !VALID_OS_MOUNTS.includes?(@hash["os_mount"].to_s) raise "Invalid os_mount: #{@hash["os_mount"]}, should be in #{VALID_OS_MOUNTS}" -- 2.23.0

1 0

[PATCH v4 lkp-tests] tests: add httpress test script
by Wei Jihui 06 Jan '21

06 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/httpress | 136 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100755 tests/httpress diff --git a/tests/httpress b/tests/httpress new file mode 100755 index 00000000..462a2705 --- /dev/null +++ b/tests/httpress @@ -0,0 +1,136 @@ +#!/bin/sh +# - nic +# - nic_ip +# - request_ip +# - request_port +# - request_index +# - core_nums +# - protocol +# - alive_type + +. $LKP_SRC/lib/env.sh + +: "${request_index:=index.html}" +: "${core_nums:=4}" +: "${protocol:=http}" +: "${alive_type:=short}" + +core_range=0-15 + +http_long_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=50000000 + threads_num=1200 + concurrent_num=1200 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=50000000 + threads_num=400 + concurrent_num=400 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + +http_short_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=400 + concurrent_num=400 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=10000000 + threads_num=300 + concurrent_num=300 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +https_long_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=5000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + +https_short_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=500000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=500000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=1000000 + threads_num=1200 + concurrent_num=1200 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +main() +{ + local request_url="$protocol://$request_ip:$request_port/$request_index" + + ip addr add $nic_ip/24 dev "$nic" + ip link set "$nic" up + + if [ "$protocol" == 'http' ]; then + if [ "$alive_type" == 'long' ]; then + http_long_requests + else + http_short_requests + fi + elif [ "$protocol" == 'https' ]; then + if [ "$alive_type" == 'long' ]; then + https_long_requests + else + https_short_requests + fi + fi +} + +main -- 2.23.0

2 1

[PATCH v5 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 06 Jan '21

06 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 71 ++++++++++++------------------------------------- 1 file changed, 17 insertions(+), 54 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index bf72630..f7ca26e 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -150,61 +150,24 @@ job yaml ��[YAML](http://yaml.org/YAM #### �� - 1. �� os �� rootfs�� openEuler �� - - �� docker �� rootfs - - 1) �� openEuler �� docker �� - - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` - - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… - ``` - 2) ��{compass-ci}/container/qcow2rootfs ��rootfs - ```bash - cd {compass-ci}/container/qcow2rootfs - ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs - ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) - ```bash - chroot openEuler-rootfs - ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 1. �� openEuler �� + ```bash + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/20.03-LTS/ + ./run + ``` + + 2. ��/�� + �� ${CCI_SRC} ��rootfs�� rootfs ��/�� + ```bash + mkdir rootfs + cd rootfs + zcat ../$os_name.cgz | cpio -idmv + find ./lib/modules/* | cpio -o -Hnewc | gzip -9 > $HOME/modules-$os_name.cgz + find ./usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > $HOME/headers-$os_name.cgz + ``` + #### FAQ + 1. �� Unable to mount root fs on unknown-block�� - �� ```bash -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/docker: copy rootfs/addon/root/. to container
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 4388e19..b28b24c 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -3,6 +3,6 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. mkdir -p /root/.ssh -cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ +cp -r /lkp/lkp/src/rootfs/addon/root/. /root cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap -- 2.23.0

1 0

[PATCH compass-ci] providers/docker: copy rootfs/addon/root/. to container
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 2 +- providers/docker/run.sh | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 4388e19..b28b24c 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -3,6 +3,6 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. mkdir -p /root/.ssh -cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ +cp -r /lkp/lkp/src/rootfs/addon/root/. /root cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 62a1f9f..57a01c0 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -22,18 +22,19 @@ busybox_path=$(command -v busybox) cmd=( docker run --rm + --hostname $host -m $memory --tmpfs /tmp:rw,exec,nosuid,nodev -e CCI_SRC=/c/compass-ci -v ${load_path}/lkp:/lkp - -v ${DIR}/bin:/root/bin:ro + -v ${DIR}/bin:/root/sbin:ro -v $CCI_SRC:/c/compass-ci:ro -v /srv/git:/srv/git:ro -v /srv/result:/srv/result:ro -v ${busybox_path}:/usr/local/bin/busybox --oom-score-adj="-1000" ${docker_image} - /root/bin/entrypoint.sh + /root/sbin/entrypoint.sh ) "${cmd[@]}" -- 2.23.0

1 1

[PATCH compass-ci] mail: rename Compass-ci to Compass-CI
by Wu Fengguang 06 Jan '21

06 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- lib/mail_job_result.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/mail_job_result.rb b/lib/mail_job_result.rb index 50e2f53..796a4f3 100644 --- a/lib/mail_job_result.rb +++ b/lib/mail_job_result.rb @@ -22,8 +22,8 @@ class MailJobResult def compose_mail set_submitter_info - subject = "[Compass-ci] job: #{@job_id} result" - signature = "Regards\nCompass-ci\nhttps://gitee.com/openeuler/compass-ci" + subject = "[Compass-CI] job: #{@job_id} result" + signature = "Regards\nCompass-CI\nhttps://gitee.com/openeuler/compass-ci" body = "Hi, Thanks for your participation in Kunpeng and software ecosystem! Your Job: #{@job_id} had finished. -- 2.23.0

1 0

[PATCH compass-ci] sparrow: add setup.yaml
by Liu Yinsi 06 Jan '21

06 Jan '21

[why] before deploy compass-ci, fill configuration setup.yaml firstly, will make the whole process cleaner. fill my_name and my_email, it's necessary to verify account when submit job. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..b1cac8c --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,11 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. +# mkdir -p /etc/compass-ci/ +# cp setup.yaml /etc/compass-ci/setup.yaml +# vim /etc/compass-ci/setup.yaml + +# account name +my_name: xxxxxx + +# email address +my_email: xxx@xxx -- 2.23.0

3 4

[PATCH compass-ci] doc: update doc
by Luan Shengde 06 Jan '21

06 Jan '21

rename email address compass-ci-reply(a)qq.com => compass-ci-robot(a)qq.com Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/mail-robot.md | 2 +- doc/manual/apply-account.md | 2 +- doc/manual/borrow-machine.en.md | 2 +- doc/manual/borrow-machine.zh.md | 2 +- ...50\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index e97912c..6158b65 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -11,7 +11,7 @@ apply account/uuid for user ## usage build email as the following section: allowed email format -send mail to: compass-ci-reply(a)qq.com +send mail to: compass-ci-robot(a)qq.com ## allowed email format: diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 08e7618..a0b2c82 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -6,7 +6,7 @@ Email template: --- From: {{ David Rientjes <rientjes(a)google.com> }} - To: compass-ci-reply(a)qq.com + To: compass-ci-robot(a)qq.com Subject: apply account my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index bed9fcc..1b4a220 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -3,7 +3,7 @@ Ensure that you have performed the following operations according to the [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc…: - Send an email to apply for an account. -- Receive an email from compass-ci-reply(a)qq.com. +- Receive an email from compass-ci-robot(a)qq.com. - Configure the local environment. # Applying for a Test Machine (VM) diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 3f500e2..bb1c08f 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -2,7 +2,7 @@ 请确认您已按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… - send apply account email. -- receive email from compass-ci-reply(a)qq.com. +- receive email from compass-ci-robot(a)qq.com. - local environment configuration. diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 6f0736b..ad933b1 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -6,7 +6,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 :exclamation: 前提条件：按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… 完成如下动作： - send apply account email -- receive email from compass-ci-reply(a)qq.com +- receive email from compass-ci-robot(a)qq.com - get reply email and follow its instructions to - setup default config -- 2.23.0

1 0

[PATCH compass-ci] providers/docker: change the mount path
by Xiao Shenwei 06 Jan '21

06 Jan '21

[why] /root/bin the dir may be used by user to stores own file --- providers/docker/run.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 62a1f9f..57a01c0 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -22,18 +22,19 @@ busybox_path=$(command -v busybox) cmd=( docker run --rm + --hostname $host -m $memory --tmpfs /tmp:rw,exec,nosuid,nodev -e CCI_SRC=/c/compass-ci -v ${load_path}/lkp:/lkp - -v ${DIR}/bin:/root/bin:ro + -v ${DIR}/bin:/root/sbin:ro -v $CCI_SRC:/c/compass-ci:ro -v /srv/git:/srv/git:ro -v /srv/result:/srv/result:ro -v ${busybox_path}:/usr/local/bin/busybox --oom-score-adj="-1000" ${docker_image} - /root/bin/entrypoint.sh + /root/sbin/entrypoint.sh ) "${cmd[@]}" -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] sbin/build-my-info.rb: modify method of inputting variables
by Hu Xuejiao 06 Jan '21

06 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/build-my-info.rb | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 40193af..2562062 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -3,16 +3,26 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -require 'io/console' require_relative '../lib/es_client' require_relative '../lib/build_my_info_client' +require 'optparse' -print 'email: ' -my_email = $stdin.echo = gets.chomp -print 'name: ' -my_name = $stdin.echo = gets.chomp -print 'lab: ' -lab = $stdin.echo = gets.chomp +option = {} +options= OptionParser.new do |opts| + opts.on('-e email', 'my_email') do |email| + option[:email] = email + end -build_my_info = BuildMyInfo.new(my_email, my_name, lab) + opts.on('-n name', 'my_name') do |name| + option[:name] = name + end + + opts.on('-l lab', 'lab') do |lab| + option[:lab] = lab + end +end + +options.parse! + +build_my_info = BuildMyInfo.new(option[:my_email],option[:my_name],option[:lab]) build_my_info.config_my_info -- 2.23.0

3 3

[PATCH lkp-tests] sshd: improve notification email
by Wu Fengguang 06 Jan '21

06 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- daemon/sshd | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index c2404dbfd..e9f380060 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -39,13 +39,11 @@ run_ssh() data_success() { data="To: $my_email -Subject: [NOTIFY Compass-ci] Machine application successful +Subject: [NOTIFY Compass-CI] Machine application successful -Dear $my_username: +Dear $my_username, - Thanks for your participation in software ecosystem! - According to your application, the test machine has been provisioned. - The datails are as follows: + Your test machine has been provisioned. Login: ssh root(a)api.compass-ci.openeuler.org -p $port @@ -59,7 +57,7 @@ Dear $my_username: $os $os_version $os_arch Regards -Compass-Ci +Compass-CI " } -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests: add httpress test script
by Wei Jihui 06 Jan '21

06 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/httpress | 138 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100755 tests/httpress diff --git a/tests/httpress b/tests/httpress new file mode 100755 index 00000000..f8b0e4d5 --- /dev/null +++ b/tests/httpress @@ -0,0 +1,138 @@ +#!/bin/sh +# - nic +# - nic_ip +# - request_ip +# - request_port +# - request_index +# - core_nums +# - protocol +# - alive_type + +. $LKP_SRC/lib/env.sh + +: "${request_index:=index.html}" +: "${core_nums:=4}" +: "${protocol:=http}" +: "${alive_type:=short}" + + +core_range=0-15 + +http_long_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=50000000 + threads_num=1200 + concurrent_num=1200 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=50000000 + threads_num=400 + concurrent_num=400 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + + +http_short_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=400 + concurrent_num=400 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=10000000 + threads_num=300 + concurrent_num=300 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +https_long_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=5000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + +https_short_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=500000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=500000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=1000000 + threads_num=1200 + concurrent_num=1200 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +main() +{ + local request_url="$protocol://$request_ip:$request_port/$request_index" + + ip addr add $nic_ip/24 dev "$nic" + ip link set "$nic" up + + if [ "$protocol" == 'http' ]; then + if [ "$alive_type" == 'long' ]; then + http_long_requests + else + http_short_requests + fi + elif [ "$protocol" == 'https' ]; then + if [ "$alive_type" == 'long' ]; then + https_long_requests + else + https_short_requests + fi + fi +} + +main -- 2.23.0

4 3

[PATCH v3 compass-ci] sbin/build-my-info.rb: modify method of inputting variables
by Hu Xuejiao 06 Jan '21

06 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/build-my-info.rb | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 40193af..cd824c6 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -3,16 +3,26 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -require 'io/console' require_relative '../lib/es_client' require_relative '../lib/build_my_info_client' +require 'optparse' -print 'email: ' -my_email = $stdin.echo = gets.chomp -print 'name: ' -my_name = $stdin.echo = gets.chomp -print 'lab: ' -lab = $stdin.echo = gets.chomp +option = {} +options= OptionParser.new do |opts| + opts.on('-e email', 'my_email') do |email| + option[:email] = email + end -build_my_info = BuildMyInfo.new(my_email, my_name, lab) + opts.on('-n name', 'my_name') do |name| + option[:name] = name + end + + opts.on('-l lab', 'lab') do |lab| + option[:lab] = lab + end +end + +options.parse! + +build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) build_my_info.config_my_info -- 2.23.0

1 0

[PATCH lkp-tests] installer/debian: fix the exit problem when source this file
by Yu Chuan 06 Jan '21

06 Jan '21

[Why] The purpose of adding the last four lines is to remove useless warn logs, but it exits at the end. This will cause the process which source this script exit. [Addition] Related commit id: 16cef539c517d0e6bd3495a1e3079214a8b4bd24 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- distro/installer/debian | 4 ---- 1 file changed, 4 deletions(-) diff --git a/distro/installer/debian b/distro/installer/debian index 3148ee3b2c2a..35b3209bd747 100755 --- a/distro/installer/debian +++ b/distro/installer/debian @@ -6,7 +6,3 @@ dpkg --add-architecture i386 && apt-get update apt-get -o Dpkg::Options::="--force-confdef" \ -o Dpkg::Options::="--force-confold" \ -y install $* >/tmp/apt-get_info 2>&1 - -ret_value=$? -grep -v "dpkg: warning: files list file for package '.*' missing;" /tmp/apt-get_info -exit $ret_value -- 2.23.0

1 0

[PATCH compass-ci 2/3] providers/docker: change the dir name bin to sbin
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/{ => sbin}/bin/entrypoint.sh | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename providers/docker/{ => sbin}/bin/entrypoint.sh (100%) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/sbin/bin/entrypoint.sh similarity index 100% rename from providers/docker/bin/entrypoint.sh rename to providers/docker/sbin/bin/entrypoint.sh -- 2.23.0

1 1

[PATCH compass-ci] addsoftware: clean the runtime dir after modifying nfsroot
by Xu Xijian 06 Jan '21

06 Jan '21

When install a software into a nfsroot, some unwanted files will be added into runtime dir, so rm them to keep nfsroot clean. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/add-software/run | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/container/add-software/run b/container/add-software/run index 62d4eff..60e5e07 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -40,7 +40,11 @@ install_cmds=( --rm -v "$rootdir":/mnt/root addsoftware:auto - yum install -y --skip-broken --installroot=/mnt/root $packages + bash -c " + yum install -y --skip-broken --installroot=/mnt/root $packages + cd /mnt/root + rm -rf srv/* opt/* mnt/* media/* home/* lost+found/* dev/* proc/* run/* sys/* + " ) echo "rsync rootfs from $basever to $newver..." -- 2.23.0

1 0

[PATCH compass-ci 3/3] providers/docker: copy rootfs/addon/root/* to container
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/sbin/entrypoint.sh | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100755 providers/docker/sbin/entrypoint.sh diff --git a/providers/docker/sbin/entrypoint.sh b/providers/docker/sbin/entrypoint.sh new file mode 100755 index 0000000..9848b4f --- /dev/null +++ b/providers/docker/sbin/entrypoint.sh @@ -0,0 +1,7 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +cp -r /lkp/lkp/src/rootfs/addon/root/. /root +cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr +sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap -- 2.23.0

1 0

[PATCH compass-ci 1/3] providers/docker: change the file path of entrypoint.sh
by Xiao Shenwei 06 Jan '21

06 Jan '21

[why] /root/bin the dir may be used by user to stores own file Thanks, Shenwei --- providers/docker/run.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 62a1f9f..d39b6cf 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -23,17 +23,18 @@ cmd=( docker run --rm -m $memory + --hostname $host --tmpfs /tmp:rw,exec,nosuid,nodev -e CCI_SRC=/c/compass-ci -v ${load_path}/lkp:/lkp - -v ${DIR}/bin:/root/bin:ro + -v ${DIR}/sbin:/root/sbin:ro -v $CCI_SRC:/c/compass-ci:ro -v /srv/git:/srv/git:ro -v /srv/result:/srv/result:ro -v ${busybox_path}:/usr/local/bin/busybox --oom-score-adj="-1000" ${docker_image} - /root/bin/entrypoint.sh + /root/sbin/entrypoint.sh ) "${cmd[@]}" -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: fix check key
by Wei Jihui 06 Jan '21

06 Jan '21

first check key starts with "#!", then check key matchs /^\w.*\|.*\w$/ Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 35e0ceee..8346db87 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -288,7 +288,6 @@ class Job def multi_args(hash) jobs_array = [] hash.each { |key, value| - next unless key =~ /^\w.*\|.*\w$/ next unless key.is_a?(String) if key.start_with?('#! ') @@ -296,6 +295,8 @@ class Job next end + next unless key =~ /^\w.*\|.*\w$/ + key_array = key.split('|').map(&:strip) [value].flatten.each do |v| next unless v =~ /^\w.*\|.*\w$/ -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: fix that project name start with capital letter can't match
by Li Yuanchao 06 Jan '21

06 Jan '21

Some repositories' name start with capital letter, but upstream-repos all start with lower case letter. So such repository can't match git_repo when use webhook container. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index ce1dc9e..bd55b33 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -333,7 +333,7 @@ class MirrorMain git_repo = "#{project}/#{fork_name}" return git_repo if check_git_repo(git_repo, webhook_url) - git_repo = "#{project[0]}/#{project}/#{project}" + git_repo = "#{project[0].downcase}/#{project}/#{project}" return git_repo if check_git_repo(git_repo, webhook_url) puts "webhook: #{webhook_url} is not found!" -- 2.23.0

1 0

[PATCH compass-ci] sparrow: make gem install system wide
by Wu Fengguang 06 Jan '21

06 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- sparrow/0-package/common | 2 +- sparrow/5-build/overcommit | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 57ab701..4a9da60 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -2,7 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -gem install git activesupport rest-client cucumber json faye-websocket elasticsearch +gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch grep -q '^lkp:' /etc/passwd || useradd -u 1090 lkp grep -q '^mailer:' /etc/passwd || useradd -u 1144 mailer diff --git a/sparrow/5-build/overcommit b/sparrow/5-build/overcommit index e8add6f..af02d84 100755 --- a/sparrow/5-build/overcommit +++ b/sparrow/5-build/overcommit @@ -10,4 +10,4 @@ cd overcommit oc_build_out=$(gem build overcommit.gemspec | grep "overcommit-.*\.gem") -gem install "$oc_build_out" +gem install --no-user-install "$oc_build_out" -- 2.23.0

1 0

[PATCH v2 compass-ci] git_mirror.rb: fix rubocop warning
by Li Yuanchao 06 Jan '21

06 Jan '21

git_mirror.rb:352:35: W: Lint/UnusedMethodArgument: Unused method argument - parent_project. If it's necessary, use _ or _parent_project as an argument name to indicate that it won't be used. def handle_submodule(submodule, parent_project) ^^^^^^^^^^^^^^ git_mirror.rb:358:7: W: Lint/NonLocalExitFromIterator: Non-local exit from iterator, without return value. next, break, Array#find, Array#any?, etc. is preferred. return unless git_repo ^^^^^^ Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 7f265d9..ce1dc9e 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -349,13 +349,13 @@ class MirrorMain end end - def handle_submodule(submodule, parent_project) + def handle_submodule(submodule) submodule.each_line do |line| next unless line.include?('url = ') url = line.split(' = ')[1].chomp git_repo = url.split('://')[1] if url.include?('://') - return unless git_repo + break unless git_repo @git_info[git_repo] = { 'url' => url, 'git_repo' => git_repo, 'is_submodule' => true } fork_stat_init(git_repo) -- 2.23.0

1 0

[PATCH lkp-tests] tests/host-info: computing vm memory
by Wei Jihui 06 Jan '21

06 Jan '21

[why] it can not compute vm memory when vm memory less than 16G. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/host-info | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index fe55d1ff..2721648c 100755 --- a/tests/host-info +++ b/tests/host-info @@ -4,8 +4,8 @@ get_memory() { memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') - memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') + memory=$(echo $memory_new | awk '{print int($0)+1}')G echo "memory: $memory" } -- 2.23.0

3 2

[PATCH v4 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 06 Jan '21

06 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 71 ++++++++++++------------------------------------- 1 file changed, 17 insertions(+), 54 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index bf72630..bab40bc 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -150,61 +150,24 @@ job yaml ��[YAML](http://yaml.org/YAM #### �� - 1. �� os �� rootfs�� openEuler �� - - �� docker �� rootfs - - 1) �� openEuler �� docker �� - - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` - - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… - ``` - 2) ��{compass-ci}/container/qcow2rootfs ��rootfs - ```bash - cd {compass-ci}/container/qcow2rootfs - ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs - ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) - ```bash - chroot openEuler-rootfs - ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 1. �� openEuler �� + ```bash + cd {compass-ci}/rootfs/initramfs/openeuler/aarch64/20.03-LTS/ + ./run + ``` + + 2. ��/�� + �� {compass-ci} ��rootfs�� rootfs ��/�� + ```bash + mkdir rootfs + cd rootfs + zcat ../$os_name.cgz | cpio -idmv + find ./lib/modules/* | cpio -o -Hnewc | gzip -9 > $HOME/modules-$os_name.cgz + find ./usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > $HOME/headers-$os_name.cgz + ``` + #### FAQ + 1. �� Unable to mount root fs on unknown-block�� - �� ```bash -- 2.23.0

2 3

[PATCH lkp-tests 1/2] stats/crystal/kernbench.cr: create kernbench.cr as same content with stats/kernbench
by Lu Kaiyi 06 Jan '21

06 Jan '21

create kernbench.cr in crystal directory from stats/kernbench first, and then, the second patch will know what has changed from original file. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 stats/crystal/kernbench.cr diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr new file mode 100644 index 000000000..0da37fe88 --- /dev/null +++ b/stats/crystal/kernbench.cr @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby + +LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) + +require "#{LKP_SRC}/lib/statistics" + +$stdin.each_line do |line| + case line + when /Elapsed Time/ + puts 'elapsed_time: ' + line.split[2] + puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /User Time/ + puts 'user_time: ' + line.split[2] + puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /System Time/ + puts 'system_time: ' + line.split[2] + puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Percent CPU/ + puts 'percent_cpu: ' + line.split[2] + puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Context Switches/ + puts 'context_switches: ' + line.split[2] + puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + when /Sleeps/ + puts 'sleeps: ' + line.split[1] + puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + end +end -- 2.23.0

2 1

[PATCH compass-ci] git_mirror.rb: optimize method to pass rubocop
by Li Yuanchao 06 Jan '21

06 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 7f265d9..ce1dc9e 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -349,13 +349,13 @@ class MirrorMain end end - def handle_submodule(submodule, parent_project) + def handle_submodule(submodule) submodule.each_line do |line| next unless line.include?('url = ') url = line.split(' = ')[1].chomp git_repo = url.split('://')[1] if url.include?('://') - return unless git_repo + break unless git_repo @git_info[git_repo] = { 'url' => url, 'git_repo' => git_repo, 'is_submodule' => true } fork_stat_init(git_repo) -- 2.23.0

2 1

[PATCH lab-z9] hosts: add new testbox host info
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- hosts/taishan200-2280-2s64p-256g--a115 | 20 ++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a117 | 20 ++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a119 | 20 ++++++++++++++++++++ hosts/vm-2p32g | 6 ++++++ 4 files changed, 66 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a115 create mode 100644 hosts/taishan200-2280-2s64p-256g--a117 create mode 100644 hosts/taishan200-2280-2s64p-256g--a119 create mode 100644 hosts/vm-2p32g diff --git a/hosts/taishan200-2280-2s64p-256g--a115 b/hosts/taishan200-2280-2s64p-256g--a115 new file mode 100644 index 0000000..64e2b1a --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a115 @@ -0,0 +1,20 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399888177f5 +rootfs_partition: +- /dev/disk/by-id/scsi-35000039988817359 +mac_addr: + - 44:a1:91:9d:f3:44 + - 44:a1:91:9d:f3:45 + - 44:a1:91:9d:f3:46 + - 44:a1:91:9d:f3:47 + - 44:a1:91:e5:94:51 + - 44:a1:91:e5:94:52 + - 44:a1:91:e5:94:53 + - 44:a1:91:e5:94:54 +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.11.8 diff --git a/hosts/taishan200-2280-2s64p-256g--a117 b/hosts/taishan200-2280-2s64p-256g--a117 new file mode 100644 index 0000000..31167f9 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a117 @@ -0,0 +1,20 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886b57d1 +rootfs_partition: +- /dev/disk/by-id/scsi-350000399886b720d +mac_addr: + - 44:67:47:85:f4:0b + - 44:67:47:85:f4:0c + - 44:67:47:85:f4:0d + - 44:67:47:85:f4:0e + - cc:05:77:ee:b3:ff + - cc:05:77:ee:b4:00 + - cc:05:77:ee:b4:01 + - cc:05:77:ee:b4:02 +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.11.10 diff --git a/hosts/taishan200-2280-2s64p-256g--a119 b/hosts/taishan200-2280-2s64p-256g--a119 new file mode 100644 index 0000000..5775be3 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a119 @@ -0,0 +1,20 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886b5dbd +rootfs_partition: +- /dev/disk/by-id/scsi-350000399886b6be9 +mac_addr: + - 44:67:47:e9:79:65 + - 44:67:47:e9:79:66 + - 44:67:47:e9:79:67 + - 44:67:47:e9:79:68 + - 44:a1:91:9f:9b:4a + - 44:a1:91:9f:9b:4b + - 44:a1:91:9f:9b:4c + - 44:a1:91:9f:9b:4d +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.11.12 diff --git a/hosts/vm-2p32g b/hosts/vm-2p32g new file mode 100644 index 0000000..fcb23ed --- /dev/null +++ b/hosts/vm-2p32g @@ -0,0 +1,6 @@ +arch: aarch64 +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 32G -- 2.23.0

1 0

[PATCH lkp-tests 2/2] stats/crystal/kernbench.cr: kernbench to kernbench.cr
by Lu Kaiyi 06 Jan '21

06 Jan '21

change kernbench into kernbench.cr Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/kernbench.cr | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/stats/crystal/kernbench.cr b/stats/crystal/kernbench.cr index 0da37fe88..9777a11ca 100644 --- a/stats/crystal/kernbench.cr +++ b/stats/crystal/kernbench.cr @@ -1,28 +1,26 @@ -#!/usr/bin/env ruby +#!/usr/bin/env crystal -LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) +require "../../lib/statistics" -require "#{LKP_SRC}/lib/statistics" - -$stdin.each_line do |line| +STDIN.each_line do |line| case line when /Elapsed Time/ - puts 'elapsed_time: ' + line.split[2] - puts 'elapsed_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "elapsed_time: " + line.split[2] + puts "elapsed_time_stddev: " + line.split[3].gsub(/[()]/, "") when /User Time/ - puts 'user_time: ' + line.split[2] - puts 'user_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "user_time: " + line.split[2] + puts "user_time_stddev: " + line.split[3].gsub(/[()]/, "") when /System Time/ - puts 'system_time: ' + line.split[2] - puts 'system_time_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "system_time: " + line.split[2] + puts "system_time_stddev: " + line.split[3].gsub(/[()]/, "") when /Percent CPU/ - puts 'percent_cpu: ' + line.split[2] - puts 'percent_cpu_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "percent_cpu: " + line.split[2] + puts "percent_cpu_stddev: " + line.split[3].gsub(/[()]/, "") when /Context Switches/ - puts 'context_switches: ' + line.split[2] - puts 'context_switches_stddev: ' + line.split[3].gsub(/[()]/, "") + puts "context_switches: " + line.split[2] + puts "context_switches_stddev: " + line.split[3].gsub(/[()]/, "") when /Sleeps/ - puts 'sleeps: ' + line.split[1] - puts 'sleeps_stddev: ' + line.split[2].gsub(/[()]/, "") + puts "sleeps: " + line.split[1] + puts "sleeps_stddev: " + line.split[2].gsub(/[()]/, "") end end -- 2.23.0

1 0

[PATCH v3 compass-ci 7/7] send-internet-mail/start: add key REDIS_PORT
by Luan Shengde 06 Jan '21

06 Jan '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start index 02c6ab6..136435d 100755 --- a/container/send-internet-mail/start +++ b/container/send-internet-mail/start @@ -13,6 +13,7 @@ names = Set.new %w[ ROBOT_EMAIL_PASSWORD SEND_MAIL_PORT SENT_MAILDIR + REDIS_PORT ] defaults = relevant_defaults(names) -- 2.23.0

1 0

[PATCH v3 compass-ci 6/7] lib/mail-post: add email checks
by Luan Shengde 06 Jan '21

06 Jan '21

add check email count only email addresses that email count within the limited value can be sent emails add check email mapping if has set email mapping for the entered address. the address will be converted to the mapping value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail-post.rb | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) mode change 100644 => 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb old mode 100644 new mode 100755 index 9d3e601..803a134 --- a/lib/mail-post.rb +++ b/lib/mail-post.rb @@ -57,14 +57,32 @@ post '/send_mail_encode' do send_mail(mail_info) end +def check_email_limit(mail_info) + email_limit = EmailRateLimit.new(mail_info) + email_limit.check_email_counts +end + +def check_email_mapping(mail_info) + email_mapping = EmailAddrMapping.new(mail_info) + email_mapping.check_email_mapping +end + def check_send_mail(mail_info) raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? + + return mail_info unless ENV['SEND_MAIL_PORT'].to_s == '49000' + + mail_info = check_email_mapping(mail_info.clone) + mail_info = check_email_limit(mail_info.clone) + return mail_info end def send_mail(mail_info) - check_send_mail(mail_info) + mail_info = check_send_mail(mail_info) + return if mail_info['to'].empty? + mail = Mail.new do from ENV['ROBOT_EMAIL_ADDRESS'] subject mail_info['subject'] @@ -79,7 +97,6 @@ def send_mail(mail_info) end def check_to_store_email(mail) - return if ENV['SEND_MAIL_PORT'].to_s != '49000' return if ENV['HOST_SERVER'] != 'z9' -- 2.23.0

1 0

[PATCH v3 compass-ci 5/7] send-internet-mail: email rate limit
by Luan Shengde 06 Jan '21

06 Jan '21

add email_in_limit to store email address that send count less than limit value add email_out_limit to store email address that send count beyond limit value the email queues email_in_limit/email_out_limit will be reset everyday at 00:00, every time send mail to an email address, the value for the email address in email_in_limit will +1, until the value up to the limited value, the email address will be moved to email_out_limit, and the email send to the address will be throw away. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../send-internet-mail/email_limit_queue.rb | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100755 container/send-internet-mail/email_limit_queue.rb diff --git a/container/send-internet-mail/email_limit_queue.rb b/container/send-internet-mail/email_limit_queue.rb new file mode 100755 index 0000000..a5ca748 --- /dev/null +++ b/container/send-internet-mail/email_limit_queue.rb @@ -0,0 +1,50 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'redis' + +# to check if the emails count has beyond the limit +class EmailRateLimit + def initialize(mail_info) + @mail_info = mail_info + @redis = Redis.new('host' => REDIS_HOST, 'port' => REDIS_PORT) + end + + def check_email_counts + email_to = @mail_info['to'].clone + email_cc = @mail_info['cc'].clone + @mail_info['to'] = check_emails(email_to) + @mail_info['cc'] = check_emails(email_cc) + + return @mail_info + end + + def check_emails(mail_list) + return if mail_list.nil? || mail_list.empty? + + mail_list.clone.each do |email| + if @redis.hexists 'email_out_limit', email + mail_list -= [email] + next + elsif @redis.hexists 'email_in_limit', email + email_account = @redis.hget 'email_in_limit', email + @redis.hset 'email_in_limit', email, email_account.to_i + 1 + else + @redis.hset 'email_in_limit', email, 1 + end + + change_queue(email) + end + return mail_list + end + + def change_queue(email) + return unless (@redis.hget 'email_in_limit', email).to_i >= 10 + + @redis.hdel 'email_in_limit', email + @redis.hset 'email_out_limit', email, 10 + end +end -- 2.23.0

1 0

[PATCH v3 compass-ci 4/7] send-internet-mail: add email mapping
by Luan Shengde 06 Jan '21

06 Jan '21

example: name => email_addr email_addr_a => email_addr email_addr_b => email_addr an email address can be set multi mappings, you can set a name or email to a specified email address when you write either a name or an email address in the send address bar, the email address will be converted to the specified address Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/email_mapping.rb | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100755 container/send-internet-mail/email_mapping.rb diff --git a/container/send-internet-mail/email_mapping.rb b/container/send-internet-mail/email_mapping.rb new file mode 100755 index 0000000..f190896 --- /dev/null +++ b/container/send-internet-mail/email_mapping.rb @@ -0,0 +1,39 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'redis' + +# to check if the emails count has beyond the limit +class EmailAddrMapping + def initialize(mail_info) + @mail_info = mail_info + @redis = Redis.new('host' => REDIS_HOST, 'port' => REDIS_PORT) + end + + def check_email_mapping + email_to = @mail_info['to'].clone + email_cc = @mail_info['cc'].clone + email_bcc = @mail_info['bcc'].clone + @mail_info['to'] = email_mapping(email_to) + @mail_info['cc'] = email_mapping(email_cc) + @mail_info['bcc'] = email_mapping(email_bcc) + + return @mail_info + end + + def email_mapping(mail_list) + return if mail_list.nil? || mail_list.empty? + + mail_list.clone.each do |email| + next unless @redis.hexists 'email_mapping', email + + mapped_email = @redis.hget 'email_mapping', email + mail_list -= [email] + mail_list << mapped_email unless mail_list.include? mapped_email + end + return mail_list + end +end -- 2.23.0

1 0

[PATCH v3 compass-ci 3/7] send-internet-mail: add timing work
by Luan Shengde 06 Jan '21

06 Jan '21

timing work: reset email rate limit queues everyday Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/email_init.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100755 container/send-internet-mail/email_init.rb diff --git a/container/send-internet-mail/email_init.rb b/container/send-internet-mail/email_init.rb new file mode 100755 index 0000000..099d89d --- /dev/null +++ b/container/send-internet-mail/email_init.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'rufus-scheduler' +require 'redis' + +redis = Redis.new(host: REDIS_HOST, port: REDIS_PORT) +email_init = Rufus::Scheduler.new + +# Timing work for email counting everyday +# the email counts queues will be reset everyday +# to enable to send mail in the new day to user +email_init.cron '0 0 * * *' do + redis.del 'email_in_limit' + redis.del 'email_out_limit' +end -- 2.23.0

1 0

[PATCH v3 compass-ci 2/7] send-internet-mail: optimize
by Luan Shengde 06 Jan '21

06 Jan '21

add init redis host/port add invoke code files Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/send-internet-mail.rb | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb index 8fdaa0d..7b357f8 100755 --- a/container/send-internet-mail/send-internet-mail.rb +++ b/container/send-internet-mail/send-internet-mail.rb @@ -6,7 +6,13 @@ require 'mail' require 'sinatra' -require "#{ENV['CCI_SRC']}/lib/mail-post" +REDIS_HOST = %x(/sbin/ip route | awk '/default/ {print $3}').chomp +REDIS_PORT = ENV['REDIS_PORT'] + +require_relative 'email_init' +require_relative 'email_limit_queue' +require_relative 'email_mapping' +require_relative '../../lib/mail-post' smtp = { address: 'smtp.qq.com', -- 2.23.0

1 0

[PATCH v3 compass-ci 1/7] send-internet-mail/Dockerfile: add install gem resources
by Luan Shengde 06 Jan '21

06 Jan '21

[why] the service need to install redis to store email counts to redis queue the service need to install rufus-scheduler to reset the queue as a timing task Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile index d151bda..03e3327 100644 --- a/container/send-internet-mail/Dockerfile +++ b/container/send-internet-mail/Dockerfile @@ -6,4 +6,4 @@ MAINTAINER shdluan(a)163.com ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \ - apt-get install -y ruby-mail ruby-json ruby-sinatra + apt-get install -y ruby-mail ruby-json ruby-sinatra ruby-redis ruby-rufus-scheduler -- 2.23.0

1 0