- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] tests/host-info: fix host-info script running error
by Zhang Yu 15 Dec '20

15 Dec '20

[why] The ipmi command just run in physical machine, the vm not support use ipmi command [how] Add a judgment only physical machine run ipmi operation Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/host-info b/tests/host-info index 84f796dcb..f222a7287 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,5 +1,7 @@ #!/bin/bash +. $LKP_SRC/lib/env.sh + get_memory() { memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') @@ -101,6 +103,7 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX + [ is_virt || is_docker ] && return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip:" $ip -- 2.23.0

3 4

[PATCH lkp-tests] lib/env.sh: make the change of is_virt
by Zhang Yu 15 Dec '20

15 Dec '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/env.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/env.sh b/lib/env.sh index 78cfd6f57..fc34aaa8d 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -69,7 +69,8 @@ is_virt() # run as root [ -n "$(virt-what)" ] else - grep -q -w hypervisor /proc/cpuinfo + grep -q -w hypervisor /proc/cpuinfo && return 1 + return 0 fi } -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix format
by Lin Jiaxin 15 Dec '20

15 Dec '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index aafc120e..dbde3e54 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,8 +2,8 @@ # - pkgbuild_repo # - upstream_repo # - upstream_commit +# - upstream_url # - PKGBUILD_TAG -# upstream_url # We'll use the below global env vars, but not treat them as parameter to avoid # duplicates in result path like /openeuler-20.03-aarch64/openeuler-aarch64-cifs-2003/ -- 2.23.0

1 0

[PATCH lkp-tests 3/3] use ENV LKP_SRC
by Liu Yinsi 15 Dec '20

15 Dec '20

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- install.sh | 2 +- sbin/install-dependencies.sh | 4 ++-- sbin/set-env.sh | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/install.sh b/install.sh index 4bcc1afb4..cea1654f2 100644 --- a/install.sh +++ b/install.sh @@ -1,6 +1,6 @@ #!/bin/bash -. lib/env.sh +. $LKP_SRC/lib/env.sh make diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 3436ee44a..7605a29ff 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -1,7 +1,7 @@ #!/bin/bash -. lib/env.sh -. sbin/set-env.sh +. $LKP_SRC/lib/env.sh +. $LKP_SRC/sbin/set-env.sh # choose install function base on # DISTRIBUTION diff --git a/sbin/set-env.sh b/sbin/set-env.sh index 27471bb56..cc7f90506 100755 --- a/sbin/set-env.sh +++ b/sbin/set-env.sh @@ -1,6 +1,6 @@ #!/bin/sh -. lib/env.sh +. $LKP_SRC/lib/env.sh write_shell_profile() { -- 2.23.0

1 0

[PATCH lkp-tests 2/3] chmod +x
by Liu Yinsi 15 Dec '20

15 Dec '20

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/install-dependencies.sh | 0 sbin/set-env.sh | 0 2 files changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 sbin/install-dependencies.sh mode change 100644 => 100755 sbin/set-env.sh diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh old mode 100644 new mode 100755 diff --git a/sbin/set-env.sh b/sbin/set-env.sh old mode 100644 new mode 100755 -- 2.23.0

1 0

[PATCH v3 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 15 Dec '20

15 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 87 ++++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..3110685 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -152,29 +152,20 @@ job yaml ��[YAML](http://yaml.org/YAM 1. �� os �� rootfs�� openEuler �� - �� docker �� rootfs - + 1) �� openEuler �� docker �� - + ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` + ``` + 2) {compass-ci}/container/docker-rootfs �� '.tar.xz' �� docker �� rootfs + ```bash + ./{compass-ci}/container/docker-rootfs/run openEuler-docker.aarch64.tar.xz /tmp/openeuler-rootfs-20.03 + ``` - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� + + 1) �� openEuler �� qcow2 �� ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… ``` @@ -183,45 +174,59 @@ job yaml ��[YAML](http://yaml.org/YAM cd {compass-ci}/container/qcow2rootfs ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) + + 2. ��rootfs + 1. �� chroot �� rootfs (�� root ��) ```bash chroot openEuler-rootfs ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 2. �� + a. �� /sbin/init + > ��systemd�� + ```bash + ls -l /sbin/init + /sbin/init -> ../lib/systemd/systemd + ``` + b. �� /etc/localtime + > �� + ```bash + ls -l /etc/localtime + /etc/localtime -> ../usr/share/zoneinfo/Asia/Shanghai + ``` + c. �� docker �� osimage �� + > docker �� /.dockerenv �� + + 3. ��rootfs + ```bash + cd $rootfs + find . | coip -o -Hnewc |gzip -9 > $os_name.cgz + ``` + + 4. ��/�� + �� {compass-ci} �� rootfs ��/�� + ```bash + find /lib/modules/* | cpio -o -Hnewc | gzip -9 > /modules-$os_name.cgz + find /usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > /headers-$os_name.cgz + ``` + #### FAQ -1. �� Unable to mount root fs on unknown-block�� +1. �� Unable to mount root fs on unknown-block�� - �� ```bash [ 0.390437] List of all partitions: - [ 0.390806] No filesystem could mount root, tried: + [ 0.390806] No filesystem could mount root, tried: [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ... [ 0.399404] Memory Limit: none [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- ``` - �� - + 1�� 2�� 644 �� - + 2. �� - + - �� -- 2.23.0

2 8

[PATCH v3 compass-ci] lib/job.cr: optimize check account error message
by Wu Zhende 15 Dec '20

15 Dec '20

Like: submit iperf.yaml failed, got job_id=0, error: Failed to verify the account. Please refer to https://gitee.com/wu_fengguang/compass-ci/doc/manual/apply-account.md Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..8ce8649 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -326,7 +326,8 @@ class Job end private def check_account_info - error_msg = "Failed to verify the account. Please check your configuration" + error_msg = "Failed to verify the account.\n" + error_msg += "Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" account_info = @es.get_account(self["my_email"]) raise account_info unless account_info.is_a?(JSON::Any) -- 2.23.0

1 0

[PATCH v2 lkp-tests] distro/depends: add some depends
by Liu Yinsi 15 Dec '20

15 Dec '20

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/cci-deploy | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index c797c1eea..2b1d7fe52 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -8,3 +8,7 @@ bridge-utils qemu dmidecode lockfile-progs +iproute2 +vim +ruby-dev +busybox -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: add some depends
by Liu Yinsi 15 Dec '20

15 Dec '20

command ip used in sparrow/2-network/br0. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/cci-deploy | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index c797c1eea..2b1d7fe52 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -8,3 +8,7 @@ bridge-utils qemu dmidecode lockfile-progs +iproute2 +vim +ruby-dev +busybox -- 2.23.0

1 1

[PATCH v2 compass-ci] lib/job.cr: optimize check account error message
by Wu Zhende 15 Dec '20

15 Dec '20

Like: submit iperf.yaml failed, got job_id=0, error: Failed to verify the account. Please refer to https://gitee.com/wu_fengguang/compass-ci/doc/manual/apply-account.md Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..697e6ae 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -326,7 +326,8 @@ class Job end private def check_account_info - error_msg = "Failed to verify the account. Please check your configuration" + error_msg = "Failed to verify the account.\n" + error_msg += "Please refer to https://gitee.com/wu_fengguang/compass-ci/doc/manual/apply-account.md" account_info = @es.get_account(self["my_email"]) raise account_info unless account_info.is_a?(JSON::Any) -- 2.23.0

2 1

[PATCH v3 lkp-tests 2/2] lib/job.rb: fix duplicate default field
by Wei Jihui 15 Dec '20

15 Dec '20

input: /etc/commpass-ci/default/crystal.yaml lab: lab1 ~/.config/.compass-ci/default/*.yaml lab: lab2 original output: #! /etc/commpass-ci/default/crystal.yaml lab: lab2 #! ~/.config/.compass-ci/default/*.yaml now output: #! /etc/commpass-ci/default/crystal.yaml #! ~/.config/.compass-ci/default/*.yaml lab: lab2 Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index 9eac4001..42b008c7 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -405,6 +405,7 @@ class Job return false end if defaults.is_a?(Hash) && !defaults.empty? + @defaults.delete_if { |key, _| defaults.has_key?(key) } @defaults[source_file_symkey(file)] = nil revise_hash(@defaults, defaults, true) @defaults.merge!(@overrides) -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/2] lib/job.rb: load config from lab yaml
by Wei Jihui 15 Dec '20

15 Dec '20

[why] my_token is per-lab account, so it will load my_token from ~/.config/compass-ci/include/lab/$lab.yaml and $lab is from ~/.config/compass-ci/defaults/*.yaml or /etc/compass-ci/defaults/*.yaml Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index b41d55a7..9eac4001 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -414,6 +414,17 @@ class Job true end + def load_self_config + self_config_path = "#{ENV['HOME']}/.config/compass-ci" + Dir.glob(['/etc/compass-ci/defaults/*.yaml', + "#{self_config_path}/defaults/*.yaml"]).each do |file| + load_one_defaults(file, @job) + end + + lab_yaml = File.join(self_config_path, 'include/lab', "#{@defaults['lab']}.yaml") + load_one_defaults(lab_yaml, @job) + end + def load_defaults(first_time = true) if @job.include? :no_defaults merge_defaults first_time @@ -426,11 +437,7 @@ class Job @file_loaded ||= {} end - Dir.glob(["/etc/compass-ci/defaults/*.yaml", - "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| - load_one_defaults(file, @job) - end - + load_self_config i = include_files job = deepcopy(@job) job['___'] = nil -- 2.23.0

1 0

[PATCH v4 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 15 Dec '20

15 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 87 ++++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..3110685 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -152,29 +152,20 @@ job yaml ��[YAML](http://yaml.org/YAM 1. �� os �� rootfs�� openEuler �� - �� docker �� rootfs - + 1) �� openEuler �� docker �� - + ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` + ``` + 2) {compass-ci}/container/docker-rootfs �� '.tar.xz' �� docker �� rootfs + ```bash + ./{compass-ci}/container/docker-rootfs/run openEuler-docker.aarch64.tar.xz /tmp/openeuler-rootfs-20.03 + ``` - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� + + 1) �� openEuler �� qcow2 �� ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… ``` @@ -183,45 +174,59 @@ job yaml ��[YAML](http://yaml.org/YAM cd {compass-ci}/container/qcow2rootfs ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) + + 2. ��rootfs + 1. �� chroot �� rootfs (�� root ��) ```bash chroot openEuler-rootfs ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 2. �� + a. �� /sbin/init + > ��systemd�� + ```bash + ls -l /sbin/init + /sbin/init -> ../lib/systemd/systemd + ``` + b. �� /etc/localtime + > �� + ```bash + ls -l /etc/localtime + /etc/localtime -> ../usr/share/zoneinfo/Asia/Shanghai + ``` + c. �� docker �� osimage �� + > docker �� /.dockerenv �� + + 3. ��rootfs + ```bash + cd $rootfs + find . | coip -o -Hnewc |gzip -9 > $os_name.cgz + ``` + + 4. ��/�� + �� {compass-ci} �� rootfs ��/�� + ```bash + find /lib/modules/* | cpio -o -Hnewc | gzip -9 > /modules-$os_name.cgz + find /usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > /headers-$os_name.cgz + ``` + #### FAQ -1. �� Unable to mount root fs on unknown-block�� +1. �� Unable to mount root fs on unknown-block�� - �� ```bash [ 0.390437] List of all partitions: - [ 0.390806] No filesystem could mount root, tried: + [ 0.390806] No filesystem could mount root, tried: [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ... [ 0.399404] Memory Limit: none [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- ``` - �� - + 1�� 2�� 644 �� - + 2. �� - + - �� -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow: fix that 0-package/os/openEuler can only be executed once
by Yu Chuan 15 Dec '20

15 Dec '20

[Error Msg] ++ rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm warning: lib64lockfile1-1.16-1-omv4002.aarch64.rpm: Header V4 RSA/SHA256 Signature, key ID bf81de15: NOKEY package lib64lockfile1-1.16-1.aarch64 is already installed file /usr/lib64/liblockfile.so.1.0 from install of lib64lockfile1-1.16-1.aarch64 conflicts with file from package lib64lockfile1-1.16-1.aarch64 package lockfile-progs-0.1.18-5.aarch64 is already installed file /usr/bin/lockfile-check from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-create from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-remove from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-touch from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-lock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-touchlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-unlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/0-package/os/openEuler | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 43e96ab8ab2e..5b32c0a23784 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -2,6 +2,17 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +install_rpm() +{ + local pkg=$1 + local pkg_url=$2 + + rpm -qa | grep -q "$pkg" && return 0 + + wget $pkg_url + rpm -i $(basename $pkg_url) && rm -rf $(basename $pkg_url) +} + pkgs=( vim git @@ -24,8 +35,10 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +install_rpm \ + "lib64lockfile1-1.16-1.$(arch)"\ + "http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas…" -rpm -i lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm -rm -rf *.rpm +install_rpm \ + "lockfile-progs-0.1.18-5.$(arch)"\ + "http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas…" -- 2.23.0

2 3

[PATCH compass-ci 1/2] sparrow: refactor the code for docker build all image concurrently
by Yu Chuan 15 Dec '20

15 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/4-docker/buildall | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index eb245008ad45..43a3a06e4222 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -8,18 +8,13 @@ export action=$1 build_depends() { local container=$1 - local should_wait=$2 for dep in $(cat $container/*-depends 2> /dev/null) do - build_depends $CONTAINER_PATH/$dep block_wait + build_depends $CONTAINER_PATH/$dep done - if [ -n "$should_wait" ]; then - do_one $container - else - do_one $container & - fi + do_one $container } do_one() @@ -44,7 +39,7 @@ tmpdir=$(mktemp -d) for dir in $CONTAINER_PATH/*/ do - build_depends $dir + build_depends $dir & done wait -- 2.23.0

3 3

[PATCH v3 compass-ci 1/2] mail-robot: use key my_token instead of my_uuid
by Luan Shengde 15 Dec '20

15 Dec '20

[why] key my_uuid is renamed to my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 31 +++++++++++++---------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb index 0fff3b5..aea1aa5 100755 --- a/container/mail-robot/lib/apply-account.rb +++ b/container/mail-robot/lib/apply-account.rb @@ -30,14 +30,14 @@ SEND_MAIL_PORT = ENV['SEND_MAIL_PORT'] || 49000 # my_info: # - my_email # - my_name -# - my_uuid +# - my_token # my_ssh_pubkey # store_account_info # call ESClient to store my_info # my_info: # - my_email # - my_name -# - my_uuid +# - my_token # - my_commit_url # - my_login_name # - my_ssh_pubkey @@ -87,21 +87,28 @@ class ApplyAccount accountx_es.query_by_id(@my_info['my_email']) end + def build_apply_info(apply_info, my_account_es) + my_ssh_pubkey_new = @my_info.delete('my_ssh_pubkey') + apply_info['my_token'] = my_account_es['my_uuid'] if my_account_es['my_token'].nil? + apply_info.update my_account_es + apply_info.update @my_info + apply_info['my_ssh_pubkey'] = (apply_info['my_ssh_pubkey'] + my_ssh_pubkey_new).uniq + @my_info.update apply_info + apply_info['is_update_account'] = true + apply_info + end + def apply_my_account my_account_es = read_my_account_es apply_info = {} if my_account_es - my_ssh_pubkey_new = @my_info.delete('my_ssh_pubkey') - apply_info.update my_account_es - apply_info.update @my_info - apply_info['my_ssh_pubkey'] = (apply_info['my_ssh_pubkey'] + my_ssh_pubkey_new).uniq - @my_info.update apply_info - apply_info['is_update_account'] = true + build_apply_info(apply_info, my_account_es) else - my_uuid = %x(uuidgen).chomp - @my_info['my_uuid'] = my_uuid + my_token = %x(uuidgen).chomp + @my_info['my_token'] = my_token apply_info.update @my_info end + apply_info['lab'] = ENV['lab'] apply_new_account(apply_info, my_account_es) end @@ -112,10 +119,6 @@ class ApplyAccount @my_info['my_login_name'] = acct_info['my_login_name'] unless my_account_es end - def check_account_es - ESQuery.new(index: 'accounts').query_by_id(@my_info['my_email']) - end - def store_account_info es = ESClient.new(index: 'accounts') es.put_source_by_id(@my_info['my_email'], @my_info) -- 2.23.0

2 2

[PATCH compass-ci] lib/job.cr: optimize check account error message
by Wu Zhende 15 Dec '20

15 Dec '20

Like: submit iperf.yaml failed, got job_id=0, error: Failed to verify the account. Please refer to https://gitee.com/wu_fengguang/compass-ci/doc/manual/apply-account.md Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..7eb8498 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -326,7 +326,8 @@ class Job end private def check_account_info - error_msg = "Failed to verify the account. Please check your configuration" + error_msg = "Failed to verify the account. " + error_msg += "Please refer to https://gitee.com/wu_fengguang/compass-ci/doc/manual/apply-account.md" account_info = @es.get_account(self["my_email"]) raise account_info unless account_info.is_a?(JSON::Any) -- 2.23.0

2 1

[PATCH lkp-tests] iso2rootfs: fix `has_cmd: command not found` error
by Yu Chuan 15 Dec '20

15 Dec '20

[Error Msg] ++ has_cmd yum /lkp/lkp/src/tests/iso2rootfs: line 213: has_cmd: command not found Signed-off-by: Yu Chuan <13186087857(a)163.com> --- tests/iso2rootfs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 8625b6484d89..9f2a77de17c6 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -6,6 +6,8 @@ set -e umask 0002 +. $LKP_SRC/lib/env.sh + # allow sut os array # - if job's os not in this array, job won't execute ALLOW_OS=( -- 2.23.0

3 2

[PATCH v2 lkp-tests 1/2] lib/job.rb: load config from lab yaml
by Wei Jihui 15 Dec '20

15 Dec '20

[why] my_uuid is per-lab account, so it will load my_uuid from ~/.config/compass-ci/include/lab/$lab.yaml and $lab is from ~/.config/compass-ci/defaults/*.yaml or /etc/compass-ci/defaults/*.yaml Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index b41d55a7..9eac4001 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -414,6 +414,17 @@ class Job true end + def load_self_config + self_config_path = "#{ENV['HOME']}/.config/compass-ci" + Dir.glob(['/etc/compass-ci/defaults/*.yaml', + "#{self_config_path}/defaults/*.yaml"]).each do |file| + load_one_defaults(file, @job) + end + + lab_yaml = File.join(self_config_path, 'include/lab', "#{@defaults['lab']}.yaml") + load_one_defaults(lab_yaml, @job) + end + def load_defaults(first_time = true) if @job.include? :no_defaults merge_defaults first_time @@ -426,11 +437,7 @@ class Job @file_loaded ||= {} end - Dir.glob(["/etc/compass-ci/defaults/*.yaml", - "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| - load_one_defaults(file, @job) - end - + load_self_config i = include_files job = deepcopy(@job) job['___'] = nil -- 2.23.0

2 1

[PATCH compass-ci 6/6] fetch-mail: start file
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/start | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 container/fetch-mail/start diff --git a/container/fetch-mail/start b/container/fetch-mail/start new file mode 100755 index 0000000..372847b --- /dev/null +++ b/container/fetch-mail/start @@ -0,0 +1,36 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +docker_rm 'fetch-mail' + +names = Set.new %w[ + FETCH_MAIL_ADDRESS + FETCH_MAIL_AUTH_CODE + FETCH_MAIL_DIR + FETCH_MAIL_BOX +] + +defaults = relevant_defaults(names) + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + -u mailer + --name=fetch-mail + -d +] + env + %W[ + -v #{defaults['FETCH_MAIL_DIR']}:#{defaults['FETCH_MAIL_DIR']}:rw + -w /home/mailer/ + fetch-mail +] + +cmd += ['sh', '-c', './run.sh'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci 5/6] fetch-mail: build file
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/fetch-mail/build diff --git a/container/fetch-mail/build b/container/fetch-mail/build new file mode 100755 index 0000000..e72bed8 --- /dev/null +++ b/container/fetch-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t fetch-mail:latest . -- 2.23.0

1 0

[PATCH compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..6e553b9 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 1000 mailer +RUN apt-get update && \ + apt-get install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer -- 2.23.0

1 0

[PATCH compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 15 Dec '20

15 Dec '20

re-edit config file for fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100644 index 0000000..d7f9f38 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +function mail_conf { + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf + +fetchmail -d 100 -- 2.23.0

1 0

[PATCH compass-ci 2/6] fetch-mail: config file for procmail
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/procmailrc.template | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 container/fetch-mail/procmailrc.template diff --git a/container/fetch-mail/procmailrc.template b/container/fetch-mail/procmailrc.template new file mode 100644 index 0000000..9cea4bc --- /dev/null +++ b/container/fetch-mail/procmailrc.template @@ -0,0 +1,25 @@ +DROPPRIVS=yes +PATH=$HOME/bin:/usr/local/bin:/usr/bin:/bin +MAILDIR=FETCH_MAIL_DIR +MAILLIST=${MAILDIR} +DEFAULT=${MAILDIR}/FETCH_MAIL_BOX/ +ORGMAIL=${DEFAULT}/FETCH_MAIL_BOX/ +LOGFILE=${MAILDIR}/log/procmail # recommended + +UMASK=002 + +DISPLAY=:0 +XAUTHORITY=$HOME/.Xauthority + +# One often receives duplicate messages for cross-postings to mailing +# lists, or explicit replies to a post from you in addition to the +# mailing list message. +# +# The following recipe eliminates all duplicate messages. It tells +# formail(1) to keep a cache file of the Message-IDs of the most recent +# messages you've received. Since Message-IDs are guaranteed unique for +# distinct messages, they are ideally suited to weed out duplicates. + +# LOCKFILE=$MAILDIR/.msgid.lock +:0 Wh: $MAILDIR/.msgid.lock +| formail -D 655360 $MAILDIR/log/msgid -- 2.23.0

1 0

[PATCH compass-ci 1/6] fetch-mail: config file for fetchmail
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/fetchmailrc.template | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/fetch-mail/fetchmailrc.template diff --git a/container/fetch-mail/fetchmailrc.template b/container/fetch-mail/fetchmailrc.template new file mode 100644 index 0000000..8c7e82b --- /dev/null +++ b/container/fetch-mail/fetchmailrc.template @@ -0,0 +1,5 @@ +poll imap.qq.com with proto IMAP +username "FETCH_MAIL_ADDRESS" +password "FETCH_MAIL_AUTH_CODE" +mda "/usr/bin/procmail -f %F -d %T" +nokeep -- 2.23.0

1 0