November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] web_backend.rb: add space line after return cmd
by Xu Xijian 19 Nov '20

19 Nov '20

When there is a return cmd in a function, and it's not the last line of this function, there should have a blank line after return cmd. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/web_backend.rb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index b8df6f4..1f7224e 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -220,7 +220,9 @@ end def wrong_size?(size, from) return true if from.negative? || size.negative? + return true if from > 1000000 || size > 1000000 + return true if size + from > 1000000 end @@ -344,6 +346,7 @@ def get_repo_url(urls) urls.each do |url| return url if url[0, 4] == 'http' end + urls[0] end -- 2.23.0

1 0

[PATCH v3 compass-ci] sparrow: fix 3-code/dev-env no such file error
by Liu Yinsi 19 Nov '20

19 Nov '20

when execute 3-code/dev-env, error message as follows: ./dev-env: line 45: ./os/openEuler: No such file or directory Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 77027d9..98da280 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -42,4 +42,5 @@ EOF source /etc/os-release -. $(dirname ${BASH_SOURCE[0]})/os/${ID} +path=$(dirname ${BASH_SOURCE[0]})/os/${ID} +[ -x "$path" ] && . "$path" -- 2.23.0

3 5

[PATCH compass-ci] lib/log.rb: add new class for json log
by Wu Zhende 19 Nov '20

19 Nov '20

[Why] The built-in log system peints logs in string format by default. The logs should be formatted in json format for subsequent analysis and processing. [Example] require './log' log = Log.new use: log.info("tset") output: {"level":"INFO","level_num":1,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"test"} use: log.error({"job_id" => "1", "job_state" => "submit"}.to_json) output: {"level":"ERROR","level_num":3,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"", "job_id":"1","job_state":"submit","caller":["/usr/share/ruby/logger.rb:4:in `format_message'", "/usr/share/ruby/logger.rb:472:in`add'","/usr/share/ruby/logger.rb:545:in `error'","./test.rb:6:in`<main>'"]} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/log.rb | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 lib/log.rb diff --git a/lib/log.rb b/lib/log.rb new file mode 100644 index 0000000..0ffc0f9 --- /dev/null +++ b/lib/log.rb @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' +require 'json' + +# log class to print logs in JSON format +class Log < Logger + LEVEL_INFO = { + 'DEBUG' => 0, + 'INFO' => 1, + 'WARN' => 2, + 'ERROR' => 3, + 'FATAL' => 4, + 'UNKNOWN' => 5 + }.freeze + + FORMATTER = proc { |severity, datetime, progname, msg| + puts + level_num = LEVEL_INFO[severity] + begin + message = JSON.parse(msg) + rescue JSON::ParserError + message = { 'message' => msg } + end + h = { + 'level' => severity.to_s, + 'level_num' => level_num, + 'datetime' => datetime, + 'progname' => progname, + 'message' => '' + } + h.merge!(message) + h.merge!({ 'caller' => caller }) if level_num >= 2 + h.to_json + } + + def initialize(logdev = STDOUT, formatter = FORMATTER) + super(logdev, formatter: formatter) + end +end -- 2.23.0

2 1

[PATCH v2 compass-ci] lib/log.rb: add new class for json log
by Wu Zhende 19 Nov '20

19 Nov '20

[Why] The built-in log system peints logs in string format by default. The logs should be formatted in json format for subsequent analysis and processing. [Example] require './log' log = Log.new log.info("tset") output: {"level":"INFO","level_num":1,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"test"} log.error({"job_id" => "1", "job_state" => "submit"}.to_json) output: {"level":"ERROR","level_num":3,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"", "job_id":"1","job_state":"submit","caller":["/usr/share/ruby/logger.rb:4:in `format_message'", "/usr/share/ruby/logger.rb:472:in`add'","/usr/share/ruby/logger.rb:545:in `error'","./test.rb:6:in`<main>'"]} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/log.rb | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 lib/log.rb diff --git a/lib/log.rb b/lib/log.rb new file mode 100644 index 0000000..0ffc0f9 --- /dev/null +++ b/lib/log.rb @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' +require 'json' + +# log class to print logs in JSON format +class Log < Logger + LEVEL_INFO = { + 'DEBUG' => 0, + 'INFO' => 1, + 'WARN' => 2, + 'ERROR' => 3, + 'FATAL' => 4, + 'UNKNOWN' => 5 + }.freeze + + FORMATTER = proc { |severity, datetime, progname, msg| + puts + level_num = LEVEL_INFO[severity] + begin + message = JSON.parse(msg) + rescue JSON::ParserError + message = { 'message' => msg } + end + h = { + 'level' => severity.to_s, + 'level_num' => level_num, + 'datetime' => datetime, + 'progname' => progname, + 'message' => '' + } + h.merge!(message) + h.merge!({ 'caller' => caller }) if level_num >= 2 + h.to_json + } + + def initialize(logdev = STDOUT, formatter = FORMATTER) + super(logdev, formatter: formatter) + end +end -- 2.23.0

2 1

[PATCH compass-ci] qcow2rootfs: change yum source to fix build timeout error
by Yu Chuan 19 Nov '20

19 Nov '20

[Error Msg] Step 4/7 : RUN yum makecache && yum install -y --skip-broken libvirt libguestfs-tools xz expect openssl && yum clean all ---> Running in c443a95dbd5f CentOS-8 - AppStream2.2 kB/s | 2.5 MB 19:23 Errors during downloading metadata for repository 'AppStream': - Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/ce3501f1421… [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds] - Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/cc0e75f5237… [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds] - Curl error (18): Transferred a partial file for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/ce3501f1421… [transfer closed with 1750223 bytes remaining to read] Error: Failed to download metadata for repo 'AppStream': Yum repo downloading error: Downloading error(s): repodata/cc0e75f5237c3f5ddfea6ab0de1056df5bb8bbf1273b5ecb907ed7c1c72897f3-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/ce3501f14219e71c21c1e40c6f91ba41bd37b59345c5a7dd6d680ef4b279b95a-filelists.xml.gz - Cannot download, all mirrors were already tried without success Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/qcow2rootfs/Dockerfile | 4 +- .../etc/yum.repos.d/CentOS-AltArch-7.repo | 43 +++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo diff --git a/container/qcow2rootfs/Dockerfile b/container/qcow2rootfs/Dockerfile index a0f8b042520b..38418c7eb226 100644 --- a/container/qcow2rootfs/Dockerfile +++ b/container/qcow2rootfs/Dockerfile @@ -1,12 +1,14 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM centos:8 +FROM centos:7 MAINTAINER Yu Chuan <13186087857(a)163.com>, Shi Zhichao <angus_robot(a)163.com> RUN rpm --import /etc/pki/rpm-gpg/RPM* +RUN rm -rf /etc/yum.repos.d/* +COPY root / RUN yum makecache && \ yum install -y --skip-broken libvirt libguestfs-tools xz expect openssl && \ yum clean all diff --git a/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo b/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo new file mode 100644 index 000000000000..b99c0a60895d --- /dev/null +++ b/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo @@ -0,0 +1,43 @@ +# CentOS-Base.repo +# +# The mirror system uses the connecting IP address of the client and the +# update status of each mirror to pick mirrors that are updated to and +# geographically close to the client. You should use this for CentOS updates +# unless you are manually picking other mirrors. +# +# If the mirrorlist= does not work for you, as a fall back you can try the +# remarked out baseurl= line instead. +# +# + +[base] +name=CentOS-$releasever - Base - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#released updates +[updates] +name=CentOS-$releasever - Updates - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/updates/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#additional packages that may be useful +[extras] +name=CentOS-$releasever - Extras - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/extras/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#additional packages that extend functionality of existing packages +[centosplus] +name=CentOS-$releasever - Plus - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/centosplus/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus +gpgcheck=0 +enabled=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add new job yaml for lmbench3 performance test
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3-performance.yaml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 jobs/lmbench3-performance.yaml diff --git a/jobs/lmbench3-performance.yaml b/jobs/lmbench3-performance.yaml new file mode 100644 index 00000000..63af5866 --- /dev/null +++ b/jobs/lmbench3-performance.yaml @@ -0,0 +1,26 @@ +suite: lmbench3 +testcase: lmbench3 +category: benchmark +os: openeuler +os_version: 20.03 +os_arch: aarch64 + +testbox: vm-2p8g-1 +queue: vm-2p8g~baijing + +test_memory_size: 4096 +on_fail: + sleep: + 300 +# Test mdoe all,hardware,os were commented temporally,since it will take very long time +# more than 2 hours on test machine lkp-bdw-ep4. +# lmbench3: +# - mode: all +# - mode: hardware +# - mode: os + +nr_threads: +- 1 + +lmbench3: + mode: all -- 2.23.0

3 4

[PATCH compass-ci] block_helper.cr: simplify if block to one line
by Xu Xijian 19 Nov '20

19 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/block_helper.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/lib/block_helper.cr b/src/lib/block_helper.cr index 37ad367..596ef7e 100644 --- a/src/lib/block_helper.cr +++ b/src/lib/block_helper.cr @@ -28,9 +28,7 @@ class BlockHelper @block_helper[uuid] = fiber end - if yield == true - spawn fiber.run - end + spawn fiber.run if yield == true until fiber.dead? Fiber.yield -- 2.23.0

1 0

[PATCH lkp-tests 2/3] create a multi-level directory
by Hu Xuejiao 19 Nov '20

19 Nov '20

[why] Because after using "rspec spec/submit_spec.rb", i need the output of different case.yaml to be placed in output (spec/submit/merge_yaml/*.yaml/output) Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/submit | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index 9bba1018..050e3d44 100755 --- a/sbin/submit +++ b/sbin/submit @@ -9,6 +9,7 @@ require "#{LKP_SRC}/lib/distro_info" require "#{LKP_SRC}/lib/scheduler_client" require 'optparse' require 'yaml' +require 'fileutils' opt_set_key_value = {} opt_cmdline_defaults = {} @@ -35,7 +36,7 @@ options = OptionParser.new do |opts| exit 1 end opt_output_dir = dir - Dir.mkdir(dir) unless File.directory? dir + FileUtils.mkdir_p(dir) unless File.directory? dir end opts.on('-a', '--auto-define-files', 'auto add define_files') do -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/cci-makepkg: simplify centos7 test
by Lin Jiaxin 19 Nov '20

19 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 905ad689..e49f6699 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -90,20 +90,20 @@ update_shared_pkg() centos_update_gcc() { - if grep -sqF "CentOS Linux release 7" /etc/centos-release; then - [ "$os_mount" = "initramfs" ] && { - yum install -y centos-release-scl-rh - yum install -y "devtoolset-8-gcc*" - } - source /opt/rh/devtoolset-8/enable - fi + grep -sqF "CentOS Linux release 7" /etc/centos-release || return + + [ "$os_mount" = "initramfs" ] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable } [ -n "$PKGBUILD_TAG" ] && download_pkgfile distro_install_depends lkp-dev -[ "$os" = "centos" ] && centos_update_gcc +centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

3 3

[PATCH compass-ci] container/result-cifs: add the container to share /srv/result
by Xiao Shenwei 19 Nov '20

19 Nov '20

[why] our os-cifs container will mount three dir, /srv/os, /srv/initrd, /srv/result in order to reduce the coupling, need split them [how] one folder mount by one container [usage] mount -t cifs o guest,port=447 //ip/result /tmp/result Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/result-cifs/Dockerfile | 22 ++++++++++++++++++++++ container/result-cifs/build | 5 +++++ container/result-cifs/smb.conf | 29 +++++++++++++++++++++++++++++ container/result-cifs/start | 24 ++++++++++++++++++++++++ 4 files changed, 80 insertions(+) create mode 100644 container/result-cifs/Dockerfile create mode 100755 container/result-cifs/build create mode 100644 container/result-cifs/smb.conf create mode 100755 container/result-cifs/start diff --git a/container/result-cifs/Dockerfile b/container/result-cifs/Dockerfile new file mode 100644 index 0000000..a0d47c2 --- /dev/null +++ b/container/result-cifs/Dockerfile @@ -0,0 +1,22 @@ +# Origin: https://github.com/Stanback/alpine-samba +# Copyright (C) 2016-2020 Eric D. Stanback +# SPDX-License-Identifier: GPL-3.0 + +FROM alpine:edge + +MAINTAINER Xiao Shenwei <xiaoshenwei96(a)163.com> + +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories +RUN adduser -u 1090 -D lkp +RUN apk add --update \ + samba-common-tools \ + samba-client \ + samba-server \ + bash +RUN rm -rf /var/cache/apk/* + +COPY ./smb.conf /etc/samba/ + +EXPOSE 447/tcp + +ENTRYPOINT ["smbd", "--foreground", "--no-process-group", "--log-stdout"] diff --git a/container/result-cifs/build b/container/result-cifs/build new file mode 100755 index 0000000..d97b6dd --- /dev/null +++ b/container/result-cifs/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t result-cifs . diff --git a/container/result-cifs/smb.conf b/container/result-cifs/smb.conf new file mode 100644 index 0000000..4ccd943 --- /dev/null +++ b/container/result-cifs/smb.conf @@ -0,0 +1,29 @@ +# refer to https://lkml.org/lkml/2019/7/16/716 and https://lkml.org/lkml/2019/9/19/586 +[global] + workgroup = MYGROUP + server string = Samba Server + map to guest = Bad User + load printers = no + printing = bsd + printcap name = /dev/null + disable spoolss = yes + disable netbios = yes + server role = standalone + server services = -dns, -nbt + smb ports = 445 + create mode = 0777 + directory mode = 0777 + guest only = yes + guest ok = yes + server min protocol = NT1 + unix extensions = yes + mangled names = no + +[result] + path = /srv/result/ + comment = result + browseable = yes + writable = yes + public = yes + force user = lkp + force group = lkp diff --git a/container/result-cifs/start b/container/result-cifs/start new file mode 100755 index 0000000..2d6acba --- /dev/null +++ b/container/result-cifs/start @@ -0,0 +1,24 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +lsmod | grep -q "^cifs\s" || { + sudo modprobe cifs +} + +docker_rm result-cifs + +cmd=( + docker run + -d + -p 447:445 + -v /etc/localtime:/etc/localtime:ro + -v /srv/result:/srv/result + --name result-cifs + --restart=always + result-cifs +) + +"${cmd[@]}" -- 2.23.0

2 2