- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 3/3] add the case of yaml_merge_included_files.yaml
by Hu Xuejiao 19 Nov '20

19 Nov '20

[description] it is situation of yaml_merge_included_files Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 3 +++ spec/submit/merge_yaml/yaml_merge_included_files.yaml | 1 + 2 files changed, 4 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..0415606a --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,3 @@ +suite: test +category: functional +test: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml -- 2.23.0

3 2

[PATCH lkp-tests 1/3] create merge_yaml and submit merge_yaml/*.yaml
by Hu Xuejiao 19 Nov '20

19 Nov '20

[why] We will add more spec cases, so it should give each type of cases for different input and output directories [how] Create merge_yaml and submit merge_yaml/*.yaml Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit_spec.rb | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 983d050e..b2ad7168 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -34,6 +34,21 @@ describe 'submit job spec' do ] system(*submit_cmd) traverse_file(output_dir) + end + end + it 'merge yaml spec' do + Dir.glob("#{LKP_SRC}/spec/submit/merge_yaml/*.yaml").each do |yaml_file| + job_name = File.basename(yaml_file, '.yaml') + output_dir = "#{LKP_SRC}/spec/submit/merge_yaml/#{job_name}/output" + submit_cmd = [ + "#{LKP_SRC}/sbin/submit", + '-o', output_dir, + '-s', 'lab: spec_lab', + '-s', 'testbox: vm-2p8g--spec_submit', + yaml_file + ] + system(*submit_cmd) + traverse_file(output_dir) + end end - end end -- 2.23.0

2 1

[PATCH lkp-tests] jobs: fix the spell error in the original lmbench3.ayml
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/lmbench3.yaml b/jobs/lmbench3.yaml index de1c9aa8..f84c6910 100644 --- a/jobs/lmbench3.yaml +++ b/jobs/lmbench3.yaml @@ -4,7 +4,7 @@ category: benchmark test_memory_size: 50% -# Test mdoe all,hardware,os were commented temporally,since it will take very long time +# Test mode all, hardware, os were commented temporally, since it will take very long time # more than 2 hours on test machine lkp-bdw-ep4. # lmbench3: # - mode: all -- 2.23.0

2 2

[PATCH v2 lkp-tests] jobs: add new job yaml for lmbench3 performance test
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3-performance.yaml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 jobs/lmbench3-performance.yaml diff --git a/jobs/lmbench3-performance.yaml b/jobs/lmbench3-performance.yaml new file mode 100644 index 00000000..c733d7d2 --- /dev/null +++ b/jobs/lmbench3-performance.yaml @@ -0,0 +1,17 @@ +suite: lmbench3 +category: benchmark + +test_memory_size: 4096 + +# Test mdoe all,hardware,os were commented temporally,since it will take very long time +# more than 2 hours on test machine lkp-bdw-ep4. +# lmbench3: +# - mode: all +# - mode: hardware +# - mode: os + +nr_threads: +- 1 + +lmbench3: + mode: all -- 2.23.0

4 6

[PATCH compass-ci] web_backend.rb: add space line after return cmd
by Xu Xijian 19 Nov '20

19 Nov '20

When there is a return cmd in a function, and it's not the last line of this function, there should have a blank line after return cmd. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/web_backend.rb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index b8df6f4..1f7224e 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -220,7 +220,9 @@ end def wrong_size?(size, from) return true if from.negative? || size.negative? + return true if from > 1000000 || size > 1000000 + return true if size + from > 1000000 end @@ -344,6 +346,7 @@ def get_repo_url(urls) urls.each do |url| return url if url[0, 4] == 'http' end + urls[0] end -- 2.23.0

1 0

[PATCH v3 compass-ci] sparrow: fix 3-code/dev-env no such file error
by Liu Yinsi 19 Nov '20

19 Nov '20

when execute 3-code/dev-env, error message as follows: ./dev-env: line 45: ./os/openEuler: No such file or directory Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 77027d9..98da280 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -42,4 +42,5 @@ EOF source /etc/os-release -. $(dirname ${BASH_SOURCE[0]})/os/${ID} +path=$(dirname ${BASH_SOURCE[0]})/os/${ID} +[ -x "$path" ] && . "$path" -- 2.23.0

3 5

[PATCH compass-ci] lib/log.rb: add new class for json log
by Wu Zhende 19 Nov '20

19 Nov '20

[Why] The built-in log system peints logs in string format by default. The logs should be formatted in json format for subsequent analysis and processing. [Example] require './log' log = Log.new use: log.info("tset") output: {"level":"INFO","level_num":1,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"test"} use: log.error({"job_id" => "1", "job_state" => "submit"}.to_json) output: {"level":"ERROR","level_num":3,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"", "job_id":"1","job_state":"submit","caller":["/usr/share/ruby/logger.rb:4:in `format_message'", "/usr/share/ruby/logger.rb:472:in`add'","/usr/share/ruby/logger.rb:545:in `error'","./test.rb:6:in`<main>'"]} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/log.rb | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 lib/log.rb diff --git a/lib/log.rb b/lib/log.rb new file mode 100644 index 0000000..0ffc0f9 --- /dev/null +++ b/lib/log.rb @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' +require 'json' + +# log class to print logs in JSON format +class Log < Logger + LEVEL_INFO = { + 'DEBUG' => 0, + 'INFO' => 1, + 'WARN' => 2, + 'ERROR' => 3, + 'FATAL' => 4, + 'UNKNOWN' => 5 + }.freeze + + FORMATTER = proc { |severity, datetime, progname, msg| + puts + level_num = LEVEL_INFO[severity] + begin + message = JSON.parse(msg) + rescue JSON::ParserError + message = { 'message' => msg } + end + h = { + 'level' => severity.to_s, + 'level_num' => level_num, + 'datetime' => datetime, + 'progname' => progname, + 'message' => '' + } + h.merge!(message) + h.merge!({ 'caller' => caller }) if level_num >= 2 + h.to_json + } + + def initialize(logdev = STDOUT, formatter = FORMATTER) + super(logdev, formatter: formatter) + end +end -- 2.23.0

2 1

[PATCH v2 compass-ci] lib/log.rb: add new class for json log
by Wu Zhende 19 Nov '20

19 Nov '20

[Why] The built-in log system peints logs in string format by default. The logs should be formatted in json format for subsequent analysis and processing. [Example] require './log' log = Log.new log.info("tset") output: {"level":"INFO","level_num":1,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"test"} log.error({"job_id" => "1", "job_state" => "submit"}.to_json) output: {"level":"ERROR","level_num":3,"datetime":"2020-11-19 16:57:17+0800","progname":null,"message":"", "job_id":"1","job_state":"submit","caller":["/usr/share/ruby/logger.rb:4:in `format_message'", "/usr/share/ruby/logger.rb:472:in`add'","/usr/share/ruby/logger.rb:545:in `error'","./test.rb:6:in`<main>'"]} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/log.rb | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 lib/log.rb diff --git a/lib/log.rb b/lib/log.rb new file mode 100644 index 0000000..0ffc0f9 --- /dev/null +++ b/lib/log.rb @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' +require 'json' + +# log class to print logs in JSON format +class Log < Logger + LEVEL_INFO = { + 'DEBUG' => 0, + 'INFO' => 1, + 'WARN' => 2, + 'ERROR' => 3, + 'FATAL' => 4, + 'UNKNOWN' => 5 + }.freeze + + FORMATTER = proc { |severity, datetime, progname, msg| + puts + level_num = LEVEL_INFO[severity] + begin + message = JSON.parse(msg) + rescue JSON::ParserError + message = { 'message' => msg } + end + h = { + 'level' => severity.to_s, + 'level_num' => level_num, + 'datetime' => datetime, + 'progname' => progname, + 'message' => '' + } + h.merge!(message) + h.merge!({ 'caller' => caller }) if level_num >= 2 + h.to_json + } + + def initialize(logdev = STDOUT, formatter = FORMATTER) + super(logdev, formatter: formatter) + end +end -- 2.23.0

2 1

[PATCH compass-ci] qcow2rootfs: change yum source to fix build timeout error
by Yu Chuan 19 Nov '20

19 Nov '20

[Error Msg] Step 4/7 : RUN yum makecache && yum install -y --skip-broken libvirt libguestfs-tools xz expect openssl && yum clean all ---> Running in c443a95dbd5f CentOS-8 - AppStream2.2 kB/s | 2.5 MB 19:23 Errors during downloading metadata for repository 'AppStream': - Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/ce3501f1421… [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds] - Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/cc0e75f5237… [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds] - Curl error (18): Transferred a partial file for http://mirror.centos.org/centos/8/AppStream/aarch64/os/repodata/ce3501f1421… [transfer closed with 1750223 bytes remaining to read] Error: Failed to download metadata for repo 'AppStream': Yum repo downloading error: Downloading error(s): repodata/cc0e75f5237c3f5ddfea6ab0de1056df5bb8bbf1273b5ecb907ed7c1c72897f3-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/ce3501f14219e71c21c1e40c6f91ba41bd37b59345c5a7dd6d680ef4b279b95a-filelists.xml.gz - Cannot download, all mirrors were already tried without success Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/qcow2rootfs/Dockerfile | 4 +- .../etc/yum.repos.d/CentOS-AltArch-7.repo | 43 +++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo diff --git a/container/qcow2rootfs/Dockerfile b/container/qcow2rootfs/Dockerfile index a0f8b042520b..38418c7eb226 100644 --- a/container/qcow2rootfs/Dockerfile +++ b/container/qcow2rootfs/Dockerfile @@ -1,12 +1,14 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM centos:8 +FROM centos:7 MAINTAINER Yu Chuan <13186087857(a)163.com>, Shi Zhichao <angus_robot(a)163.com> RUN rpm --import /etc/pki/rpm-gpg/RPM* +RUN rm -rf /etc/yum.repos.d/* +COPY root / RUN yum makecache && \ yum install -y --skip-broken libvirt libguestfs-tools xz expect openssl && \ yum clean all diff --git a/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo b/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo new file mode 100644 index 000000000000..b99c0a60895d --- /dev/null +++ b/container/qcow2rootfs/root/etc/yum.repos.d/CentOS-AltArch-7.repo @@ -0,0 +1,43 @@ +# CentOS-Base.repo +# +# The mirror system uses the connecting IP address of the client and the +# update status of each mirror to pick mirrors that are updated to and +# geographically close to the client. You should use this for CentOS updates +# unless you are manually picking other mirrors. +# +# If the mirrorlist= does not work for you, as a fall back you can try the +# remarked out baseurl= line instead. +# +# + +[base] +name=CentOS-$releasever - Base - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#released updates +[updates] +name=CentOS-$releasever - Updates - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/updates/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#additional packages that may be useful +[extras] +name=CentOS-$releasever - Extras - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/extras/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras +gpgcheck=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch + +#additional packages that extend functionality of existing packages +[centosplus] +name=CentOS-$releasever - Plus - mirrors.huaweicloud.com +baseurl=https://mirrors.huaweicloud.com/centos-altarch/7/centosplus/$basearch/ +#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus +gpgcheck=0 +enabled=0 +gpgkey=https://mirrors.huaweicloud.com/centos-altarch/7/os/$basearch/RPM-GPG-KEY-CentOS-7-$basearch -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add new job yaml for lmbench3 performance test
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3-performance.yaml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 jobs/lmbench3-performance.yaml diff --git a/jobs/lmbench3-performance.yaml b/jobs/lmbench3-performance.yaml new file mode 100644 index 00000000..63af5866 --- /dev/null +++ b/jobs/lmbench3-performance.yaml @@ -0,0 +1,26 @@ +suite: lmbench3 +testcase: lmbench3 +category: benchmark +os: openeuler +os_version: 20.03 +os_arch: aarch64 + +testbox: vm-2p8g-1 +queue: vm-2p8g~baijing + +test_memory_size: 4096 +on_fail: + sleep: + 300 +# Test mdoe all,hardware,os were commented temporally,since it will take very long time +# more than 2 hours on test machine lkp-bdw-ep4. +# lmbench3: +# - mode: all +# - mode: hardware +# - mode: os + +nr_threads: +- 1 + +lmbench3: + mode: all -- 2.23.0

3 4