November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] container/rabbitmq: separate docker pull and run
by Liu Yinsi 19 Nov '20

19 Nov '20

"docker pull" in build script, "docker run" in start script looks more reasonable. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/rabbitmq/build | 5 +++++ container/rabbitmq/start | 1 - 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100755 container/rabbitmq/build diff --git a/container/rabbitmq/build b/container/rabbitmq/build new file mode 100755 index 0000000..c0bc179 --- /dev/null +++ b/container/rabbitmq/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker pull rabbitmq:3-management diff --git a/container/rabbitmq/start b/container/rabbitmq/start index 7e25417..6b875f3 100755 --- a/container/rabbitmq/start +++ b/container/rabbitmq/start @@ -5,7 +5,6 @@ . $CCI_SRC/container/defconfig.sh docker_rm rabbitmq -docker pull rabbitmq:3-management cmd=( docker run -- 2.23.0

2 1

[PATCH lkp-tests] jobs: add new job yaml for unixbench performance tuning
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/unixbench-tuning.yaml | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 jobs/unixbench-tuning.yaml diff --git a/jobs/unixbench-tuning.yaml b/jobs/unixbench-tuning.yaml new file mode 100644 index 00000000..d18e0dfc --- /dev/null +++ b/jobs/unixbench-tuning.yaml @@ -0,0 +1,10 @@ +suite: unixbench-tuning +category: benchmark + +runtime: 300s + +nr_task: + - 1 + - 96 + +unixbench-tuning: -- 2.23.0

1 0

[PATCH lkp-tests] stats/sysbench-mariadb: fix rubocop offenses
by Zhang Yale 19 Nov '20

19 Nov '20

[errors] Inspecting 1 file C offenses: sysbench-mariadb:4:1: C: Layout/IndentationWidth: Use 2 (not 4) spaces for indentation. case line.chomp! ^^^^ 1 file inspected, 1 offense detected Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/sysbench-mariadb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/stats/sysbench-mariadb b/stats/sysbench-mariadb index 0c7ba1d9..0bed3bc0 100755 --- a/stats/sysbench-mariadb +++ b/stats/sysbench-mariadb @@ -1,10 +1,10 @@ #!/usr/bin/env ruby while (line = STDIN.gets) - case line.chomp! - when /transactions:\s+(\d+)/ - puts "transactions: #{$1}" - when /95th percentile:\s+(\S+)/ - puts "95th percentile: #{$1}" - end + case line.chomp! + when /transactions:\s+(\d+)/ + puts "transactions: #{$1}" + when /95th percentile:\s+(\S+)/ + puts "95th percentile: #{$1}" + end end -- 2.23.0

3 5

[PATCH lkp-tests] add the case that on_fail is nil
by Hu Xuejiao 19 Nov '20

19 Nov '20

[why] When sh_on_fail is nil, on_fail will add to the job.yaml Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/submit | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sbin/submit b/sbin/submit index a325e912..9bba1018 100755 --- a/sbin/submit +++ b/sbin/submit @@ -121,11 +121,14 @@ ARGV.each do |jobfile| 'job2sh' => { 'run_job' => sh_run_job, 'extract_stats' => sh_extract_stats, - 'define_files' => sh_define_files, - 'on_fail' => sh_on_fail + 'define_files' => sh_define_files } } + if sh_on_fail + sh_hash['job2sh']['on_fail'] = sh_on_fail + end + # merge job info job_hash = job.to_hash job_hash_list << job_hash -- 2.23.0

2 1

[PATCH v2 compass-ci] sbin/es-jobs-mapping.sh: disable default dynamic mapping
by Zhang Yuhang 19 Nov '20

19 Nov '20

[why] By default, fields can be added dynamically to a document, or to inner objects within a document, just by indexing a document containing the new field. However, there are many fields that we do not need, and "Limit of total fields [1000] in index [jobs] has been exceeded" exception will be reported when the number of fields exceeds 1000. [how] Disable default dynamic mapping. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- sbin/es-jobs-mapping.sh | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sbin/es-jobs-mapping.sh b/sbin/es-jobs-mapping.sh index 9d420f7..076bdd3 100755 --- a/sbin/es-jobs-mapping.sh +++ b/sbin/es-jobs-mapping.sh @@ -38,6 +38,7 @@ else curl -H 'Content-Type: Application/json' -XPUT 'http://localhost:9200/jobs' -d '{ "mappings": { "_doc": { + "dynamic": false, "dynamic_templates": [ { "pp": { @@ -61,6 +62,10 @@ else } ], "properties": { + "pp": { + "dynamic": true, + "properties": {} + }, "suite": { "type": "keyword" }, -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats/sysbench-mariadb: fix rubocop offenses
by Zhang Yale 19 Nov '20

19 Nov '20

[errors] Inspecting 1 file C offenses: sysbench-mariadb:4:1: C: Layout/IndentationWidth: Use 2 (not 4) spaces for indentation. case line.chomp! ^^^^ 1 file inspected, 1 offense detected Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/sysbench-mariadb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/stats/sysbench-mariadb b/stats/sysbench-mariadb index 0c7ba1d9..54a31be0 100755 --- a/stats/sysbench-mariadb +++ b/stats/sysbench-mariadb @@ -1,10 +1,10 @@ #!/usr/bin/env ruby while (line = STDIN.gets) - case line.chomp! - when /transactions:\s+(\d+)/ - puts "transactions: #{$1}" - when /95th percentile:\s+(\S+)/ - puts "95th percentile: #{$1}" - end + case line + when /transactions:\s+(\d+)/ + puts "transactions: #{$1}" + when /95th percentile:\s+(\S+)/ + puts "95th percentile: #{$1}" + end end -- 2.23.0

1 0

[PATCH v8 compass-ci] make the container/initrd-cifs to share /srv/initrd
by Xiao Shenwei 19 Nov '20

19 Nov '20

[why] our os-cifs container will mount three dir, /srv/os, /srv/initrd, /srv/result in order to reduce the coupling, need split them [how] one folder mount by one container [usage] two ways for the client to mount initrd: 1. initrd mount -t cifs -o guest,port=446 //ip/initrd /tmp/initrd 2. osimage: for lkp compatibility mount -t cifs -o guest,port=446 //ip/osimage /tmp/osimage Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/initrd-cifs/Dockerfile | 22 ++++++++++++++++++ container/initrd-cifs/build | 5 +++++ container/initrd-cifs/smb.conf | 38 ++++++++++++++++++++++++++++++++ container/initrd-cifs/start | 24 ++++++++++++++++++++ 4 files changed, 89 insertions(+) create mode 100644 container/initrd-cifs/Dockerfile create mode 100755 container/initrd-cifs/build create mode 100644 container/initrd-cifs/smb.conf create mode 100755 container/initrd-cifs/start diff --git a/container/initrd-cifs/Dockerfile b/container/initrd-cifs/Dockerfile new file mode 100644 index 0000000..c2ec808 --- /dev/null +++ b/container/initrd-cifs/Dockerfile @@ -0,0 +1,22 @@ +# Origin: https://github.com/Stanback/alpine-samba +# Copyright (C) 2016-2020 Eric D. Stanback +# SPDX-License-Identifier: GPL-3.0 + +FROM alpine:edge + +MAINTAINER Xiao Shenwei <xiaoshenwei96(a)163.com> + +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories +RUN adduser -u 1090 -D lkp +RUN apk add --update \ + samba-common-tools \ + samba-client \ + samba-server \ + bash +RUN rm -rf /var/cache/apk/* + +COPY ./smb.conf /etc/samba/ + +EXPOSE 446/tcp + +ENTRYPOINT ["smbd", "--foreground", "--no-process-group", "--log-stdout"] diff --git a/container/initrd-cifs/build b/container/initrd-cifs/build new file mode 100755 index 0000000..4b93647 --- /dev/null +++ b/container/initrd-cifs/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t initrd-cifs . diff --git a/container/initrd-cifs/smb.conf b/container/initrd-cifs/smb.conf new file mode 100644 index 0000000..eaddef7 --- /dev/null +++ b/container/initrd-cifs/smb.conf @@ -0,0 +1,38 @@ +# refer to https://lkml.org/lkml/2019/7/16/716 and https://lkml.org/lkml/2019/9/19/586 +[global] + workgroup = MYGROUP + server string = Samba Server + map to guest = Bad User + load printers = no + printing = bsd + printcap name = /dev/null + disable spoolss = yes + disable netbios = yes + server role = standalone + server services = -dns, -nbt + smb ports = 445 + create mode = 0777 + directory mode = 0777 + guest only = yes + guest ok = yes + server min protocol = NT1 + unix extensions = yes + mangled names = no + +[initrd] + path = /srv/initrd/ + comment = initrd + browseable = yes + writable = yes + public = yes + force user = lkp + force group = lkp + +[osimage] + path = /srv/initrd/ + comment = osimage + browseable = yes + writable = yes + public = yes + force user = lkp + force group = lkp diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start new file mode 100755 index 0000000..77558a1 --- /dev/null +++ b/container/initrd-cifs/start @@ -0,0 +1,24 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +lsmod | grep -q "^cifs\s" || { + sudo modprobe cifs +} + +docker_rm initrd-cifs + +cmd=( + docker run + -d + -p 446:445 + -v /etc/localtime:/etc/localtime:ro + -v /srv/initrd:/srv/initrd + --name initrd-cifs + --restart=always + initrd-cifs +) + +"${cmd[@]}" -- 2.23.0

2 2

[PATCH compass-ci] sched: fix cast from Nil to Array failed
by Cao Xueliang 19 Nov '20

19 Nov '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/jobfile_operate.cr | 1 + 1 file changed, 1 insertion(+) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 16a4d06..98bf011 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -85,6 +85,7 @@ module Jobfile::Operate job_sh_array = [] of JSON::Any tmp_job_sh_content.as_h.each do |_key, val| + next if val == nil job_sh_array += val.as_a end else -- 2.23.0

1 0

[PATCH lkp-tests] tests/cci-makepkg: simplify centos7 test
by Lin Jiaxin 19 Nov '20

19 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 905ad689..8b72f395 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -90,20 +90,21 @@ update_shared_pkg() centos_update_gcc() { - if grep -sqF "CentOS Linux release 7" /etc/centos-release; then - [ "$os_mount" = "initramfs" ] && { - yum install -y centos-release-scl-rh - yum install -y "devtoolset-8-gcc*" - } - source /opt/rh/devtoolset-8/enable - fi + [ "$os" = "centos" ] || return + grep -sqF "CentOS Linux release 7" /etc/centos-release || return + + [ "$os_mount" = "initramfs" ] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable } [ -n "$PKGBUILD_TAG" ] && download_pkgfile distro_install_depends lkp-dev -[ "$os" = "centos" ] && centos_update_gcc +centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

3 2

[PATCH lkp-tests 6/6] iso2rootfs: allow the use of job.yaml and test script from the internet
by Yu Chuan 19 Nov '20

19 Nov '20

[Why] Not all the yaml and script of test job will be committed to the lkp-tests repo. [How] Support three optional fields for every test: - testX_git_url: the url of git repo which contains yaml and script; - testX_git_yaml: the relative path of the yaml file in the git repo; - testX_git_script: the relative path of the test script in the git repo; Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 5 +++++ tests/iso2rootfs | 25 +++++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 3c69a9d98430..c9a31cd0ad00 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -22,3 +22,8 @@ iso2rootfs: test1_yaml: test1_os_mount: test1_testbox: + ## follow three fields is required when your job.yaml and script for + ## this test are from the internet. + test1_git_url: + test1_git_yaml: + test1_git_script: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index a38685118b7f..1ba2e8a31ef8 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -447,10 +447,35 @@ generate_submit_append_str() echo "os=${iso_os} os_arch=${iso_arch} os_version=${iso_version} os_mount=$1 testbox=$2" } +prepare_submit() +{ + local submit_dir="/tmp/iso2rootfs/submit_tmp" + [ -d "${submit_dir}" ] || mkdir -p "${submit_dir}" + local submit_git_dir="${submit_dir}/${test_git_url##*/}" + submit_git_dir="${submit_git_dir%.git}" + + [ -d "${submit_git_dir}" ] || { + cd "${submit_dir}" + git clone "${test_git_url}" + } + + local test_git_yaml="${submit_git_dir}/$(eval echo "\$${test_num}_git_yaml")" + local test_git_script="${submit_git_dir}/$(eval echo "\$${test_num}_git_script")" + + [ -f "${test_git_yaml}" ] || die "cannot find file: ${test_num}_git_yaml." + cp "${test_git_yaml}" "${LKP_SRC}/jobs/" + + [ -f "${test_git_script}" ] || return 0 + cp "${test_git_script}" "${LKP_SRC}/tests/" +} + submit_one_yaml() { log_info "starting submit ${test_yaml} ..." + local test_git_url="$(eval echo "\$${test_num}_git_url")" + [ -z "${test_git_url}" ] || prepare_submit + cd "${LKP_SRC}/jobs" [ -f "${test_yaml}" ] || die "cannot find test yaml in LKP_SRC/jobs. -- 2.23.0

3 6