November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 4/6] container/webhook: start
by Li Yuanchao 06 Nov '20

06 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/webhook/start | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 container/webhook/start diff --git a/container/webhook/start b/container/webhook/start new file mode 100755 index 0000000..373396a --- /dev/null +++ b/container/webhook/start @@ -0,0 +1,20 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative '../defconfig.rb' + +docker_rm "webhook" + +cmd = %w[ + docker run + --restart=always + --name webhook + -d + -p 11301:11301 + node:webhook +] + +cmd += ['sh', '-c', 'node webhook.js'] +system(*cmd) -- 2.23.0

1 0

[PATCH v3 compass-ci 3/6] container/webhook: build
by Li Yuanchao 06 Nov '20

06 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/webhook/build | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100755 container/webhook/build diff --git a/container/webhook/build b/container/webhook/build new file mode 100755 index 0000000..bb5c0a7 --- /dev/null +++ b/container/webhook/build @@ -0,0 +1,6 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t node:webhook . + -- 2.23.0

1 0

[PATCH v3 compass-ci 2/6] container/webhook: Dockerfile
by Li Yuanchao 06 Nov '20

06 Nov '20

add container webhook Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/webhook/Dockerfile | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 container/webhook/Dockerfile diff --git a/container/webhook/Dockerfile b/container/webhook/Dockerfile new file mode 100644 index 0000000..9299bb6 --- /dev/null +++ b/container/webhook/Dockerfile @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM node:10.21.0-alpine +MAINTAINER Li Yuanchao <lyc163mail(a)163.com> + +ADD root / + +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories + +RUN apk update && \ + apk upgrade && \ + apk add --no-cache git && \ + apk add npm ruby-dev + +RUN echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc + +RUN npm install git-webhook-handler && \ + gem install bunny + -- 2.23.0

1 0

[PATCH v3 compass-ci 1/6] git_mirror: add webhook handler
by Li Yuanchao 06 Nov '20

06 Nov '20

webhook will send a message when user push. try to find the git_repo when get url of the repository. if found, push it to queue. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 65d6940..9d6a45d 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -88,10 +88,16 @@ class MirrorMain @git_queue = Queue.new @es_client = Elasticsearch::Client.new(url: "http://#{ES_HOST}:#{ES_PORT}") load_fork_info + connection_init + handle_webhook + end + + def connection_init connection = Bunny.new('amqp://172.17.0.1:5672') connection.start channel = connection.create_channel @message_queue = channel.queue('new_refs') + @webhook_queue = channel.queue('web_hook') end def fork_stat_init(stat_key) @@ -287,3 +293,39 @@ class MirrorMain es_repo_update(git_repo) end end + +# main thread +class MirrorMain + def check_git_repo(git_repo, webhook_url) + return @git_info.key?(git_repo) && Array(@git_info[git_repo]['url'])[0] == webhook_url + end + + # example + # url: https://gitee.com/theprocess/oec-hardware git_repo: oec-hardware/oec-hardware + # url: https://github.com/berkeley-abc/abc git_repo: a/abc/abc + # url: https://github.com/Siguyi/AvxToNeon git_repo: AvxToNeon/Siguyi + def get_git_repo(webhook_url) + fork_name, project = webhook_url.split('/')[-2, 2] + + git_repo = "#{project}/#{project}" + return git_repo if check_git_repo(git_repo, webhook_url) + + git_repo = "#{project}/#{fork_name}" + return git_repo if check_git_repo(git_repo, webhook_url) + + git_repo = "#{project[0]}/#{project}/#{project}" + return git_repo if check_git_repo(git_repo, webhook_url) + + puts "webhook: #{webhook_url} is not found!" + end + + def handle_webhook + Thread.new do + @webhook_queue.subscribe(block: true) do |_delivery, _properties, webhook_url| + git_repo = get_git_repo(webhook_url) + do_push(git_repo) if git_repo + sleep(0.1) + end + end + end +end -- 2.23.0

1 0

[PATCH v4 lkp-tests] tests: add hackbench-git test script
by Zhang Yale 06 Nov '20

06 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] So need to add new test script for hackbench-git. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/hackbench-git | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 tests/hackbench-git diff --git a/tests/hackbench-git b/tests/hackbench-git new file mode 100755 index 00000000..cd133d9e --- /dev/null +++ b/tests/hackbench-git @@ -0,0 +1,37 @@ +#!/bin/sh +# - runtime +# - nr_threads +# - mode +# - ipc +# - iterations +# - loops + +## Hackbench-git is an old version, It is both a benchmark and a stress test for the Linux kernel scheduler. +## It's main job is to create a specified number of schedulable entities pairs. +## (either threads or traditional processes) which communicate via either sockets or pipes and time how long +## it takes for each pair to send data back and forth. + +[ "$ipc" = 'pipe' ] && ipc_option='-pipe' + +[ -n "$runtime" ] || runtime=600 +[ -n "$iterations" ] || iterations=10 +[ -n "$loops" ] || loops=100 +[ -n "$nr_threads" ] || nr_threads=1 + +run_hackbench() +{ + # args: groups number, mode [process/thread], times + local hackbench_path + if [ -x /lkp/benchmarks/hackbench-git/hackbench/hackbench ]; then + hackbench_path=/lkp/benchmarks/hackbench-git/hackbench/hackbench + else + echo "The hackbench executable file cannot be found." && exit 1 + fi + + echo "--------$ipc $mode num=$nr_threads--------" + for i in $(seq 1 $iterations); do + log_cmd $hackbench_path $ipc_option $nr_threads $mode $loops + done +} + +run_hackbench -- 2.23.0

2 4

[PATCH v1 lab-z9] cluster: add cluster config file
by Zhang Yu 06 Nov '20

06 Nov '20

About config filename: "cs-s1-a102-c1": - cs : cluster - s1 : one server node - a102 : server node name - c1 : one client node Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- cluster/cs-s1-a102-c1 | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 cluster/cs-s1-a102-c1 diff --git a/cluster/cs-s1-a102-c1 b/cluster/cs-s1-a102-c1 new file mode 100644 index 0000000..af57123 --- /dev/null +++ b/cluster/cs-s1-a102-c1 @@ -0,0 +1,10 @@ +switch: Switch-P10 +ip0: 1 +nodes: + taishan200-2280-2s48p-512g--a102: + roles: [ server ] + macs: [ "44:67:47:c9:ea:37" ] + + taishan200-2280-2s48p-256g--a101: + roles: [ client ] + macs: [ "44:67:47:d7:6c:a3" ] -- 2.23.0

1 0

[PATCH compass-ci 1/6] git_mirror: add webhook handler
by Li Yuanchao 06 Nov '20

06 Nov '20

webhook will send a message when user push. try to find the git_repo when get url of the repository. if found, push it to queue. if not found, it maybe an illegal request, do nothing Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 65d6940..11cf3f1 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -88,10 +88,16 @@ class MirrorMain @git_queue = Queue.new @es_client = Elasticsearch::Client.new(url: "http://#{ES_HOST}:#{ES_PORT}") load_fork_info + connection_init + handle_webhook + end + + def connection_init connection = Bunny.new('amqp://172.17.0.1:5672') connection.start channel = connection.create_channel @message_queue = channel.queue('new_refs') + @webhook_queue = channel.queue('web_hook') end def fork_stat_init(stat_key) @@ -287,3 +293,35 @@ class MirrorMain es_repo_update(git_repo) end end + +# main thread +class MirrorMain + def check_git_repo(git_repo, webhook_url) + return @git_info.key?(git_repo) && Array(@git_info[git_repo]['url'])[0] == webhook_url + end + + def get_git_repo(webhook_url) + strings = webhook_url.split('/') + project = strings[-1] + fork_name = strings[-2] + + git_repo = "#{project}/#{project}" + return git_repo if check_git_repo(git_repo, webhook_url) + + git_repo = "#{project}/#{fork_name}" + return git_repo if check_git_repo(git_repo, webhook_url) + + git_repo = "#{project[0]}/#{project}/#{project}" + return git_repo if check_git_repo(git_repo, webhook_url) + end + + def handle_webhook + Thread.new do + @webhook_queue.subscribe(block: true) do |_delivery, _properties, webhook_url| + git_repo = get_git_repo(webhook_url) + do_push(git_repo) if git_repo + sleep(0.1) + end + end + end +end -- 2.23.0

4 7

[PATCH v6 lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 06 Nov '20

06 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..026b97e2 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write,read,rand_rw block_size: - 64k -- 2.23.0

2 2

[PATCH lkp-tests] pkg: add nginx PKGBUILD
by Wei Jihui 06 Nov '20

06 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/nginx/PKGBUILD | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 pkg/nginx/PKGBUILD diff --git a/pkg/nginx/PKGBUILD b/pkg/nginx/PKGBUILD new file mode 100644 index 00000000..00b30118 --- /dev/null +++ b/pkg/nginx/PKGBUILD @@ -0,0 +1,23 @@ +pkgname=nginx +pkgver=1.14 +pkgrel=2 +arch=('i386' 'x86_64') +url="https://nginx.org" +license=('BSD') +source=("https://nginx.org/download/nginx-${pkgver}.${pkgrel}.tar.gz") +md5sums=('SKIP') + +build() +{ + benchmark_path="/lkp/benchmarks/${pkgname}" + mkdir -p "${benchmark_path}" + cd "${srcdir}/${pkgname}-${pkgver}.${pkgrel}" + ./configure --with-http_ssl_module --prefix="${benchmark_path}" + make +} + +package() +{ + cd "${srcdir}/${pkgname}-${pkgver}.${pkgrel}" + make install +} -- 2.23.0

1 0

[PATCH v6 lkp-tests 2/2] tests/iozone: change the way of parsing parameter
by Lu Kaiyi 06 Nov '20

06 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool also need change the way of parsing parameter. [how] change the way of parsing parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 66 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..b6a1e206 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,52 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + # reset IFS to split $test by "," and then restore default + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + case $ele in + "write") args+=" -i 0" + ;; + "read") args+=" -i 1" + ;; + "rand_rw") args+=" -i 2" + ;; + "backwards") args+=" -i 3" + ;; + "record") args+=" -i 4" + ;; + "stride") args+=" -i 5" + ;; + "fwrite") args+=" -i 6" + ;; + "fread") args+=" -i 7" + ;; + "rand_mix") args+=" -i 8" + ;; + "pwrite") args+=" -i 9" + ;; + "pread") args+=" -i 10" + ;; + "pwritev") args+=" -i 11" + ;; + "preadv") args+=" -i 12" + ;; + esac + done else args+=" -a" fi -- 2.23.0

1 0