October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v6 lkp-tests 2/2] tests/iozone: add specified parameter for iozone test
by Lu Kaiyi 30 Oct '20

30 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for different block_size. [how] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..db6a588f 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,13 +1,48 @@ #!/bin/sh - +# - block_size +# - file_size +# - write_rewrite +# - read_reread +# - random_read_write +# - read_backwards +# - rewrite_record +# - stride_read +# - fwrite_refwrite +# - fread_refread +# - random_mix +# - pwrite_repwrite +# - pread_repread +# - pwritev_repwritev +# - preadv_repreadv ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh +cmd="iozone" +if [ -n "$block_size" ]; then + cmd+=" -r $block_size" + [ -n "$file_size" ] && cmd+=" -s $file_size" + [ -n "$write_rewrite" ] && cmd+=" -i 0" + [ -n "$read_reread" ] && cmd+=" -i 1" + [ -n "$random_read_write" ] && cmd+=" -i 2" + [ -n "$read_backwards" ] && cmd+=" -i 3" + [ -n "$rewrite_record" ] && cmd+=" -i 4" + [ -n "$stride_read" ] && cmd+=" -i 5" + [ -n "$fwrite_refwrite" ] && cmd+=" -i 6" + [ -n "$fread_refread" ] && cmd+=" -i 7" + [ -n "$random_mix" ] && cmd+=" -i 8" + [ -n "$pwrite_repwrite" ] && cmd+=" -i 9" + [ -n "$pread_repread" ] && cmd+=" -i 10" + [ -n "$pwritev_repwritev" ] && cmd+=" -i 11" + [ -n "$preadv_repreadv" ] && cmd+=" -i 12" +else + cmd+=" -a" +fi +cmd+=" -f $mnt/iozonetest" + for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH v6 lkp-tests 1/2] jobs/iozone_with_block_size.yaml: add iozone_with_block_size.yaml
by Lu Kaiyi 30 Oct '20

30 Oct '20

[why] need test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 jobs/iozone-bs.yaml diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml new file mode 100644 index 00000000..868979fc --- /dev/null +++ b/jobs/iozone-bs.yaml @@ -0,0 +1,19 @@ +suite: iozone +category: benchmark + +file_size: 4g +write_rewrite: true +read_reread: true +random_read_write: true + +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m + +disk: 1HDD +fs: xfs +iosched: kyber +iozone: -- 2.23.0

1 0

[PATCH v7 compass-ci 5/5] container/ntp-server: start
by Wang Chenglong 30 Oct '20

30 Oct '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/start | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100755 container/ntp-server/start diff --git a/container/ntp-server/start b/container/ntp-server/start new file mode 100755 index 0000000..d5e3394 --- /dev/null +++ b/container/ntp-server/start @@ -0,0 +1,16 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +docker_rm ntp-server +cmd=( + docker run + -id + --name=ntp-server + -p 123:123/udp + ntp-server +) + +"${cmd[@]}" -- 2.23.0

1 0

[PATCH v7 compass-ci 4/5] container/ntp-server: ntp.conf
by Wang Chenglong 30 Oct '20

30 Oct '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/ntp.conf | 60 +++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 container/ntp-server/ntp.conf diff --git a/container/ntp-server/ntp.conf b/container/ntp-server/ntp.conf new file mode 100644 index 0000000..67dd1b0 --- /dev/null +++ b/container/ntp-server/ntp.conf @@ -0,0 +1,60 @@ +# For more information about this file, see the ntp.conf(5) man page. + +# Record the frequency of the system clock. +driftfile /var/lib/ntp/drift + +# Permit time synchronization with our time source, but do not +# permit the source to query or modify the service on this system. +restrict default nomodify notrap nopeer noepeer noquery + +# Permit association with pool servers. +restrict source nomodify notrap noepeer noquery + +# Permit all access over the loopback interface. This could +# be tightened as well, but to do so would effect some of +# the administrative functions. +restrict 127.0.0.1 +restrict ::1 + +# Hosts on local network are less restricted. +restrict 172.0.0.0 mask 255.0.0.0 nomodify notrap + +# Use public servers from the pool.ntp.org project. +# Please consider joining the pool (http://www.pool.ntp.org/join.html) +# pool 2.openEuler.pool.ntp.org iburst +server 0.centos.pool.ntp.org +server 0.asia.pool.ntp.org +server ntp.sjtu.edu.cn +server ntp1.aliyun.com + +# When all ntp servers on Internet are inaccessible, it will use +# it's local time to give service to the ntp clients. +# If not set, the ntp server will go to stratum 16 and cannot serve +# any ntp clients when it loses connection with upstream ntp servers. +# This is common in a LAN environment isolated with Internet. +server 127.127.1.0 +fudge 127.127.1.0 stratum 8 + +# Reduce the maximum number of servers used from the pool. +tos maxclock 5 + +# Enable public key cryptography. +# crypto + +includefile /etc/ntp/crypto/pw + +# Key file containing the keys and key identifiers used when operating +# with symmetric key cryptography. +keys /etc/ntp/keys + +# Specify the key identifiers which are trusted. +# trustedkey 4 8 42 + +# Specify the key identifier to use with the ntpdc utility. +# requestkey 8 + +# Specify the key identifier to use with the ntpq utility. +# controlkey 8 + +# Enable writing of statistics records. +# statistics clockstats cryptostats loopstats peerstats -- 2.23.0

1 0

[PATCH v7 compass-ci 3/5] container/ntp-server: build
by Wang Chenglong 30 Oct '20

30 Oct '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/ntp-server/build diff --git a/container/ntp-server/build b/container/ntp-server/build new file mode 100755 index 0000000..7826fb2 --- /dev/null +++ b/container/ntp-server/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t ntp-server . -- 2.23.0

1 0

[PATCH v3 compass-ci] sparrow/4-docker/buildall: don't skip sub-fluentd
by Wu Zhende 30 Oct '20

30 Oct '20

[Why] Now, primary server also need to startup sub-fluentd. It collects Docker logs and VM serial port logs on the server. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/4-docker/buildall | 1 - 1 file changed, 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 6e0a2f8..eb24500 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -30,7 +30,6 @@ do_one() mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container - [ "$container_name" == 'sub-fluentd' ] && exit if [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install -- 2.23.0

1 0

Re: [PATCH v2 compass-ci] sparrow/4-docker/buildall: container sub-fluentd don't exit
by Li Ping 30 Oct '20

30 Oct '20

dont't ==> doesn't Reply-To: In-Reply-To: <20201030042516.1237314-1-wuzhende666(a)163.com> On Fri, Oct 30, 2020 at 12:25:16PM +0800, Wu Zhende wrote: >[Why] > Now, primary server also need to startup sub-fluentd. > It collects Docker logs and VM serial port logs on the server. > >Signed-off-by: Wu Zhende <wuzhende666(a)163.com> >--- > sparrow/4-docker/buildall | 1 - > 1 file changed, 1 deletion(-) > >diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall >index 6e0a2f8..eb24500 100755 >--- a/sparrow/4-docker/buildall >+++ b/sparrow/4-docker/buildall >@@ -30,7 +30,6 @@ do_one() > mkdir $tmpdir/$container_name 2>/dev/null && > ( > cd $container >- [ "$container_name" == 'sub-fluentd' ] && exit > if [ "$action" != "reboot" ]; then > [ -x build ] && ./build > [ -x install ] && ./install >-- >2.23.0 >

1 0

[PATCH v2 compass-ci] sparrow/4-docker/buildall: container sub-fluentd don't exit
by Wu Zhende 30 Oct '20

30 Oct '20

[Why] Now, primary server also need to startup sub-fluentd. It collects Docker logs and VM serial port logs on the server. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/4-docker/buildall | 1 - 1 file changed, 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 6e0a2f8..eb24500 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -30,7 +30,6 @@ do_one() mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container - [ "$container_name" == 'sub-fluentd' ] && exit if [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install -- 2.23.0

1 0

[PATCH v4 lkp-tests] hackbench/PKGBUILD: fix source and build function
by Zhang Yale 30 Oct '20

30 Oct '20

[Why] Old source "https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-1.8.tar.gz" is not available, and need build before package. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- pkg/hackbench/PKGBUILD | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/pkg/hackbench/PKGBUILD b/pkg/hackbench/PKGBUILD index 797a00e6..94049599 100644 --- a/pkg/hackbench/PKGBUILD +++ b/pkg/hackbench/PKGBUILD @@ -2,13 +2,19 @@ pkgname=hackbench pkgver=1.8 pkgrel=1 arch=('i386' 'x86_64') -url="https://www.kernel.org/pub/linux/utils/rt-tests" +url="https://github.com/lacombar/hackbench.git" license=('GPL') -source=("https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-${pkgver}.tar.gz") +source=("https://github.com/lacombar/hackbench.git") sha256sums=('SKIP') -package() { +build() +{ + cd "${srcdir}/${pkgname}/${pkgname}" || exit + gcc -o hackbench hackbench.c -lpthread +} + +package() +{ mkdir -p "${pkgdir}/lkp/benchmarks/${pkgname}" - cd "$srcdir/rt-tests-${pkgver}" - make install DESTDIR="${pkgdir}/lkp/benchmarks/${pkgname}" + cp -rf "${srcdir}/${pkgname}/${pkgname}" "${pkgdir}/lkp/benchmarks/${pkgname}" } -- 2.23.0

4 5

[PATCH v3 compass-ci] save hostname and queues relation in redis
by Xiao Shenwei 30 Oct '20

30 Oct '20

[why] the test machine has the ability to know which queues should be requested [how] for VM HW, we will register mac2hostname and hostname2queues. then they can find queues based on mac. API: boot.ipxe/mac/${mac} for docker, we will register hostname2queues, then container can find queues based on hostname. API: boot.container/hostname/${hostname} Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 8 ++++++++ src/scheduler/scheduler.cr | 24 ++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index b2ef6ce..80cf0b0 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -38,6 +38,14 @@ class Sched @redis.hash_del("sched/mac2host", normalize_mac(mac)) end + def set_host2queues(hostname : String, queues : String) + @redis.hash_set("sched/host2queues", hostname, queues) + end + + def del_host2queues(hostname : String) + @redis.hash_del("sched/host2queues", hostname) + end + # return: # Hash(String, Hash(String, String)) def get_cluster_state(cluster_id) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 371e4ed..a946ea0 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -113,6 +113,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/set_host2queues?queues=vm-2p8g.aarch64&host=vm-2p8g.a…" + put "/set_host2queues" do |env| + env.response.headers["Connection"] = "close" + if (client_queues = env.params.query["queues"]?) && (client_host = env.params.query["host"]?) + sched.set_host2queues(client_host, client_queues) + + "Done" + else + "No yet" + end + end + # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| env.response.headers["Connection"] = "close" @@ -125,6 +137,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/del_host2queues?host=vm-2p8g.aarch64" + put "/del_host2queues" do |env| + env.response.headers["Connection"] = "close" + if client_host = env.params.query["host"]? + sched.del_host2queues(client_host) + + "Done" + else + "No yet!" + end + end + # client(runner) report job's status # /~lkp/cgi-bin/lkp-jobfile-append-var # ?job_file=/lkp/scheduled/job.yaml&job_state=running&job_id=10 -- 2.23.0

2 2