- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 3/4] test/cci-makepkg: Refactor
by Sun Yukui 16 Nov '20

16 Nov '20

Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a105f189..da7050f3 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -92,10 +92,6 @@ cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy -echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf -PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf - update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return @@ -109,4 +105,11 @@ update_softlink() rm $sync_dest/$cgz_name } -update_softlink && echo "Makepkg finished successfully" +build_source_pkg() +{ + echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf +} + +build_source_pkg -- 2.23.0

2 1

[PATCH v2 compass-ci] remove actual ip in compass-ci code
by Xiao Shenwei 16 Nov '20

16 Nov '20

obtain the actual ip address from configuration file, Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- doc/develop-environment.md | 2 +- src/features/jobs/right_iperf.yaml | 2 +- src/scheduler/README.md | 2 +- user-client/helper/iperf.yaml | 2 +- user-client/jobs/iperf-pxe.yaml | 2 +- user-client/jobs/iperf-vm.yaml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..ca86e88 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -39,7 +39,7 @@ the returned data for setup_jumper_account_info like: { "account" => "guest", "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "jumper_ip" => "0.0.0.0", "jumper_port" => "10000" } diff --git a/doc/develop-environment.md b/doc/develop-environment.md index 8760878..ac408a2 100644 --- a/doc/develop-environment.md +++ b/doc/develop-environment.md @@ -4,7 +4,7 @@ cat >> ~/.ssh/config <<-EOF Host crystal - Hostname 124.90.34.227 + Hostname xxx.xxx.xxx.xxx Port 22113 EOF diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index f81af38..e239ccd 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/src/scheduler/README.md b/src/scheduler/README.md index fdc7027..8e3a197 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -6,7 +6,7 @@ - And set the environment variables needed for running (modify "/etc/compass-ci/defaults/crystal.yaml", and run $CCI_SRC/container/defconfig.rb | defconfig.sh) LKP_SRC: ~/code/lkp-tests # path to your lkp-tests repo CCI_SRC: ~/code/compass-ci # path to your compass-ci repo - SCHED_HOST: 172.168.131.131 # hostname or ip of the scheduler + SCHED_HOST: 172.17.0.1 # hostname or ip of the scheduler SCHED_PORT: 3000 # port of the scheduler JOB_INDEX_TYPE: "jobs/_doc" # es job document index and type - '$'{variable} is used at shell command, means the value of variable diff --git a/user-client/helper/iperf.yaml b/user-client/helper/iperf.yaml index 1509470..30e929f 100644 --- a/user-client/helper/iperf.yaml +++ b/user-client/helper/iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-pxe.yaml b/user-client/jobs/iperf-pxe.yaml index 209c5d6..cb5a513 100644 --- a/user-client/jobs/iperf-pxe.yaml +++ b/user-client/jobs/iperf-pxe.yaml @@ -53,7 +53,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-vm.yaml b/user-client/jobs/iperf-vm.yaml index 434b964..c976bed 100644 --- a/user-client/jobs/iperf-vm.yaml +++ b/user-client/jobs/iperf-vm.yaml @@ -34,7 +34,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x -- 2.23.0

2 2

[PATCH compass-ci 2/2] remove actual ip in compass-ci code
by Xiao Shenwei 16 Nov '20

16 Nov '20

if need ip: obtain the actual ip address from configuration file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- doc/develop-environment.md | 2 +- src/features/jobs/right_iperf.yaml | 2 -- src/scheduler/README.md | 2 +- user-client/helper/iperf.yaml | 1 - 5 files changed, 3 insertions(+), 6 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..ca86e88 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -39,7 +39,7 @@ the returned data for setup_jumper_account_info like: { "account" => "guest", "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "jumper_ip" => "0.0.0.0", "jumper_port" => "10000" } diff --git a/doc/develop-environment.md b/doc/develop-environment.md index 8760878..ac408a2 100644 --- a/doc/develop-environment.md +++ b/doc/develop-environment.md @@ -4,7 +4,7 @@ cat >> ~/.ssh/config <<-EOF Host crystal - Hostname 124.90.34.227 + Hostname xxx.xxx.xxx.xxx Port 22113 EOF diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index f81af38..db83832 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -30,8 +30,6 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x - diff --git a/src/scheduler/README.md b/src/scheduler/README.md index fdc7027..c92c1e8 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -6,7 +6,7 @@ - And set the environment variables needed for running (modify "/etc/compass-ci/defaults/crystal.yaml", and run $CCI_SRC/container/defconfig.rb | defconfig.sh) LKP_SRC: ~/code/lkp-tests # path to your lkp-tests repo CCI_SRC: ~/code/compass-ci # path to your compass-ci repo - SCHED_HOST: 172.168.131.131 # hostname or ip of the scheduler + SCHED_HOST: 172.17.0.1 # hostname or ip of the scheduler SCHED_PORT: 3000 # port of the scheduler JOB_INDEX_TYPE: "jobs/_doc" # es job document index and type - '$'{variable} is used at shell command, means the value of variable diff --git a/user-client/helper/iperf.yaml b/user-client/helper/iperf.yaml index 1509470..50c9ac6 100644 --- a/user-client/helper/iperf.yaml +++ b/user-client/helper/iperf.yaml @@ -30,7 +30,6 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x -- 2.23.0

1 0

[PATCH compass-ci 1/2] delete unnecessary yaml files
by Xiao Shenwei 16 Nov '20

16 Nov '20

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- user-client/jobs/iperf-pxe.yaml | 59 --------------------------------- user-client/jobs/iperf-vm.yaml | 40 ---------------------- user-client/jobs/myjobs.yaml | 2 -- 3 files changed, 101 deletions(-) delete mode 100644 user-client/jobs/iperf-pxe.yaml delete mode 100644 user-client/jobs/iperf-vm.yaml delete mode 100644 user-client/jobs/myjobs.yaml diff --git a/user-client/jobs/iperf-pxe.yaml b/user-client/jobs/iperf-pxe.yaml deleted file mode 100644 index 209c5d6..0000000 --- a/user-client/jobs/iperf-pxe.yaml +++ /dev/null @@ -1,59 +0,0 @@ ---- - -#! jobs/iperf.yaml - -#this job yaml file is used to run qemu test by using pxe, in compass-ci/sparrow/6-test/qemu, -#usage like this: 0_addjob iperf-pxe.yaml - -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-pxe-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-pxe-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -iostat: -heartbeat: -vmstat: -numa-numastat: -numa-vmstat: -numa-meminfo: -proc-vmstat: -proc-stat: -meminfo: - -LKP_SERVER: 172.168.131.113 -LKP_CGI_PORT: 3000 -result_root: /result/iperf -LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-vm.yaml b/user-client/jobs/iperf-vm.yaml deleted file mode 100644 index 434b964..0000000 --- a/user-client/jobs/iperf-vm.yaml +++ /dev/null @@ -1,40 +0,0 @@ ---- - -#! jobs/iperf.yaml - -#this job yaml file is used to run qemu test by using kvm, in compass-ci/sparrow/6-test/qemu, -#usage like this: 0_addjob iperf-vm.yaml - -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -iostat: -heartbeat: -vmstat: -numa-numastat: -numa-vmstat: -numa-meminfo: -proc-vmstat: -proc-stat: -meminfo: - -LKP_SERVER: 172.168.131.113 -LKP_CGI_PORT: 3000 -result_root: /result/iperf -LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/myjobs.yaml b/user-client/jobs/myjobs.yaml deleted file mode 100644 index 73bbb63..0000000 --- a/user-client/jobs/myjobs.yaml +++ /dev/null @@ -1,2 +0,0 @@ -#! jobs/myjobs.yaml -test: -- 2.23.0

1 0

[PATCH compass-ci] container/git-mirror: get the lkp and comitter ID from host
by Cao Xueliang 16 Nov '20

16 Nov '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/git-mirror/start | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/container/git-mirror/start b/container/git-mirror/start index ee79578..6ec0001 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -9,11 +9,14 @@ DOCKER_CCI = '/c/compass-ci' DOCKER_REPO = '/c/upstream-repos' docker_rm "git-mirror" +lkp = %x(id -u lkp) +committer = %x(id -u committer) + cmd = %W[ docker run --restart=always --name git-mirror - -u 1090:1999 + -u #{lkp}:#{committer} -d -e CCI_SRC=#{DOCKER_CCI} -e REPO_SRC=#{DOCKER_REPO} -- 2.23.0

1 0

[PATCH lkp-tests] tests: add hackbench-git test script
by Zhang Yale 16 Nov '20

16 Nov '20

[Why] hackbench-git is an old version tool required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] So need to add new test script for hackbench-git. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/hackbench-git | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100755 tests/hackbench-git diff --git a/tests/hackbench-git b/tests/hackbench-git new file mode 100755 index 00000000..f2be0c9d --- /dev/null +++ b/tests/hackbench-git @@ -0,0 +1,30 @@ +#!/bin/sh +# - nr_threads +# - mode +# - ipc +# - iterations +# - loops + +## Hackbench-git is both a benchmark and a stress test for the Linux kernel scheduler. +## Its main job is to create a specified number of schedulable entities pairs. +## (either threads or traditional processes)which communicate via either sockets +## or pipes and time how long it takes for each pair to send data back and forth. + +[ "$ipc" = 'pipe' ] && ipc_option='-pipe' +[ -n "$iterations" ] || iterations=10 +[ -n "$loops" ] || loops=100 +[ -n "$nr_threads" ] || nr_threads=1 + +run_hackbench() +{ + # args: groups number, mode [process/thread], loops, ipc [pipe/socket] + hackbench_path="$BENCHMARK_ROOT"/hackbench-git/hackbench/hackbench + [ ! -x "${hackbench_path}" ] && exit 1 + + echo "--------$ipc $mode num=$nr_threads--------" + for i in $(seq 1 $iterations); do + log_cmd $hackbench_path $ipc_option $nr_threads $mode $loops + done +} + +run_hackbench -- 2.23.0

2 2

[PATCH compass-ci 2/2] lib/matrix2.rb: judge the value.size condition first
by Lu Kaiyi 16 Nov '20

16 Nov '20

[why] avoid to execute some unnecessary operations and improve performance. [how] keep the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 62d174c..86cfc93 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

2 1

[PATCH v3 lkp-tests] sysbench-mysql/PKGBUILD: fix "No such file or directory"
by Zhang Yu 16 Nov '20

16 Nov '20

[why] only openeuler have dir "/usr/local/mysql/lib/" Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- pkg/sysbench-mysql/PKGBUILD | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index d9f342a5..c214d8a2 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -18,6 +18,8 @@ build() { package() { cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + [[ $os == openeuler ]] && { + mkdir -p $pkgdir/usr/lib64 + cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + } } -- 2.23.0

1 0

[PATCH lkp-tests v3] depends/sysbench-mysql: add depends
by Zhang Yu 16 Nov '20

16 Nov '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- distro/depends/sysbench-mysql | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/depends/sysbench-mysql b/distro/depends/sysbench-mysql index cfbc4518..b0d33ffe 100644 --- a/distro/depends/sysbench-mysql +++ b/distro/depends/sysbench-mysql @@ -1,2 +1,4 @@ libnuma1 xfsprogs +ethtool +libmariadb3 -- 2.23.0

1 0

[PATCH v2 lkp-tests] sysbench-mysql/PKGBUILD: fix "No such file or directory"
by Zhang Yu 16 Nov '20

16 Nov '20

[why] when run this PKGBUILD in centos error like: /lkp/lkp/src/pkg/sysbench/PKGBUILD: line 23: usr/local/mysql/lib/: No No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- pkg/sysbench-mysql/PKGBUILD | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index d9f342a5..119aea46 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -18,6 +18,8 @@ build() { package() { cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + [[ $os == openeuler ]] && { + mkdir -p $pkgdir/usr/lib64 + cp -r /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + } } -- 2.23.0

2 2