- Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] stats/crystal: add sysbench-mysql.cr
by Zhang Yu 26 Nov '20

26 Nov '20

Make ruby2crystal convert Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- stats/crystal/sysbench-mysql.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 stats/crystal/sysbench-mysql.cr diff --git a/stats/crystal/sysbench-mysql.cr b/stats/crystal/sysbench-mysql.cr new file mode 100644 index 00000000..1f6709c2 --- /dev/null +++ b/stats/crystal/sysbench-mysql.cr @@ -0,0 +1,17 @@ +#!/usr/bin/env ruby + +# SQL statistics: +# queries performed: +# read: 210212086 +# write: 60060591 +# other: 30030296 +# total: 300302973 +# transactions: 15015147 (25024.26 per sec.) +# queries: 300302973 (500485.16 per sec.) +# ignored errors: 2 (0.00 per sec.) +# reconnects:0 (0.00 per sec.) + +while (line = STDIN.gets) + next unless line =~ /transactions:.*\(\s*(\S+)/ + puts "transactions: #{$1}" +end -- 2.23.0

3 3

[PATCH lkp-tests 2/3] iso2rootfs: delete the unnecessary fields
by Yu Chuan 26 Nov '20

26 Nov '20

[Why] 'INITRD_HTTP_HOST' and 'INITRD_HTTP_PORT' should be added by the user on demand at runtime, no need to merge into the git repository. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 2 -- tests/iso2rootfs | 5 ++--- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 4f3e9c83e6cd..a0ffd280676d 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -7,8 +7,6 @@ iso2rootfs: rootfs_protocol: rootfs_server: rootfs_path: - initrd_http_host: - initrd_http_port: # config rootfs related fields ## supported fields: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 761fbb8b70cf..4a19d42ddd94 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -49,8 +49,6 @@ check_yaml_vars() "rootfs_protocol" "rootfs_server" "rootfs_path" - "initrd_http_host" - "initrd_http_port" ) for yaml_t in "${yaml_vars[@]}" @@ -190,7 +188,8 @@ check_os() get_cgz() { - local cgz_url="http://${initrd_http_host}:${initrd_http_port}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/iso2rootfs.cgz" + + local cgz_url="http://${INITRD_HTTP_HOST}:${INITRD_HTTP_PORT}/initrd/pkg/nfs/${os}/${os_arch}/${os_version}/iso2rootfs.cgz" local cgz_name cgz_name=$(basename "${cgz_url}") local cgz_path_t=${ENV_DIR}/${cgz_name} -- 2.23.0

2 2

[PATCH v2 compass-ci] create define_files in result_root
by Lu Weitao 26 Nov '20

26 Nov '20

[Why] User need debug their result-parse-script which haven't commit in lkp-tests/stats. [How] We already support user write script as "define_file" => {"stats/$script_name" => "$script_content"} in job.yaml, the job.yaml will be save as job to ES DB, write the job['define_file']['stats/$script_name'](value) at $result_root/stats/$script, extract-stats service will parse result by $result_root/stats/$script(user's result-parse-script) firstly Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- src/extract-stats/stats_worker.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..74afee1 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]? + define_files = job["define_files"]? + write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any) result_post_processing(job_id, result_root.to_s, queue_path) @tq.delete_task(queue_path + "/in_process", "#{job_id}") end @@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root) + + define_files.each do |k, v| + if k.match(/^stats/) + Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end + end + end end -- 2.23.0

2 2

Re: [PATCH v2 lkp-tests] tats/crystal: add hackbench-git.cr
by Bai Jing 26 Nov '20

26 Nov '20

^^^ stats Thanks, Baijing Reply-To: In-Reply-To: <20201125115826.760733-1-ylzhangah(a)qq.com> On Wed, Nov 25, 2020 at 07:58:26PM +0800, Zhang Yale wrote: >Make ruby2crystal convert > >Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> >--- > stats/crystal/hackbench-git.cr | 22 ++++++++++++++++++++++ > 1 file changed, 22 insertions(+) > create mode 100644 stats/crystal/hackbench-git.cr >diff --git a/stats/crystal/hackbench-git.cr b/stats/crystal/hackbench-git.cr >new file mode 100644 >index 00000000..2c043fc9 >--- /dev/null >+++ b/stats/crystal/hackbench-git.cr >@@ -0,0 +1,22 @@ >+#!/usr/bin/env crystal >+ >+# --------socket thread num=20-------- >+# 2020-11-10 19:51:31 /lkp/benchmarks/hackbench-git/hackbench/hackbench 20 thread 1000 >+# Running with 20*40 (== 800) tasks. >+# Time: 0.495 >+# ... >+ >+time = [] of Float64 >+ >+while (line = gets) >+ case line >+ when /^-+(\w+ \w+ \w+)/ >+ args = $1.tr(" ", "_") >+ when /^Running with .* $== (\d+)$ tasks/ >+ tasks = $1.to_i >+ when /^Time:/ >+ time << line.split[1].to_f >+ end >+end >+ >+puts "#{args}_#{tasks}: #{time.sum / time.size}" >-- >2.23.0 >

2 1

[PATCH v3 lkp-tests 3/3] tests/iozone: refactor format optimization
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/iozone b/tests/iozone index 5186766f..923d8331 100755 --- a/tests/iozone +++ b/tests/iozone @@ -47,8 +47,7 @@ if [ -n "$block_size" ]; then ;; "preadv") args+=" -i 12" ;; - *) echo "invalid test option, please check again!" - exit 1 + *) echo "invalid test option, please check again!" && exit 1 ;; esac done -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/3] stats/mysql-server: remove redundant whitespace
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/mysql-server | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stats/mysql-server b/stats/mysql-server index d3f3fd79..776f3f2e 100755 --- a/stats/mysql-server +++ b/stats/mysql-server @@ -16,9 +16,9 @@ while (line = STDIN.gets) # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 when /\[\s*\d+\%\]/ - results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') - results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') - results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') + results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') + results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') + results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') end end -- 2.23.0

1 0

[PATCH v2 lkp-tests] tats/crystal: add hackbench-git.cr
by Zhang Yale 25 Nov '20

25 Nov '20

Make ruby2crystal convert Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/crystal/hackbench-git.cr | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 stats/crystal/hackbench-git.cr diff --git a/stats/crystal/hackbench-git.cr b/stats/crystal/hackbench-git.cr new file mode 100644 index 00000000..2c043fc9 --- /dev/null +++ b/stats/crystal/hackbench-git.cr @@ -0,0 +1,22 @@ +#!/usr/bin/env crystal + +# --------socket thread num=20-------- +# 2020-11-10 19:51:31 /lkp/benchmarks/hackbench-git/hackbench/hackbench 20 thread 1000 +# Running with 20*40 (== 800) tasks. +# Time: 0.495 +# ... + +time = [] of Float64 + +while (line = gets) + case line + when /^-+(\w+ \w+ \w+)/ + args = $1.tr(" ", "_") + when /^Running with .* $== (\d+)$ tasks/ + tasks = $1.to_i + when /^Time:/ + time << line.split[1].to_f + end +end + +puts "#{args}_#{tasks}: #{time.sum / time.size}" -- 2.23.0

1 0

[PATCH lkp-tests] stats/crystal: add sysbench-mysql.cr
by Zhang Yu 25 Nov '20

25 Nov '20

[why] Make ruby2crystal convert. Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- stats/crystal/sysbench-mysql.cr | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 stats/crystal/sysbench-mysql.cr diff --git a/stats/crystal/sysbench-mysql.cr b/stats/crystal/sysbench-mysql.cr new file mode 100644 index 00000000..1bab78c0 --- /dev/null +++ b/stats/crystal/sysbench-mysql.cr @@ -0,0 +1,19 @@ +#!/usr/bin/env crystal + +# SQL statistics: +# queries performed: +# read: 210212086 +# write: 60060591 +# other: 30030296 +# total: 300302973 +# transactions: 15015147 (25024.26 per sec.) +# queries: 300302973 (500485.16 per sec.) +# ignored errors: 2 (0.00 per sec.) +# reconnects:0 (0.00 per sec.) + +while (line = STDIN.gets) + case line + when /transactions:.*\(\s*(\S+)/ + puts "transactions: #{$1}" + end +end -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/job: add runtime.md
by Zhang Yale 25 Nov '20

25 Nov '20

Add a document for introducing the runtime field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/job/runtime.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 doc/job/runtime.md diff --git a/doc/job/runtime.md b/doc/job/runtime.md new file mode 100644 index 0000000..c604bfa --- /dev/null +++ b/doc/job/runtime.md @@ -0,0 +1,27 @@ +# runtime + +Meaning: +- Seconds to run before exiting. +- Limit the test's run time in $runtime. + +Usage example: + +```bash +#!/bin/sh +# - runtime +``` + +`runtime` is a parameter of test script. +For any test case needing parameter(s), +need such comment line and follow exactly +the same format `# - parameter`. + +```yaml +runtime: 300s +``` + +```yaml +runtime: 1d +sleep: +``` + -- 2.23.0

1 0

[PATCH lkp-tests 3/3] iso2rootfs: add comment to iso2rootfs.yaml
by Yu Chuan 25 Nov '20

25 Nov '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index a0ffd280676d..2c4d3a93ff62 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,9 +1,18 @@ suite: iso2rootfs category: benchmark iso2rootfs: + # iso related fields to be used to generate rootfs iso_os: iso_arch: iso_version: + + # place the result rootfs related fields + ## 1. result rootfs will be placed in the following localtion on the + ## remote file server: + ## - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ + ## 2. remote file server protocols current supported: + ## - nfs + ## - cifs rootfs_protocol: rootfs_server: rootfs_path: -- 2.23.0

1 0