November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] doc/manual: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...273\273\345\212\241\347\273\223\346\236\234" | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..db6cc75 --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,17 @@ +# �� + ��https://compass-ci.openeuler.org�� + ��id��job state�� + +## job.yaml�� + job.yaml��job�� + �� + +## output�� + output��check_exit_code��0�� + +## �� + �� + +## stats.json�� + ��xxx.json�� + stats.json��json�� -- 2.23.0

3 2

[PATCH v2 compass-ci] doc/manual: how to check the result files
by Wei Jihui 26 Nov '20

26 Nov '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- ...3\273\345\212\241\347\273\223\346\236\234" | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 "doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" diff --git "a/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" new file mode 100644 index 0000000..855e9be --- /dev/null +++ "b/doc/manual/\345\246\202\344\275\225\346\237\245\347\234\213\344\273\273\345\212\241\347\273\223\346\236\234" @@ -0,0 +1,19 @@ +# �� + ��https://compass-ci.openeuler.org�� + ��suite��id��job state�� + +# ��compass-ci�� +## job.yaml�� + job.yaml��job�� + �� + +## output�� + output��check_exit_code��0�� + +## �� + �� + ��iperf,��iperf��iperf�� + +## stats.json�� + ��xxx.json�� + stats.json��json�� -- 2.23.0

1 0

[PATCH compass-ci] kernel_version.md: update kernel information
by Xu Xijian 26 Nov '20

26 Nov '20

1. delete redundant soft links of kernel files I've created before and update this information in kernel_version.md. 2. the kernel version of centos/aarch64/7.6.1810 we use has been changed into 4.14.0-115.el7.0.1.aarch64. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index 3e94ca8..4a193fb 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -6,10 +6,10 @@ Meaning: - If kernel_version is not given by users, it will use the default one. - Here are the kernel versions we support for different os: openeuler/aarch64/1.0 - - 4.19.90-2001(default) + - 4.19.90-vhulk2001.1.0.0026.aarch64(default) openeuler/aarch64/20.03 - - 4.19.90-2003(default) + - 4.19.90-2003.4.0.0036.oe1.aarch64(default) - 4.19.90-mysql - 4.19.90-nginx @@ -18,7 +18,7 @@ Meaning: - 5.8.0-1-arm64 centos/aarch64/7.6.1810 - - 4.14.0-115.el7a.0.1(default) + - 4.14.0-115.el7.0.1.aarch64(default) centos/aarch64/7.8.2003 - 4.18.0-147.8.1.el7(default) @@ -29,10 +29,10 @@ Meaning: Related files: - In initramfs boot process, every kernel version is related with a vmlinuz, module and headers. - Files like below under $boot_dir, an example $boot_dir can be "/srv/os/openeuler/aarch64/20.03/boot". -├── headers-4.19.90-2003.cgz -├── modules-4.19.90-2003.cgz -├── vmlinuz-4.19.90-2003 -└── vmlinuz -> vmlinuz-4.19.90-2003 +├── headers-4.19.90-2003.4.0.0036.oe1.aarch64.cgz +├── modules-4.19.90-2003.4.0.0036.oe1.aarch64.cgz +├── vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 +└── vmlinuz -> vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 Usage example: -- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 +- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003.4.0.0036.oe1.aarch64 -- 2.23.0

1 0

[PATCH compass-ci] fill 0 for nil standard_deviation with template compare mode
by Lu Weitao 26 Nov '20

26 Nov '20

[Why] With template compare mode, it's not friendly for echart that return null for nil standard_deviation of a metric like: { "tables": { "vmstat.system.cs": { "standard_deviation": { "dimensions": ["compare_dimension", "openeuler 20.03"], "source": [ ["4G", "8G", "16G"], ["openeuler 20.03", 23, null, null] ] } } } } [How] fill 0 for nil standard_deviation, instead of "null" Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 1550435..dd9b86e 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -461,7 +461,7 @@ def assign_metric_values(metrics_values, dim, metric, values) metrics_values[metric]['standard_deviation'] ||= {} metric_value = get_values(values, true) metrics_values[metric]['average'][dim] = metric_value[:average] - metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] + metrics_values[metric]['standard_deviation'][dim] = metric_value[:stddev] || 0 end def assign_metric_change(metrics_values) -- 2.23.0

2 2

[PATCH v3 compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..02161f0 --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,49 @@ +# Introduction of compare feature + The compare feature used to analyze the results of different jobs, showing the performance waves and changes + under different influencing factors, for users to analyze performance influencing factor. + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope. + choose at least one option as filter. + +### dimension + Dimension can select: os, os_version, os_arch, suite, tbox_group. + Within filter, we will compare all different job result by dimension + and keep other test conditions are same. + +## example: + filter: suite = iperf + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

1 0

[PATCH v2 compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..94b139a --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,49 @@ +# Introduction of compare feature + The compare feature used to analyze the results of different jobs, showing the performance waves and changes + under different influencing factors, for users to analyze performance influencing factor. + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope. + choose at least one option as filter. + +### dimension + Dimension can select: os, os_version, os_arch, suite, tbox_group. + Within filter, we will compare all different job result by dimension + and keep other test conditions are same. + +## example: + filter: suite = iperf | os_arch = aarch64 | tbox_group = vm-2p8g + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

1 0

[PATCH compass-ci] add compare.md
by Lu Weitao 26 Nov '20

26 Nov '20

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- doc/compare.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/compare.md diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..24b8f73 --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,46 @@ +# Introduction of compare feature + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope + +### dimension + dimension can select: os, os_version, os_arch, suite, box_group. + within compare scope, we will compare all defferent job result by dimension + and keep other test conditions are same. + +## example: + filter: suite=iperf | os_arch=aarch63 | tbox_group=2p8g + dimension: os_version + + result: +os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric +-------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ +4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 -- 2.23.0

3 4

[PATCH v6 lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template_yaml | 3 +++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 4 files changed, 27 insertions(+) create mode 100644 spec/submit/merge_yaml/template_yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template_yaml b/spec/submit/merge_yaml/template_yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template_yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..8d459a32 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template_yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH v3 lkp-tests] spec/submit_spec.rb: submit job_on_fail.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to debug job_on_fail.yaml, and after the task is failed, it is be waiting state. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/job_on_fail/job_on_fail.yaml | 5 +++++ .../job_on_fail/job_on_fail-3000.yaml | 22 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 3 files changed, 31 insertions(+) create mode 100644 spec/submit/job_on_fail/job_on_fail.yaml create mode 100644 spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml diff --git a/spec/submit/job_on_fail/job_on_fail.yaml b/spec/submit/job_on_fail/job_on_fail.yaml new file mode 100644 index 00000000..bf68ff40 --- /dev/null +++ b/spec/submit/job_on_fail/job_on_fail.yaml @@ -0,0 +1,5 @@ +suite: atomic + +on_fail: + sleep: 3000 +atomic: diff --git a/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml b/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml new file mode 100644 index 00000000..3d7ad9b5 --- /dev/null +++ b/spec/submit/job_on_fail/job_on_fail/job_on_fail-3000.yaml @@ -0,0 +1,22 @@ +--- +suite: atomic +on_fail: + sleep: 3000 +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + sleep: 3000 + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..52d9a158 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'job on fail' do + submit_job('job_on_fail') + end end -- 2.23.0

1 0

[PATCH compass-ci] add a document for adding open source project
by Li Yuanchao 26 Nov '20

26 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- ...-compass-ci-to-test-open-source-project.md | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 doc/how-to-use-compass-ci-to-test-open-source-project.md diff --git a/doc/how-to-use-compass-ci-to-test-open-source-project.md b/doc/how-to-use-compass-ci-to-test-open-source-project.md new file mode 100644 index 0000000..219cd0c --- /dev/null +++ b/doc/how-to-use-compass-ci-to-test-open-source-project.md @@ -0,0 +1,51 @@ +### 首先，将想要测试的代码仓信息以yaml格式文件的形式添加到upstream-repos仓库 + +仓库地址为：https://gitee.com/wu_fengguang/upstream-repos。 + +你可以先fork这个仓库，新建文件之后，通过Pull Request的方式完成添加。 + +以仓库backlight为例： + +它的url为：https://github.com/baskerville/backlight +1. 以首字母和仓库名创建路径 +``` +mkdir -p b/backlight +``` +2. 在该目录下新建同名文件backlight： +``` +cd b/backlight +touch backlight +``` +3. 使用vim写入url： +``` +vim backlight +``` +内容格式为 +``` +--- +url: +- https://github.com/baskerville/backlight +``` +请保持格式一致，若不确定，可参考upstream-repos仓库中已有文件。 + +pkgbuild_repo字段暂不需要，因为自助添加pkgbuild的功能尚未开放。 + +### 然后，将测试case以yaml文件的形式添加到lkp-tests仓库的jobs目录 + +仓库地址为 https://gitee.com/wu_fengguang/lkp-tests + +这其中有一些已经适配好的测试case，如果你想要测试的case其中正好有，那就可以直接使用。使用方式以iperf为例。 + +iperf是已经适配的一个测试case，在lkp-tests仓库的jobs下有一个iperf.yaml文件，里面有一些测试的基本参数。 + +然后，你要在compass-ci仓库下面的 sbin/auto_submit.yaml 文件中添加 +``` +b/backlight/backlight: +- testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml +``` +并通过Pull Request的方式提交。 + +参考文档： +- [compass-ci测试平台使用教程--submit命令详解](https://gitee.com/wu_fengguang/compass-ci/t… +- [如何适配测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-test… + -- 2.23.0

1 0