[Compass-ci] Re: [PATCH compass-ci] add compare.md

On Thu, Nov 26, 2020 at 03:54:23PM +0800, Cao Xueliang wrote:

On Thu, Nov 26, 2020 at 03:25:33PM +0800, Lu Weitao wrote:

...

Signed-off-by: Lu Weitao luweitaobe@163.com

doc/compare.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/compare.md

diff --git a/doc/compare.md b/doc/compare.md new file mode 100644 index 0000000..24b8f73 --- /dev/null +++ b/doc/compare.md @@ -0,0 +1,46 @@ +# Introduction of compare feature

+## url: https://compass-ci.openeuler.org/compare

+## options directions +### filters:

• • suite: iperf, netperf, mysql, ...
• • OS: openeuler 20.03, centos 7.6, ...
• • os_arch: aarch64, x86
• • tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ...
• we can combine the above options arbitrarily to limit compare scope

+### dimension

• dimension can select: os, os_version, os_arch, suite, box_group.
• within compare scope, we will compare all defferent job result by dimension

Is this filters scope?

yes,

Thanks, Weitao

Thanks, Xueliang

...

• and keep other test conditions are same.

+## example:

• filter: suite=iperf | os_arch=aarch63 | tbox_group=2p8g
• dimension: os_version
• result:

+os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same

•           20.09                           20.03  metric
  

+-------------------- ------------------------------ ------------------------------

•  fails:runs        change        fails:runs
  

•       |               |               |
  

•      3:3          -100.0%            0:3         last_state.exit_fail
  

•      3:3          -100.0%            0:3         last_state.is_incomplete_run
  

•      3:3          -100.0%            0:3         last_state.test.iperf.exit_code.127
  

•      3:3          -100.0%            0:3         stderr.perf_command_failed
  

•           20.09                           20.03  metric
  

+-------------------- ------------------------------ ------------------------------

•      %stddev       change            %stddev
  

•         \             |                 \
  

+4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps +4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps

•   94.82            -44.0%           53.10        boot-time.boot
  

•  123.11            -58.4%           51.19        boot-time.idle
  

•    0.00               0              4.87        boot-time.kernel_boot
  

• 4165.50 ± 12%      -99.9%            5.00        interrupts.38:GICv3.36.Level.virtio0
  

-- 2.23.0