- Compass-ci - mailweb.openeuler.org

[PATCH] lib/stats.rb: forcely show more stats if --no-hide-noises is set in sbin/compare
by Zheng Zengkai 06 Aug '25

06 Aug '25

Reuse the "--no-hide-noises" option and allow to bypass some checks for stats and show more stats comparisions if "--no-hide-noises" option is given when using lkp compare. It is useful when tester wants to see more stats comparisions between different runs, for example when the change ratio is less than 10%. Before the patch: [root@localhost /]# lkp compare -twa -d commit 5.10.0-4a4dbce046f7+ 5.10.0-47abaacd89b8+ info: "export LKP_GIT_WORK_TREE=/path/to/project/git/repo/" to enable parsing non-sha1 commit names. tests: 1 [root@localhost /]# lkp compare -twa --no-hide-noises -d commit 5.10.0-4a4dbce046f7+ 5.10.0-47abaacd89b8+ info: "export LKP_GIT_WORK_TREE=/path/to/project/git/repo/" to enable parsing non-sha1 commit names. tests: 1 [root@localhost /]# After the patch: [root@localhost /]# lkp compare -twa --no-hide-noises -d commit 5.10.0-4a4dbce046f7+ 5.10.0-47abaacd89b8+ info: "export LKP_GIT_WORK_TREE=/path/to/project/git/repo/" to enable parsing non-sha1 commit names. tests: 1 suite/path_params/tbox_group/run: unixbench/syscall/localhost 5.10.0-4a4dbce04 5.10.0-47abaacd89b8+ ---------------- -------------------------- %stddev change %stddev \ | \ 62340 -8% 57520 unixbench.System_Call_Overhead fail:runs %reproduction fail:runs | | | 2:2 0% 3:3 dmesg.tstage.last 2:2 0% 3:3 kmsg.eid.ACPI:IORT:[Firmware_Bug]:[map(____ptrval____)]conflicting_mapping_for_input_ID 2:2 0% 3:3 kmsg.eid.ACPI:IORT:[Firmware_Bug]:applying_workaround 2:2 0% 3:3 kmsg.eid.integrity:Unable_to_open_file:/etc/keys/x509_ima.der(-#) 2:2 0% 3:3 kmsg.eid.integrity:Unable_to_open_file:/etc/keys/x509_evm.der(-#) 2:2 0% 3:3 kmsg.eid.hisi_sas_v3_hw#:#:#:#hw_queues 2:2 0% 3:3 kmsg.eid.hisi_sas_v3_hw###:#hw_queues 158:2 0% 237:3 kmsg.msg.ACPI:IORT:[Firmware_Bug]:[map(____ptrval____)]conflicting_mapping_for_input_ID.message 100:2 0% 150:3 kmsg.msg.ACPI:IORT:[Firmware_Bug]:applying_workaround.message 114:2 0% 171:3 kmsg.msg.integrity:Unable_to_open_file:/etc/keys/x509_ima.der(-#).message 114:2 0% 171:3 kmsg.msg.integrity:Unable_to_open_file:/etc/keys/x509_evm.der(-#).message 84:2 0% 126:3 kmsg.msg.hisi_sas_v3_hw#:#:#:#hw_queues.message 84:2 0% 126:3 kmsg.msg.hisi_sas_v3_hw###:#hw_queues.message 0.72:2 0% 1.08:3 kmsg.ts.ACPI:IORT:[Firmware_Bug]:[map(____ptrval____)]conflicting_mapping_for_input_ID 0.72:2 0% 1.08:3 kmsg.ts.ACPI:IORT:[Firmware_Bug]:applying_workaround 83.19:2 -4% 124.69:3 kmsg.ts.last 28.85:2 -2% 43.22:3 kmsg.ts.integrity:Unable_to_open_file:/etc/keys/x509_evm.der(-#) 28.85:2 -2% 43.22:3 kmsg.ts.integrity:Unable_to_open_file:/etc/keys/x509_ima.der(-#) 33.27:2 -24% 49.18:3 kmsg.ts.hisi_sas_v3_hw#:#:#:#hw_queues 48.59:2 -171% 67.75:3 kmsg.ts.hisi_sas_v3_hw###:#hw_queues [root@localhost /]# Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> --- lib/stats.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/stats.rb b/lib/stats.rb index 3ef5c937a..be2331936 100755 --- a/lib/stats.rb +++ b/lib/stats.rb @@ -630,7 +630,7 @@ def __get_changed_stats(a, b, is_incomplete_run, options) b.each_key { |k| a[k] = [0] * cols_a unless a.include?(k) } # rubocop:disable Style/CombinableLoops a.each do |k, v| - is_force_stat = options["force_#{k}"] + is_force_stat = options["force_#{k}"] || $opt_no_hide_noises next if v[-1].is_a?(String) next if options['perf'] && !perf_metric?(k) -- 2.20.1

1 0

[PATCH] sbin/split-job: get_kernel_info and init job result_root elements
by Zheng Zengkai 04 Aug '25

04 Aug '25

In order to avoid potential problems while assigning values to dimension keys(like commit,compiler,kconfig) from option parameters, assign default values to these dimension variables. Implement a helper function get_kernel_info() to retrieve the kernel version and the version of the compiler that is used to compile the kernel. Invoke the get_kernel_info() during collecting distribution information of local system. Initialize the dimension commit key with value kernelversion got from the distroinfo if no commit option is setting. Initialize the dimension compiler key with value kernelcompiler if it is not assigned. Initialize the dimension kconfig key with value "defconfig" if no kconfig option is setting, as we can't guarantee that we can get the right kconfig file name in various environment. Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> --- lib/detect-system.sh | 24 ++++++++++++++++++++++++ lib/distro_info.rb | 9 ++++++--- sbin/split-job | 5 +++-- 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/lib/detect-system.sh b/lib/detect-system.sh index 384480b29..da3cdea63 100755 --- a/lib/detect-system.sh +++ b/lib/detect-system.sh @@ -108,6 +108,30 @@ detect_libc_version() return 1 } +get_kernel_compiler() +{ + local system_dir=${1:-/} + local proc_version=$(<${system_dir}proc/version) + local pattern1='gcc[[:space:]].GCC.[[:space:]]*([0-9]+\.[0-9]+\.[0-9]+)' + local pattern2='gcc[[:space:]]+version[[:space:]]+([0-9]+\.[0-9]+\.[0-9]+)' + export _kernel_compiler='unknown' + + if [[ "$proc_version" =~ $pattern1 ]]; then + _kernel_compiler="${BASH_REMATCH[1]}" + elif [[ "$proc_version" =~ $pattern2 ]]; then + _kernel_compiler="${BASH_REMATCH[1]}" + fi +} + +get_kernel_info() +{ + local system_dir=${1:-/} + export _kernel_version='unknown' + + _kernel_version=$(command uname -r) + get_kernel_compiler $system_dir +} + detect_system() { local system_dir=${1:-/} diff --git a/lib/distro_info.rb b/lib/distro_info.rb index b17c77fa7..066be440c 100755 --- a/lib/distro_info.rb +++ b/lib/distro_info.rb @@ -6,7 +6,7 @@ module LKP # # DistroInfo is singleton, and provide information to distribution information of local system - # Include: system name, system version, system arch + # Include: system name, system version, system arch, kernel version, kernel compiler # In the backend, it's invoking detect-system.sh to get environment information. # Example of properties on debian # p systemName => Debian @@ -18,12 +18,12 @@ module LKP # class DistroInfo include Singleton - attr_reader :systemname, :systemnamel, :systemversion, :systemarch + attr_reader :systemname, :systemnamel, :systemversion, :systemarch, :kernelversion, :kernelcompiler def initialize(rootfs = '/') path_to_script = "#{LKP_SRC}/lib/detect-system.sh" - @systemname, @systemnamel, @systemversion, @systemarch = %x[ + @systemname, @systemnamel, @systemversion, @systemarch, @kernelversion, @kernelcompiler = %x[ export LKP_SRC=#{LKP_SRC} . #{path_to_script} detect_system #{rootfs} @@ -31,6 +31,9 @@ module LKP echo $_system_name_lowercase echo $_system_version get_system_arch #{rootfs} + get_kernel_info #{rootfs} + echo $_kernel_version + echo $_kernel_compiler ].split end end diff --git a/sbin/split-job b/sbin/split-job index bef7e970f..63b68bc84 100755 --- a/sbin/split-job +++ b/sbin/split-job @@ -97,8 +97,9 @@ ARGV.each do |jobfile| jobfile = Job.find_jobfile(jobfile) jobfile = refine_job_file(jobfile) if @opt_compatible jobs.load(jobfile, expand_template: true) || next - jobs['kconfig'] = @opt_kconfig if @opt_kconfig - jobs['commit'] = @opt_commit if @opt_commit + jobs['kconfig'] = @opt_kconfig || "defconfig" + jobs['commit'] = @opt_commit || distroinfo.kernelversion + jobs['compiler'] ||= distroinfo.kernelcompiler jobs['testbox'] = @opt_testbox jobs['arch'] ||= distroinfo.systemarch jobs['tbox_group'] = tbox_group(@opt_testbox) -- 2.20.1

1 0

[PATCH] lib/result.rb: Restore PATH_SCHEME of LOCAL_RUN in ResultPath
by Zheng Zengkai 31 Jul '25

31 Jul '25

Commit cd1d0c6cff23 ("run-local: simplify result path") changes LOCAL_RUN path scheme to "%w[path_params run]" like below: /root/.cache/lkp/result/unixbench/1-100-syscall/0 /root/.cache/lkp/result/unixbench/1-100-syscall/1 /root/.cache/lkp/result/unixbench/96-100-syscall/0 According to the lkp compare usage, user can use the -d option with "commit, kconfig, fs, etc" dimensions to compare results generated by different kernel versions with different commits, kconfigs, or rootfs, it's useful for evaluating the impact of different dimensions(like commits) on performance by running benchmarks on local server. " [root@localhost ~]# lkp compare --help Usage: compare [options] RESULT_ROOT... options: -d, --dimension DIMENSION DIMENSION to compare: commit, kconfig, fs, etc. " But with the simplified result path scheme for local run, this usage is not supported, as the remove of commit, kconfig and rootfs elements, the result_root path will not contain these sub-directories and when setup_result_root_hash greps "dim value" in the "~/.cache/lkp/paths", nothing will be matched, then $_result_root_tuple is empty and no stats can be compared and shown. This patch changes the PATH_SCHEME of LOCAL_RUN to support this usage. Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> --- lib/result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/result.rb b/lib/result.rb index 492e43b82..8260c8671 100755 --- a/lib/result.rb +++ b/lib/result.rb @@ -28,7 +28,7 @@ class ResultPath < Hash AXIS_KEYS = (MAXIS_KEYS + ['run']).freeze PATH_SCHEME = { - 'LOCAL_RUN' => %w[path_params run], + 'LOCAL_RUN' => %w[path_params tbox_group rootfs kconfig compiler commit run], 'default' => %w[path_params tbox_group rootfs kconfig compiler commit run], 'kvm:default' => %w[path_params tbox_group rootfs kconfig compiler commit run], 'health-stats' => %w[path_params run], -- 2.43.0

1 0

[PATCH] sbin/compare: Fix undefined method for false in sort_stats
by Zheng Zengkai 30 Jul '25

30 Jul '25

when doing lkp compare, it complains: /home/lkp-tests/sbin/compare:826:in `block in sort_stats': undefined method `=~' for false:FalseClass (NoMethodError) from /home/lkp-tests/sbin/compare:814:in `each' from /home/lkp-tests/sbin/compare:814:in `sort_by' from /home/lkp-tests/sbin/compare:814:in `sort_by!' from /home/lkp-tests/sbin/compare:814:in `sort_stats' from /home/lkp-tests/sbin/compare:860:in `<main>' After checking the code and operator priority in ruby, since the 'Logical NOT operator !' has a higher priority than the 'Regex match operator =~', there is a mistake introduced by the commit 8fa05e8aae9f ("cleanup rubpcop issue Style/AndOr"). Add Parentheses to fix it. Fixes: 8fa05e8aae9f ("cleanup rubpcop issue Style/AndOr") Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> --- sbin/compare | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/compare b/sbin/compare index 11c4fbb05..f5db5ef33 100755 --- a/sbin/compare +++ b/sbin/compare @@ -823,7 +823,7 @@ def sort_stats(stats) if x =~ ABS_CHANGE_STATS_RE [stat_priority(fields[0]), fields[1], abs_change] - elsif fields.size >= 3 && !fields[1] =~ /^node\d+$/ + elsif fields.size >= 3 && !(fields[1] =~ /^node\d+$/) # when a monitor can be broken into sub-groups of stats # under which values can be logically compared [stat_priority(fields[0]), fields[1], rel_change] -- 2.20.1

1 0

kuasar master、openEuler-25.03、openEuler-24.03-sp2分支申请root权限构建
by xiadanni 20 May '25

20 May '25

一、背景 OpenEuler-2403sp2分支kuasar-cc-1.0.0-23.oe2403sp2.aarch64.rpm包失败。报错日志： [cid:image002.png@01DBC96C.D1A67680] cp拷贝二进制文件因为权限问题失败 [cid:image004.png@01DBC96C.D1A67680] 制作kuasar rootfs时存在mount报错问题经私人工程验证，加上root权限能解决上述问题，且当前master、openEuler-25.03分支的kuasar构建也存在类似问题。二、期望添加的配置在master、openEuler-25.03、openEuler-24.03-sp2分支的everything工程的root权限配置中添加kuasar 三、涉及工程范围 1.Eulermaker中master、openEuler-25.03、openEuler-24.03-sp2分支kuasar包编译工程

1 0

openEuler 2503版本构建部分使能mold相关说明
by yinchuang 27 Feb '25

27 Feb '25

各位伙伴好：背景：为了提升openEuler软件包的编译效率，从而进一步提升门禁和开发者本地开发效率，我们在2503版本使用mold链接器来缩短C/C++库的链接时间。 Mold（A Modern Linker）是一种对比其他链接器（GNU ld、GNU gold、LLVM lld）链接效率更高的开源链接器, 其通过更高效的Identical Comdat Folding (ICF)算法、更好的并行编程，更高效的多线程库等技术来提升链接的效率，具体可参考：https://github.com/rui314/mold/blo… 具体方案：由于mold链接器本身存在一定的功能欠缺（对kernel的不支持，对linker script支持不完善）我们决定对一些编译时间较长且不受mold影响的包打开mold 链接： 1. 在2503版本上白名单内的软件包默认配置使用mold进行链接，白名单可参见：https://gitee.com/src-openeuler/openE… 2. 为了提供更加灵活的链接器配置方式，各软件包可在spec文件中覆写_ld_use变量来切换链接器，如： 1) %define _ld_use %{nil} 取消因为软件包在白名单中所添加的mold选项 2) %define _ld_use -fuse-ld=xxx 切换到不同的链接器，注意当定义多个-fuse-ld 选项时编译器会取最后一个配置项 3. 版本上会预装mold链接器来提供各软件包对mold的调用收益：经统计通过使能mold的软件包编译效率总体可提升5+% 我们会持续在社区讨论mold使能可行性与风险，感谢大家。 openEuler Compiler SIG 印闯

1 0

[Compass-ci]mysql包申请添加openEuler-20.03-LTS-SP4:everything工程配置中use_root选项
by 王凯 27 Aug '24

27 Aug '24

一.背景 20.03-LTS-SP4分支mysql源码包因尺寸超出上传门禁限制，使用wget下载覆盖空文件源码包，这个过程中需要被赋予root权限。报错日志：https://eulermaker.compass-ci.openeuler.openatom.cn/package/build-record?osProject=wk333:openEuler-20.03-LTS-SP4:everything-testmysql0823&packageName=mysql&jobId=cbs.6075654 2024-08-23 17:30:56 + wget -qO /home/lkp/rpmbuild/SOURCES/mysql-boost-8.0.38.tar.gz https://user-repo.openeuler.openatom.cn/lfs-tar/mysql/mysql-boost-8.0.38.ta… 2024-08-23 17:30:56 /home/lkp/rpmbuild/SOURCES/mysql-boost-8.0.38.tar.gz: Permission denied 2024-08-23 17:30:56 error: Bad exit status from /var/tmp/rpm-tmp.ulSf9W (%prep) 二.期望添加的配置在openEuler-20.03-LTS-SP4:everything工程的use_root配置中添加mysql 三.涉及工程范围 1.Eulermaker中20.03-SP4分支mysql包编译工程

1 0

Compass-CI使用相关问题
by 黄顺聪 12 Aug '24

12 Aug '24

1.申请账号未通过，同时没有任何未通过原因邮件反馈。原始邮件如下，希望优化申请流程，对申请失败情况有个简要的回复。 2.https://gitee.com/wu_fengguang/lkp-tests/tree/master/doc 文档与实际不符（存在部… 3.希望提供一个本地部署Compass-CI的操作文档 4.lkp-test希望出一期运行调用逻辑说明以及新增用例demo演示，降低入门成本发自我的企业微信

1 0

[Compass-ci]takari-archiver包申请添加openEuler-master:everything工程配置中use_root选项
by dear wang 16 Jul '24

16 Jul '24

一.背景 master分支takari-archiver包因takari-pom包升级后jdkversion被配置为11，重新归档的takari系列包提供的jar包版本全部升级为55版本，且系列包高度耦合，需要从takari-archiver包打破耦合，编译工程中需要删除55版本的jar包重新编译。这个过程中需要被赋予root权限。报错日志：https://eulermaker.compass-ci.openeuler.openatom.cn/api/result/rpmbuild/2024-07-16/dc-64g/openeuler-mainline-aarch64/aarch64-common-gitee.com-src-openeuler-ta/cbs.5701400/dmesg 二.期望添加的配置在openEuler-master:everything工程的use_root配置中添加takari-archiver 三.涉及工程范围 1.Eulermaker中master分支takari-archiver包编译工程

1 0

[Compas-ci]libabigail包申请去除openEuler-master:everything工程配置中use git lfs选项
by 707078654 09 Jul '24

09 Jul '24

一.背景 master分支的 libabigail 包已将tar包放至 user-repo （https://user-repo.openeuler.openatom.cn/lfs-tar/libabigail/libabigail-2.5.tar.xz）路径，不再采用git lfs方式获取tar包，且该选项会导致clone代码失败，申请移除use_git_lfs配置报错日志：https://eulermaker.compass-ci.openeuler.openatom.cn/package/build-record?osProject=xuping:openEuler-master:everything-0709-bak&packageName=libabigail&jobId=cbs.5628463 二.期望添加的配置移除openEuler-master:everything工程use_git_lfs配置中的libabigail 三.涉及工程范围 1.Eulermaker中master 分支libabigail 包编译工程 2.后续从 master分支拉取出来的新分支均需继承该工程许萍 18902872027 Best wishes

1 0