October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] sbin/submit: support monitor multiple jobs
by Wu Zhende 31 Oct '20

31 Oct '20

[Why] When submit -m xxx.yaml, may split to many jobs, and return many job_id. In this scenario, these jobs need to be monitored at the same time. [How] Use array replace string, like this: {"job_id": ["1", "2", "3"]} [Example] submit -m iperf.yaml submit iperf.yaml, got job_id=z9.109165 submit iperf.yaml, got job_id=z9.109166 query=>{"job_id":["z9.109165","z9.109166"]} connect to ws://localhost:11310/filter {"job_id"=>"z9.109165", "job_state"=>"boot"} {"job_id"=>"z9.109166", "job_state"=>"boot"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 4 ++++ sbin/submit | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index c20794e8..e0098390 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -99,6 +99,10 @@ class Monitor def run(timeout = nil) merge_overrides field_check + + @query.each do |k, v| + @query[k] = JSON.parse(v) + end query = @query.to_json puts "query=>#{query}" diff --git a/sbin/submit b/sbin/submit index 4884ad63..bc90e2e0 100755 --- a/sbin/submit +++ b/sbin/submit @@ -157,7 +157,7 @@ end if opt_monitor job_hash_list[0].delete('define_files') - opt_monitor_query.merge!({'job_id' => job_ids[0]}) + opt_monitor_query.merge!({'job_id' => job_ids}) cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_hash_list[0].to_json}' -a #{actions.join(',')} -s 'job_state: extract_finished'" exec cmd end -- 2.23.0

1 0

[PATCH v1 lkp-tests] hackbench-git: add PKGBUILD
by Zhang Yale 31 Oct '20

31 Oct '20

[Why] hackbench-git is an old version required by the customer. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- pkg/hackbench-git/PKGBUILD | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 pkg/hackbench-git/PKGBUILD diff --git a/pkg/hackbench-git/PKGBUILD b/pkg/hackbench-git/PKGBUILD new file mode 100644 index 00000000..22ed9f0f --- /dev/null +++ b/pkg/hackbench-git/PKGBUILD @@ -0,0 +1,20 @@ +pkgname=hackbench-git +pkgver=git +pkgrel=1 +arch=('i386' 'x86_64') +url="https://github.com/lacombar/hackbench.git" +license=('GPL') +source=("https://github.com/lacombar/hackbench.git") +sha256sums=('SKIP') + +build() +{ + cd "${srcdir}/${pkgname}/${pkgname}" || exit + gcc -o hackbench hackbench.c -lpthread +} + +package() +{ + mkdir -p "${pkgdir}/lkp/benchmarks/${pkgname}" + cp -rf "${srcdir}/${pkgname}/${pkgname}" "${pkgdir}/lkp/benchmarks/${pkgname}" +} -- 2.23.0

1 1

[PATCH compass-ci] service/monitoring: multiple values can be set for a key
by Wu Zhende 31 Oct '20

31 Oct '20

[Why] Only one key and one value can be used as search criteria. However, in some scenarios,one-to-many query is required. Such like: {"job_id": ["1", "2", "3"]} {"level": ["error", "warning"]} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/monitoring/filter.cr | 16 ++++++++++++++-- src/monitoring/monitoring.cr | 3 ++- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/src/monitoring/filter.cr b/src/monitoring/filter.cr index 5cd6532..bc6f901 100644 --- a/src/monitoring/filter.cr +++ b/src/monitoring/filter.cr @@ -12,10 +12,21 @@ class Filter end def add_filter_rule(query : JSON::Any, socket : HTTP::WebSocket) + query = convert_hash_value_to_array(query) + @hash[query] = Array(HTTP::WebSocket).new unless @hash[query]? @hash[query] << socket end + private def convert_hash_value_to_array(query) + new_query = Hash(String, Array(JSON::Any)).new + + query.as_h.each do |key, value| + new_query[key] = value.as_a? || [value] + end + JSON.parse(new_query.to_json) + end + def remove_filter_rule(query : JSON::Any, socket : HTTP::WebSocket) return unless @hash[query]? @@ -45,10 +56,11 @@ class Filter def match_query(query : Hash(String, JSON::Any), msg : Hash(String, JSON::Any)) query.each do |key, value| - if value == nil + value = value.as_a + if value.includes?(nil) return false unless msg.has_key?(key) else - return false unless value == msg[key]? + return false unless value.includes?(msg[key]?) end end return true diff --git a/src/monitoring/monitoring.cr b/src/monitoring/monitoring.cr index e665d5f..dff1902 100644 --- a/src/monitoring/monitoring.cr +++ b/src/monitoring/monitoring.cr @@ -15,7 +15,8 @@ module Monitoring query = JSON::Any.new("") socket.on_message do |msg| - # query like {"job_id": 1} + # query like {"job_id": "1"} + # also can be {"job_id": ["1", "2"]} query = JSON.parse(msg) if query.as_h? filter.add_filter_rule(query, socket) -- 2.23.0

1 0

[PATCH v6 compass-ci 1/3] addsoftware/run: change the example os version
by Xu Xijian 31 Oct '20

31 Oct '20

[why] As the old os version centos/aarch64/76 is not exist, change the version number 76 in given example. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- container/add-software/run | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/add-software/run b/container/add-software/run index 99490ee..f4c6833 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -5,8 +5,8 @@ [ -L "$1/vmlinuz" -a -n "$2" ] || { cat << 'EOF' Example usage: - 1> ./run /os/centos/aarch64/76 pack1 pack2 ... - 2> ./run /os/centos/aarch64/76 $(show-depends-packages centos) + 1> ./run /os/centos/aarch64/7.6.1810 pack1 pack2 ... + 2> ./run /os/centos/aarch64/7.6.1810 $(show-depends-packages centos) centos is an adaption file contain packages mapping from debian to centos, the whole path is '$LKP_SRC/distro/adaptation/centos'. EOF -- 2.23.0

1 0

[PATCH v5 compass-ci 2/2] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 31 Oct '20

31 Oct '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..8bf7a65 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -87,6 +87,10 @@ class Job kernel_append_root kernel_params docker_image + kernel_version + vmlinuz_path + modules_path + headers_path ) macro method_missing(call) @@ -135,6 +139,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,9 +298,22 @@ class Job return true end + private def set_kernel_version + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + if self["kernel_version"]? + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz-#{kernel_version}" + self["modules_path"] = "#{boot_dir}/modules-#{kernel_version}.cgz" + self["headers_path"] = "#{boot_dir}/headers-#{kernel_version}.cgz" + else + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz" + self["modules_path"] = "#{boot_dir}/modules.cgz" + self["headers_path"] = "#{boot_dir}/headers.cgz" + end + end + private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{vmlinuz_path}")}" end private def common_initrds @@ -316,6 +334,10 @@ class Job "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{modules_path}")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{headers_path}")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

1 0

[PATCH v5 compass-ci 1/2] addsoftware/run: change the example os version
by Xu Xijian 31 Oct '20

31 Oct '20

[why] As the old os version centos/aarch64/76 is not exist, change the version number 76 in given example. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- container/add-software/run | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/add-software/run b/container/add-software/run index 99490ee..f4c6833 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -5,8 +5,8 @@ [ -L "$1/vmlinuz" -a -n "$2" ] || { cat << 'EOF' Example usage: - 1> ./run /os/centos/aarch64/76 pack1 pack2 ... - 2> ./run /os/centos/aarch64/76 $(show-depends-packages centos) + 1> ./run /os/centos/aarch64/7.6.1810 pack1 pack2 ... + 2> ./run /os/centos/aarch64/7.6.1810 $(show-depends-packages centos) centos is an adaption file contain packages mapping from debian to centos, the whole path is '$LKP_SRC/distro/adaptation/centos'. EOF -- 2.23.0

1 0

[PATCH v4 compass-ci 2/2] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 31 Oct '20

31 Oct '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..2656635 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -87,6 +87,10 @@ class Job kernel_append_root kernel_params docker_image + kernel_version + vmlinuz_path + modules_path + headers_path ) macro method_missing(call) @@ -135,6 +139,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,9 +298,22 @@ class Job return true end + private def set_kernel_version + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + if self["kernel_version"]? + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz-#{kernel_version}" + self["modules_path"] = "#{boot_dir}/modules-#{kernel_version}.cgz" + self["headers_path"] = "#{boot_dir}/headers-#{kernel_version}.cgz" + else + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz" + self["modules_path"] = "#{boot_dir}/modules.cgz" + self["headers_path"] = "#{boot_dir}/headers.cgz" + end + end + private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{vmlinuz_path}")}" end private def common_initrds @@ -312,10 +330,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{modules_path}")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{headers_path}")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

1 0

[PATCH v4 compass-ci 1/2] addsoftware/run: change the example os version
by Xu Xijian 31 Oct '20

31 Oct '20

[why] As the old os version centos/aarch64/76 is not exist, change the version number 76 in given example. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- container/add-software/run | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/add-software/run b/container/add-software/run index 99490ee..f4c6833 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -5,8 +5,8 @@ [ -L "$1/vmlinuz" -a -n "$2" ] || { cat << 'EOF' Example usage: - 1> ./run /os/centos/aarch64/76 pack1 pack2 ... - 2> ./run /os/centos/aarch64/76 $(show-depends-packages centos) + 1> ./run /os/centos/aarch64/7.6.1810 pack1 pack2 ... + 2> ./run /os/centos/aarch64/7.6.1810 $(show-depends-packages centos) centos is an adaption file contain packages mapping from debian to centos, the whole path is '$LKP_SRC/distro/adaptation/centos'. EOF -- 2.23.0

1 0

[PATCH lkp-tests] jobs: rename hackbench-20.yaml and fix datasize to loops.
by Zhang Yale 31 Oct '20

31 Oct '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] It's different from hackbench-1.8 version. The hackbench-1.8 usage: hackbench [-p|--pipe] [-s|--datasize <bytes>] [-l|--loops <num loops>] [-g|groups <num groups] [-f|--fds ,num fds>] [-T|--threads] [-P|--process] [--help] So add hackbench-git.yaml for users. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/{hackbench-20.yaml => hackbench-git.yaml} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename jobs/{hackbench-20.yaml => hackbench-git.yaml} (100%) diff --git a/jobs/hackbench-20.yaml b/jobs/hackbench-git.yaml similarity index 100% rename from jobs/hackbench-20.yaml rename to jobs/hackbench-git.yaml -- 2.23.0

1 0

[PATCH v5 lkp-tests] hackbench/PKGBUILD: fix source
by Zhang Yale 31 Oct '20

31 Oct '20

[Why] Old source "https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-1.8.tar.gz" is not available, So change the source. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- pkg/hackbench/PKGBUILD | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pkg/hackbench/PKGBUILD b/pkg/hackbench/PKGBUILD index 797a00e6..842b2596 100644 --- a/pkg/hackbench/PKGBUILD +++ b/pkg/hackbench/PKGBUILD @@ -2,9 +2,9 @@ pkgname=hackbench pkgver=1.8 pkgrel=1 arch=('i386' 'x86_64') -url="https://www.kernel.org/pub/linux/utils/rt-tests" +url="https://git.kernel.org/pub/scm/utils/rt-tests" license=('GPL') -source=("https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-${pkgver}.tar.gz") +source=("https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/snapshot/rt-test…") sha256sums=('SKIP') package() { -- 2.23.0

1 0