- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 2/3] lib/compare_error_messages.rb: add the credibility check of job
by Lin Jiaxin 18 Jan '21

18 Jan '21

defect: may match the wrong result error_id="build-pkg.pixz.c:warning:'subsuf'defined-but-not-used[-Wunused-function]" filename="pixz.c" correct result: gcc -DHAVE_CONFIG_H -I. -I.. -D_FORTIFY_SOURCE=2 -pthread -Wall -Wno-unknown-pragmas -march=native -O2 -pipe -fstack-protector-strong -fno-plt -Wl,-O1, --sort-common,--as-needed,-z,relro,-z,now -MT pixz-pixz.o -MD -MP -MF .deps/pixz-pixz.Tpo -c -o pixz-pixz.o `test -f 'pixz.c' || echo './'`pixz.c ^^^^^^ wrong result: mv -f .deps/pixz.cpu.Tpo .deps/pixz.cpu.Po ^^^^^^ Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 0b51fc3..d9171bf 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -8,6 +8,40 @@ require_relative 'es_query' require_relative 'error_messages' require_relative "#{ENV['LKP_SRC']}/lib/common" +# previous_job_id's error_ids include error_id +# If later_job_id's output include error filename that extract from error_id, +# the later_job_id is credible. +def credible?(previous_job_id, later_job_id, error_id) + es = ESQuery.new + + previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') + later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') + + return false if filenames_check(previous_result_file, later_result_file, error_id).value?(false) + + return true +end + +def filenames_check(previous_result_file, later_result_file, error_id) + filenames = Set.new + filenames_check = Hash.new { |h, k| h[k] = false } + + error_lines = ErrorMessages.new(previous_result_file).obtain_error_messages_by_error_id(error_id, true) + error_lines.each do |error_line| + # "src/ssl_sock.c:1454:104: warning: unused parameter ��al�� [-Wunused-parameter]" => "src/ssl_sock.c" + filenames << $1 if error_line =~ /(.*)(:\d+){2}: (error|warning):/ + end + + File.open(later_result_file).each_line do |line| + filenames.each do |filename| + filenames_check[filename] + filenames_check[filename] = true if line.include?(filename) + end + end + + return filenames_check +end + def get_compare_result(previous_job_id, later_job_id) es = ESQuery.new -- 2.23.0

1 0

[PATCH v3 compass-ci 1/3] lib/error_messages.rb: obtain_error_messages_by_error_id function add keyword argument
by Lin Jiaxin 18 Jan '21

18 Jan '21

Determine whether to return the error line or error messages by the value of return_error_line Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/error_messages.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/error_messages.rb b/lib/error_messages.rb index 3e1564b..d83d890 100644 --- a/lib/error_messages.rb +++ b/lib/error_messages.rb @@ -40,12 +40,12 @@ class ErrorMessages return @error_messages end - def obtain_error_messages_by_error_id(error_id) + def obtain_error_messages_by_error_id(error_id, return_error_line = false) error_messages_by_error_id = [] error_messages = obtain_error_messages error_messages.each do |k, v| if ('build-pkg.' + build_pkg_error_id(k)) == error_id - error_messages_by_error_id += v.to_a + error_messages_by_error_id += return_error_line ? [k] : v.to_a end end error_messages_by_error_id -- 2.23.0

1 0

[PATCH v2 compass-ci] lib/job.cr: auto install dependency of lkp
by Wei Jihui 18 Jan '21

18 Jan '21

Note that the 'lkp' depends should best be pre-installed in each rootfs. So in theory (if that step is well done) only docker image need this, since we cannot modify 3rd party docker image. However to be robust, we can always include this. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 07bfda0..ae1f170 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -556,7 +556,7 @@ class Job private def get_depends_initrd(program_params, initrd_deps_arr, initrd_pkg_arr) initrd_http_prefix = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup - program_params["initramfs-common"] = JSON::Any.new("") if os_mount == "initramfs" + program_params["lkp"] = JSON::Any.new("") program_params.keys.each do |program| if program =~ /^(.*)-\d+$/ -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: delete initramfs-common
by Wei Jihui 18 Jan '21

18 Jan '21

distro/depends/lkp will pre-installed in each rootfs Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/initramfs-common | 4 ---- 1 file changed, 4 deletions(-) delete mode 100644 distro/depends/initramfs-common diff --git a/distro/depends/initramfs-common b/distro/depends/initramfs-common deleted file mode 100644 index 5f766da6..00000000 --- a/distro/depends/initramfs-common +++ /dev/null @@ -1,4 +0,0 @@ -time -wget -perf -which -- 2.23.0

1 0

[PATCH compass-ci] doc: sparrow/README.md: update format and content
by Liu Yinsi 18 Jan '21

18 Jan '21

1. git config, this info will be used to build account info, fix verification problem when submit job. 2. setenforce 0, if not set selinux, cci-network.service will not work right. 3. update rootfs download link. 4. division operation instruction Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/README.md | 68 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 21 deletions(-) diff --git a/sparrow/README.md b/sparrow/README.md index c7a80e7..f0fea3f 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -24,30 +24,49 @@ >openEuler ��[��](https://openeuler.org/zh/docs/20.03_LTS/docs/Installation/%E5%AE%89%E8%A3%85%E5%87%86%E5%A4%87.html) ### �� + +#### ��compass-ci + 1. �� openEuler �� -2. �� +2. ��git�� ```bash - mkdir demo && cd demo && umask 002 + git config --golbal user.name "XXX" + git config --golbal user.email "xxx(a)aa.com" + ``` + +3. ��SELINUX + ``` + umask 002 && setenforce 0 ``` -3. �� compass-ci �� demo �� + + >**��** + >setenforce 0 ��SELINUX��/etc/selinux/config��SELINUX=enforcing��SELINUX=permissive��disabled�� + >umask 002 ��umask��/etc/bashrc��umask��002�� + +4. �� compass-ci �� ```bash + mkdir /c/ && ln -s /c/compass-ci /c/cci git clone https://gitee.com/wu_fengguang/compass-ci.git ``` -4. �� install-tiny + +5. �� install-tiny ```bash cd compass-ci/sparrow && ./install-tiny ``` -5. ��lkp-aarch64.cgz�� + +#### �� + +1. ��lkp-aarch64.cgz�� ```bash cd /c/compass-ci/container/lkp-initrd && ./run ``` -6. �� +2. �� ```bash cd /c/compass-ci/sbin && ./build-my-info.rb - �� + ``` -7. ��job�� +3. ��job�� ```bash submit iperf.yaml testbox=vm-2p8g ``` @@ -55,33 +74,40 @@ ��: submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory - compass-ci��8��rootfs�� + compass-ci��4��rootfs�� -8. ��rootfs��openeuler/aarch64/20.03��/srv/os/��wget��cgz�� +4. ��rootfs��rootfs��[��](http://124.90.34.227:11300/os/)��cgz�� ```bash - mkdir -p /srv/os/openeuler/aarch64/ - cd /srv/os/openeuler/aarch64 - wget http://124.90.34.227:11300/os/test/openeuler/aarch64/20.03.cgz + mkdir -p /srv/os/openeuler/aarch64/20.03 + cd /srv/os/openeuler/aarch64/20.03 + wget http://124.90.34.227:11300/os/openeuler/aarch64/20.03.cgz ``` -9. ��cgz�� - ```bash +5. ��rootfs cgz �� + ``` gzip -dc 20.03.cgz | cpio -idv ``` -10. [�� compass-ci ��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-project.zh.md) +#### ��compass-ci +��iperf.yaml�� -11. [��](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) +1. [�� compass-ci ��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-project.zh.md) + +2. [��](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) + +3. ��[submit��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit-job.zh.md)�� + ``` + submit iperf.yaml + ``` -12. ��[submit��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit-job.zh.md)�� - ��os_mount ��cifs +#### �� -13. �� +1. �� ```bash cd /c/compass-ci/providers/ && ./my-qemu.sh ``` -14. [��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/browse-results.zh.md) +2. [��](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/browse-results.zh.md) ��compass-ci��srv/result��yaml��/��/$testbox/$os-$os-version-$os-arch/$job_id��output��(��tab��) ��iperf�� ```bash -- 2.23.0

1 0

[PATCH lkp-tests 2/4] distro/depends: add vim in initramfs-common
by Zhang Yale 18 Jan '21

18 Jan '21

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- distro/depends/initramfs-common | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/initramfs-common b/distro/depends/initramfs-common index 5f766da6..a2ccf411 100644 --- a/distro/depends/initramfs-common +++ b/distro/depends/initramfs-common @@ -2,3 +2,4 @@ time wget perf which +vim -- 2.23.0

2 3

[PATCH v3 compass-ci] lib/job.cr: enable the account check function
by Wu Zhende 18 Jan '21

18 Jan '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 03ca5af..305984c 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -392,15 +392,16 @@ class Job private def check_account_info error_msg = "Failed to verify the account.\n" error_msg += "Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" + account_info = @es.get_account(self["my_email"]) - # raise account_info unless account_info.is_a?(JSON::Any) + raise error_msg unless account_info.is_a?(JSON::Any) - # @account_info = account_info.as_h - @account_info = account_info.as_h if account_info.is_a?(JSON::Any) + @account_info = account_info.as_h - # raise error_msg if @account_info["found"]? == false - # raise error_msg unless self["my_name"] == @account_info["my_name"].to_s - # raise error_msg unless self["my_token"] == @account_info["my_token"] + # my_name can be nil in es + # my_token can't be nil in es + raise error_msg unless self["my_name"] == @account_info["my_name"]?.to_s + raise error_msg unless self["my_token"] == @account_info["my_token"]? @hash.delete("my_uuid") @hash.delete("my_token") -- 2.23.0

1 0

[PATCH compass-ci] container/extract-stats: fix start script
by Ren Wen 18 Jan '21

18 Jan '21

- LAB -> lab - Add 'SRV_HTTP_HOST', 'SRV_HTTP_PORT' to 'names', 'names' is envs what you will get from default configs. Signed-off-by: Ren Wen <15991987063(a)163.com> --- container/extract-stats/start | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/container/extract-stats/start b/container/extract-stats/start index 6510564..c158977 100755 --- a/container/extract-stats/start +++ b/container/extract-stats/start @@ -14,7 +14,9 @@ names = Set.new %w[ ES_PORT SEND_MAIL_HOST SEND_MAIL_PORT - LAB + SRV_HTTP_HOST + SRV_HTTP_PORT + lab ] defaults = relevant_defaults(names) -- 2.23.0

1 0

[PATCH compass-ci] providers/qemu: fix script hangs when $initrds is null
by Liu Yinsi 18 Jan '21

18 Jan '21

[Error Msg] ++ set_initrd ++ initrd=initrd ++ cat <<< hang here [Root Cause] Function set_initrd() have follow line: cat $initrds > $initrd when $initrds is null, the script will hang when execute cat command. [how] before: [ -n "$initrds" ] || exit after: [ -n "$initrds" ] && cat $initrds > initrd || exit [why] 1. before execute cat command, check wether initrds is null. 2. initrd is a brief file name, it can be used directly, so remove $initrd, just us initrd. 3. set_initrd() is unnecessary, so remove. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 854e913..52b10f2 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -60,13 +60,7 @@ check_option_value() # debian has both qemu-system-x86_64 and qemu-system-riscv64 command [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 - [ -n "$initrds" ] || exit -} - -set_initrd() -{ - initrd=initrd - cat $initrds > $initrd + [ -n "$initrds" ] && cat $initrds > initrd || exit } set_bios() @@ -125,7 +119,7 @@ public_option() kvm=( $qemu -kernel $kernel - -initrd $initrd + -initrd initrd -smp $nr_cpu -m $memory -rtc base=localtime @@ -183,7 +177,6 @@ run_qemu() set_options() { - set_initrd set_bios set_helper set_nic -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: default queue for libvirt
by Ren Wen 18 Jan '21

18 Jan '21

testbox : vt-2p8g -> tbox_group : vt-2p8g -> default queue : vt-2p8g.<arch> Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 03ca5af..7d08610 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -310,7 +310,7 @@ class Job # set default value self["queue"] = tbox_group - if tbox_group.to_s.starts_with?(/(vm|dc)-/) + if tbox_group.to_s.starts_with?(/(vm|dc|vt)-/) self["queue"] = "#{tbox_group}.#{arch}" end end -- 2.23.0

1 0