- Compass-ci - mailweb.openeuler.org

git count
by Wu Fengguang 02 Apr '21

02 Apr '21

Cao Xueliang is our code star, thanks Cao Xueliang! baijing@crystal ~/todo/bin% ./git-count 19 Cao Xueliang 9 Liu Yinsi 8 Wu Zhende 6 Liu Shaofei 6 Lu Weitao 6 Wang Yong 4 Xu Xijian 4 Cui Yili 4 Luan Shengde 3 Wei Jihui 3 Yu Chuan 3 Li Yuanchao 3 Lin Jiaxin 3 Li Ping 2 Wang Chenglong 1 Bai Jing 1 Zhang Dewan 0 Du Kaitian 0 Wu Fengguang 0 Zhang Yale

1 0

[PATCH lkp-tests] test/ansible_test: modify ansible code to output the module name
by Wang Chenglong 02 Apr '21

02 Apr '21

[Why] modify ansible code to output the name of the module for tasks execution. [How] print("{'action':'%s'}" % task_result.__dict__['_task_fields']['action']) {'action':'package'} Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/ansible_test | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/tests/ansible_test b/tests/ansible_test index 56926bbed..1c19c2b0f 100755 --- a/tests/ansible_test +++ b/tests/ansible_test @@ -58,7 +58,18 @@ ansible_playbook_run() EOF [ $? -eq 0 ] && { echo "Playbook has been generated!" - } + } + [ $(command -v ansible-playbook) ] || { + echo "ansible-playbook: command not found" + exit 1 + } + + sed -i "536a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + /usr/local/lib/python3.7/site-packages/ansible/plugins/strategy/__init__.py + + sed -i "550a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + /usr/local/lib/python3.7/site-packages/ansible/plugins/strategy/__init__.py + ansible-playbook "$ansible_path"/site.yml if [ $? -eq 0 ]; then -- 2.23.0

2 1

[PATCH v2 lkp-tests] stats/ansible_test: optimize the output of error_id
by Wang Chenglong 02 Apr '21

02 Apr '21

[Why] optimize the output of error_id standard format: @module_name:error_id before: error.[message].Could-not-find-or-access-RedHat-20-yml-Searched-in-root-ansible-roles-ansible-role-postgresql-vars-RedHat-20-yml-root-ansible- roles-ansible-role-postgresql-RedHat-20-yml-root-ansible-roles-ansible-role-postgresql-tasks-vars-RedHat-20-yml-root-ansible-roles-ansible-role- postgresql-tasks-RedHat-20-yml-root-ansible-vars-RedHat-20-yml-root-ansible-RedHat-20-yml-on-the-Ansible-Controller-If-you-are-using-a-module- and-expect-the-file-to-exist-on-the-remote-see-the-remote_src-option: 1 after: @inculde_vars:Could-not-find-or-access-RedHat-20-yml: 1 Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 101 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 85 insertions(+), 16 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 3d3b0b0cb..d7f084027 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,40 +1,109 @@ #!/usr/bin/env ruby require 'json' +require 'yaml' -def output_error(error_msg, error_key) - return if error_msg.nil? - +def output_error(error_msg) if error_msg.is_a? Array error_msg.each do |i| error_id = common_error_id i - puts "error.[#{error_key}].#{error_id}: 1" - puts "error.#{error_id}.message: #{@ansible_failed_info}" + puts "@#{@module_name['action']}:#{error_id}: 1" + puts "@#{@module_name['action']}:#{error_id}.message: #{@ansible_failed_info}" end elsif error_msg.is_a? String error_id = common_error_id error_msg - puts "error.[#{error_key}].#{error_id}: 1" - puts "error.#{error_id}.message: #{@ansible_failed_info}" + puts "@#{@module_name['action']}:#{error_id}: 1" + puts "@#{@module_name['action']}:#{error_id}.message: #{@ansible_failed_info}" end end def common_error_id(line) - line.gsub!(/[^\w]/, '-') + line.gsub!(/[^\w]/, '--') + line.gsub!(/[0-9]{3,}/, '#') line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one line.gsub!(/^-|-$/, '') line end +def parse_msg(ansible_failed_json) + case ansible_failed_json['msg'] + when /package (.*64)/ + output_error "cannot install the #{$1} for the job" + when /(The error appears to be in .*')/ + output_error $1 + when /(Failed to import the required Python library .*python3\.)/ + output_error $1 + # when /(An unhandled exception occurred while running the lookup plugin .*'\.)/ + when /original message: (.*)/ + output_error $1 + when /(Unable to start service (\w{1,}):)/ + output_error $1 + when /(Failed to download metadata for repo .*'):/ + output_error $1 + when /(Failed to download packages: (.*):)/ + output_error $2 + when /(Failed to find required .* in paths):/ + output_error $1 + when /(Failure downloading .*),/ + output_error $1 + when /(Could not find or access.*)'/ + output_error $1 + when /(Could not find .*):/ + output_error $1 + when /(Unsupported parameters) for/ + output_error $1 + when /aajdhch/ + output_error $1 + when /non-zero return code|Failed to install some of the specified packages/ + return + else + output_error ansible_failed_json['msg'] + end +end + +def parse_stderr(ansible_failed_json) + case ansible_failed_json['stderr'] + when /^$/ + return + else + ansible_failed_json['stderr_lines'].each do |i| + output_error i + end + end +end + +def parse_message(ansible_failed_json) + case ansible_failed_json['message'] + when /(Could not find or access.*)'/ + output_error $1 + else + output_error ansible_failed_json['message'] + end +end + +def parse_failures(ansible_failed_json) + ansible_failed_json['failures'].each do |i| + case i + when /(.*)/ + output_error $1 + end + end +end + while (line = STDIN.gets) - next unless line =~ /(FAILED!|failed:).*=>(.*)/ + case line + when /({'action'.*})/ + @module_name = YAML.load($1) + when /(FAILED!|failed:).*=>(.*)/ - @ansible_failed_info = $2 - next if @ansible_failed_info.empty? + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? - ansible_failed_json = JSON.parse @ansible_failed_info + ansible_failed_json = JSON.parse @ansible_failed_info - output_error ansible_failed_json['msg'],'msg' - output_error ansible_failed_json['message'],'message' - output_error ansible_failed_json['cmd'],'cmd' - output_error ansible_failed_json['failures'],'failures' + parse_msg ansible_failed_json unless ansible_failed_json['msg'].nil? + parse_stderr ansible_failed_json unless ansible_failed_json['stderr'].nil? + parse_message ansible_failed_json unless ansible_failed_json['message'].nil? + parse_failures ansible_failed_json unless ansible_failed_json['failures'].nil? + end end -- 2.23.0

2 1

[PATCH compass-ci] service/scheduler: modify the data saved in the testbox
by Wu Zhende 02 Apr '21

02 Apr '21

1. add name: for search in es 2. add type: vm/dc/physical 3. delete history content: no need this info testbox's history info can find in logging Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 19 ++++++++++++++++++- src/scheduler/elasticsearch_client.cr | 7 ++----- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 305d070..2e4fd30 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -160,18 +160,35 @@ class Sched state = "requesting" end + type = get_type(testbox) queues = JSON.parse(queues.to_json) hash = { "job_id" => job_id, "state" => state, "time" => get_time, "deadline" => deadline, - "queues" => queues + "queues" => queues, + "type" => type, + "name" => testbox } + @redis.update_wtmp(testbox.to_s, hash) @es.update_tbox(testbox.to_s, hash) end + def get_type(testbox) + return unless testbox + + if testbox.includes?("vm") + type = "vm" + elsif testbox.includes?("dc") + type = "dc" + else + type = "physical" + end + type + end + def report_ssh_port testbox = @env.params.query["tbox_name"] ssh_port = @env.params.query["ssh_port"].to_s diff --git a/src/scheduler/elasticsearch_client.cr b/src/scheduler/elasticsearch_client.cr index 219cf3d..f32256b 100644 --- a/src/scheduler/elasticsearch_client.cr +++ b/src/scheduler/elasticsearch_client.cr @@ -117,17 +117,14 @@ class Elasticsearch::Client result = @client.get_source(query) raise result unless result.is_a?(JSON::Any) - history = result["history"].as_a? result = result.as_h else result = wtmp_hash end - history ||= [] of JSON::Any - history << JSON.parse(wtmp_hash.to_json) unless wtmp_hash["state"]?.to_s == "requesting" - history = JSON.parse(history.to_json) - + history = JSON::Any.new([] of JSON::Any) body = { "history" => history} + body.any_merge!(result) body.any_merge!(wtmp_hash) -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/rpmbuild-pkg: add parameters annotation
by Wang Yong 02 Apr '21

02 Apr '21

it's for choosing parameters clearly Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/rpmbuild-pkg | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 7a101b6a0..a91ae4a08 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -3,6 +3,7 @@ # - compat_os # - repo_name # - repo_addr +# Git repo jobs only use the first two parameters, SRPMs jobs use all . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh -- 2.23.0

1 0

[PATCH v2 lkp-tests] stat: optimize to obtain error for build DockerFile
by Liu Shaofei 02 Apr '21

02 Apr '21

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- stats/openeuler_docker.rb | 101 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100755 stats/openeuler_docker.rb diff --git a/stats/openeuler_docker.rb b/stats/openeuler_docker.rb new file mode 100755 index 000000000..706b5b249 --- /dev/null +++ b/stats/openeuler_docker.rb @@ -0,0 +1,101 @@ +#!/usr/bin/env ruby + +def pre_handel(result, file_path) + status = false + repo_set = Set[] + sys_set = Set[] + + File.readlines(file_path).each do |line| + case line.chomp! + # Error: Unable to find a match: docker-registry mock xx + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + repo_set << repo + end + + # RUN yum -y swap -- remove fakesystemd -- install systemd systemd-libs + # yum swap: error: unrecognized arguments: install systemd systemd-libs + when /yum swap: error: .*: install (.+)/ + $1.split.each do |sys| + sys_set << sys + end + + # curl: (22) The requested URL returned error: 404 Not Found + # error: skipping https://dl.fedoraproject.org/pub/epel/bash-latest-7.noarch.rpm - transfer failed + when /.*error: .* (https.*)/ + result['requested-URL-returned.error'] = [1] + result['requested-URL-returned.error.message'] = [line.to_s] + status = true + + # Error: Unknown repo: 'powertools' + when /Error: Unknown repo: (.+)/ + repo = $1.delete!("'") + result["unknown-repo.#{repo}"] = [1] + result["unknown-repo.#{repo}.message"] = [line.to_s] + status = true + + # Error: Module or Group 'convert' does not exist. + when /Error: Module or Group ('[^\s]+')/ + repo = $1.delete!("'") + result["error.not-exist-module-or-group.#{repo}"] = [1] + result["error.not-exist-module-or-group.#{repo}.message"] = [line.to_s] + status = true + # /bin/sh: passwd: command not found + when /\/bin\/sh: (.+): command not found/ + result["sh.command-not-found.#{$1}"] = [1] + result["sh.command-not-found.#{$1}.message"] = [line.to_s] + status = true + end + + repo_set.each do |repo| + result["yum.error.Unable-to-find-a-match.#{repo}"] = [1] + result["yum.error.Unable-to-find-a-match.#{repo}.message"] = ["Error: Unable to find a match #{repo}"] + status = true + end + + sys_set.each do |sys| + result["yum.swap.error.unrecognized-arguments-install.#{sys}"] = [1] + result["yum.swap.error.unrecognized-arguments.#{sys}.message"] = + ["yum swap: error: unrecognized arguments install #{sys}"] + status = true + end + end + status +end + +def handle_unknown_error(_result, file_path) + line_num = %x(cat #{file_path} | grep -n 'Step ' | tail -1 | awk -F: '{print $1}') + + index = 1 + message = '' + File.readlines(file_path).each do |line| + if index == Integer(line_num) + message += line + else + index += 1 + end + end + + message = $1 if message =~ %r(\u001b\[91m(.+)) + message +end + +def openeuler_docker(log_lines) + result = Hash.new { |hash, key| hash[key] = [] } + + log_lines.each do |line| + next unless line =~ %r(([^\s]+).(build|run)\.fail) + + key, value = line.split(':') + key.chomp! + result[key] << value.to_i + + file_path = "#{RESULT_ROOT}/#{$1}" # $1 named by docker-image name + next unless File.exist?(file_path) + next if pre_handel(result, file_path) + + result["#{key}.message"] << handle_unknown_error(result, file_path) + end + + result +end -- 2.23.0

3 3

[PATCH compass-ci] sched: stop ectd watcher before close etcd client
by Cao Xueliang 01 Apr '21

01 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/find_job_boot.cr | 43 +++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 6e8d5d1..74a27af 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -129,13 +129,26 @@ class Sched jobs += job.kvs end + ec.close + return jobs, revisions.min end def consume_by_watch(queues, revision) ready_queues = split_ready_queues(queues) - channel = watch_queues(ready_queues, revision) - loop_handle_event(channel) + + channel = Channel(Array(Etcd::Model::WatchEvent)).new + ech = Hash(EtcdClient, Etcd::Watch::Watcher).new + ready_queues.each do |queue| + ec = EtcdClient.new + watcher = ec.watch_prefix(queue, start_revision: revision.to_i64, progress_notify: false, filters: [Etcd::Watch::Filter::NODELETE]) do |events| + channel.send(events) + end + ech[ec] = watcher + end + + watchers = start_watcher(ech) + loop_handle_event(channel, ech) end def split_ready_queues(queues) @@ -148,25 +161,25 @@ class Sched ready_queues.uniq end - def watch_queues(queues, revision) - puts "watch #{queues}, revision #{revision}" - channel = Channel(Array(Etcd::Model::WatchEvent)).new - queues.each do |queue| - watcher = EtcdClient.new.watch_prefix(queue, start_revision: revision.to_i64, progress_notify: false ,filters: [Etcd::Watch::Filter::NODELETE]) do |events| - channel.send(events) - end + def start_watcher(ech) + ech.each do |ec, watcher| spawn { watcher.start } Fiber.yield end - - return channel end - def loop_handle_event(channel) + def loop_handle_event(channel, ech) while true events = channel.receive events.each do |event| - return event.kv if ready2process(event.kv) + if ready2process(event.kv) + ech.each do |ec, watcher| + watcher.stop + ec.close + end + + return event.kv + end end end end @@ -176,7 +189,9 @@ class Sched f_queue = job.key t_queue = f_queue.gsub("/ready/", "/in_process/") value = job.value - ec.move(f_queue, t_queue, value) + res = ec.move(f_queue, t_queue, value) + ec.close + return res end def get_job_boot(host, boot_type) -- 2.23.0

1 0

[PATCH compass-ci] sched: close etcd client in every http request
by Cao Xueliang 01 Apr '21

01 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/etcd_client.cr | 4 ++++ src/lib/sched.cr | 4 ++++ src/scheduler/scheduler.cr | 4 ++++ 3 files changed, 12 insertions(+) diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr index 7891eb3..e0d4239 100644 --- a/src/lib/etcd_client.cr +++ b/src/lib/etcd_client.cr @@ -13,6 +13,10 @@ class EtcdClient @etcd = Etcd.client(host, port, version) end + def close + @etcd.close + end + def put(queue, content) queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) @etcd.kv.put_not_exists(queue, content) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6fd6657..7e01221 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -49,6 +49,10 @@ class Sched @log.info(%({"from": "#{(a)env.request.remote_address}", "response": #{response.to_json}})) end + def etcd_close + @etcd.close + end + def alive(version) debug_message("Env= {\n#{`export`}}") "LKP Alive! The time is #{Time.local}, version = #{version}" diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 6b032ee..ee13fd3 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -41,6 +41,10 @@ module Scheduler env.create_sched end + after_all do |env| + env.sched.etcd_close + end + # echo alive get "/" do |env| env.sched.alive(VERSION) -- 2.23.0

1 0

[PATCH compass-ci] providers/my-docker: use dc-8g for universality
by Lin Jiaxin 01 Apr '21

01 Apr '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- providers/my-docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/my-docker b/providers/my-docker index c8420c7..0a9f8b0 100755 --- a/providers/my-docker +++ b/providers/my-docker @@ -5,7 +5,7 @@ require_relative './docker/docker' -tbox_group = 'dc-1g' +tbox_group = 'dc-8g' hostname = "#{tbox_group}.#{ENV['USER']}-#{Process.pid}" # specify which queues will be request, use "," to separate more than 2 values queues = "#{tbox_group}~#{ENV['USER']}" -- 2.23.0

1 0

[PATCH v3 compass-ci] doc: update sparrow README.md
by Liu Yinsi 01 Apr '21

01 Apr '21

Locally deploy compass-ci just for sparrow, if users want to use other services, prompt users to apply for account and borrow machine. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/README.md | 60 ++++++++++++++++++----------------------------- 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/sparrow/README.md b/sparrow/README.md index d7b8e81..532b7f9 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -41,7 +41,7 @@ 3. 创建工作目录并克隆 compass-ci 项目代码 ```bash mkdir /c/ && ln -s /c/compass-ci /c/cci - git clone https://gitee.com/wu_fengguang/compass-ci.git + git clone https://gitee.com/wu_fengguang/compass-ci.git ``` 4. 编辑setup.yaml配置用户名和邮箱 @@ -57,53 +57,39 @@ cd compass-ci/sparrow && ./install-tiny ``` -#### 提交测试任务前的准备 - -1. 测试环境是否可以提交job测试 +#### 提交测试任务 +本文以/c/lkp-tests/jobs/目录下已有的通用测试用例host-info.yaml为例 +- 使环境变量生效 ```bash - submit iperf.yaml - ``` - - 执行上述命令正常情况下会提示信息如下: + source /etc/profile.d/compass-ci ``` - submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 - submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - ``` - compass-ci搭建完毕，下面就可以开始进行测试了。 - -#### 提交测试任务到本地compass-ci -本文以测试用例iperf.yaml为例 -1. [使用 compass-ci 平台测试开源项目](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… - -2. 根据测试需要[编写测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add… - -3. 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… +- 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… ```bash - submit iperf.yaml + submit host-info.yaml ``` -#### 运行测试任务并查看任务结果 - -1. 运行测试任务 - ```bash - cd /c/compass-ci/providers/ && ./my-qemu.sh + 执行上述命令会打印提示信息如下: + ``` + submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 + submit /c/lkp-tests/jobs/host-info.yaml, got job id=nolab.1 ``` -2. 在本地/srv/result/目录下根据测试用例名称/日期/[testbox](https://gitee.com/wu_fengguang/lab-… [查看任务结果](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/br… +- 查看任务结果 +等待约1分钟，可根据上一步骤中打印的job id查看任务结果。 ```bash - cd /srv/result/iperf/2020-12-29/vm-2p8g/openeuler-20.03-aarch64/nolab.1 - cat output + cd $(es-find id=nolab.1 |grep result_root|awk -F '"' '{print "/srv/"$4}') && ls ``` - >**说明：** - >[登陆测试环境调测任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + 结果文件介绍 + job.yaml文件 + job.yaml 文件中部分字段是用户提交上来的，其他字段是平台根据提交的 job 自动添加进来的。此文件包含了测试任务需要的所有参数。 -# FAQ + output文件 + output 文件记录了用例的执行过程，文件最后部分一般会有 check_exit_code 这个状态码，非 0 代表测试用例错误。 -* 选择下载os rootfs + stats.json + 测试用例执行完成会生成一个与测试用例同名的文件，记录它们的测试命令及标准化输出结果。compass-ci 会对这些文件进行解析，生成后缀名是 .json 的文件。 + stats.json 是所有的 json 文件的汇总，所有测试命令的关键结果都会统计到这个文件中，便于后续的比较和分析。 - 启动qemu测试机需要[os rootfs文件](http://api.compass-ci.openeuler.org:11304/os/)， - 一键部署默认下载os rootfs为openeuler aarch64 20.03，当部署完毕后，可在该目录下查看: /srv/os/openeuler/aarch64/20.03 - 如需要其他os rootfs，可使用该脚本下载: /c/compass-ci/sbin/download-rootfs +体验更多功能例如[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-project.zh.md)、[调测环境登录](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/log-in-machine-debug.md)、[测试结果分析](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/compare-results.zh.md)等，请[申请账号](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.zh.md)、[申请测试机](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/borrow-machine.zh.md)。 -- 2.23.0

1 0