- Compass-ci - mailweb.openeuler.org

review star
by Wu Fengguang 02 Apr '21

02 Apr '21

Luan Shengde is our review star, thanks Luan Shengde! baijing@crystal ~/todo/bin% ./review-star Review email records in the last 7 days total review paragraph point people 81 11 11 25 Cao Xueliang 59 12 19 12 Wei Jihui 24 13 28 5 Luan Shengde 22 7 6 3 Cui Yili 19 3 4 3 Wu Zhende 16 3 5 2 Yu Chuan 12 5 5 2 Xu Xijian 2 1 1 1 Wu Fengguang 25 0 0 1 Li Ping 17 1 1 0 Wang Chenglong 20 0 0 0 Liu Yinsi 6 0 0 0 Lu Weitao 1 0 0 0 Zhang Dewan 20 1 1 0 Wang Yong 6 0 0 0 Li Yuanchao 1 0 0 0 Bai Jing 9 4 4 0 Lin Jiaxin 18 5 4 0 Liu Shaofei

2 1

help star
by Wu Fengguang 02 Apr '21

02 Apr '21

Wu Zhende, Yu Chuan is our help star again, thanks Wu Zhende, Yu Chuan! baijing@crystal ~/todo/bin% ./help-star ../people/*/credit/2021-04-01.md 14.3 Wu Zhende 14.2 Yu Chuan 14.0 Wei Jihui 11.3 Wu Fengguang 11.0 Liu Shaofei 10.7 Wang Yong 10.2 Zhang Dewan 9.4 Lin Jiaxin 7.5 Luan Shengde 6.8 Cao Xueliang 5.8 Liu Yinsi 5.4 Bai Jing 4.4 Lu Weitao 4.2 Du Kaitian 4.0 Li Ping 2.2 Cui Yili 1.8 Wang Chenglong 1.5 Li Yuanchao 1.4 Xu Xijian 0.0 Zhang Yale

1 0

git count
by Wu Fengguang 02 Apr '21

02 Apr '21

Cao Xueliang is our code star, thanks Cao Xueliang! baijing@crystal ~/todo/bin% ./git-count 19 Cao Xueliang 9 Liu Yinsi 8 Wu Zhende 6 Liu Shaofei 6 Lu Weitao 6 Wang Yong 4 Xu Xijian 4 Cui Yili 4 Luan Shengde 3 Wei Jihui 3 Yu Chuan 3 Li Yuanchao 3 Lin Jiaxin 3 Li Ping 2 Wang Chenglong 1 Bai Jing 1 Zhang Dewan 0 Du Kaitian 0 Wu Fengguang 0 Zhang Yale

1 0

[PATCH lkp-tests] test/ansible_test: modify ansible code to output the module name
by Wang Chenglong 02 Apr '21

02 Apr '21

[Why] modify ansible code to output the name of the module for tasks execution. [How] print("{'action':'%s'}" % task_result.__dict__['_task_fields']['action']) {'action':'package'} Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/ansible_test | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/tests/ansible_test b/tests/ansible_test index 56926bbed..1c19c2b0f 100755 --- a/tests/ansible_test +++ b/tests/ansible_test @@ -58,7 +58,18 @@ ansible_playbook_run() EOF [ $? -eq 0 ] && { echo "Playbook has been generated!" - } + } + [ $(command -v ansible-playbook) ] || { + echo "ansible-playbook: command not found" + exit 1 + } + + sed -i "536a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + /usr/local/lib/python3.7/site-packages/ansible/plugins/strategy/__init__.py + + sed -i "550a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + /usr/local/lib/python3.7/site-packages/ansible/plugins/strategy/__init__.py + ansible-playbook "$ansible_path"/site.yml if [ $? -eq 0 ]; then -- 2.23.0

2 1

[PATCH v2 lkp-tests] stats/ansible_test: optimize the output of error_id
by Wang Chenglong 02 Apr '21

02 Apr '21

[Why] optimize the output of error_id standard format: @module_name:error_id before: error.[message].Could-not-find-or-access-RedHat-20-yml-Searched-in-root-ansible-roles-ansible-role-postgresql-vars-RedHat-20-yml-root-ansible- roles-ansible-role-postgresql-RedHat-20-yml-root-ansible-roles-ansible-role-postgresql-tasks-vars-RedHat-20-yml-root-ansible-roles-ansible-role- postgresql-tasks-RedHat-20-yml-root-ansible-vars-RedHat-20-yml-root-ansible-RedHat-20-yml-on-the-Ansible-Controller-If-you-are-using-a-module- and-expect-the-file-to-exist-on-the-remote-see-the-remote_src-option: 1 after: @inculde_vars:Could-not-find-or-access-RedHat-20-yml: 1 Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 101 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 85 insertions(+), 16 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 3d3b0b0cb..d7f084027 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,40 +1,109 @@ #!/usr/bin/env ruby require 'json' +require 'yaml' -def output_error(error_msg, error_key) - return if error_msg.nil? - +def output_error(error_msg) if error_msg.is_a? Array error_msg.each do |i| error_id = common_error_id i - puts "error.[#{error_key}].#{error_id}: 1" - puts "error.#{error_id}.message: #{@ansible_failed_info}" + puts "@#{@module_name['action']}:#{error_id}: 1" + puts "@#{@module_name['action']}:#{error_id}.message: #{@ansible_failed_info}" end elsif error_msg.is_a? String error_id = common_error_id error_msg - puts "error.[#{error_key}].#{error_id}: 1" - puts "error.#{error_id}.message: #{@ansible_failed_info}" + puts "@#{@module_name['action']}:#{error_id}: 1" + puts "@#{@module_name['action']}:#{error_id}.message: #{@ansible_failed_info}" end end def common_error_id(line) - line.gsub!(/[^\w]/, '-') + line.gsub!(/[^\w]/, '--') + line.gsub!(/[0-9]{3,}/, '#') line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one line.gsub!(/^-|-$/, '') line end +def parse_msg(ansible_failed_json) + case ansible_failed_json['msg'] + when /package (.*64)/ + output_error "cannot install the #{$1} for the job" + when /(The error appears to be in .*')/ + output_error $1 + when /(Failed to import the required Python library .*python3\.)/ + output_error $1 + # when /(An unhandled exception occurred while running the lookup plugin .*'\.)/ + when /original message: (.*)/ + output_error $1 + when /(Unable to start service (\w{1,}):)/ + output_error $1 + when /(Failed to download metadata for repo .*'):/ + output_error $1 + when /(Failed to download packages: (.*):)/ + output_error $2 + when /(Failed to find required .* in paths):/ + output_error $1 + when /(Failure downloading .*),/ + output_error $1 + when /(Could not find or access.*)'/ + output_error $1 + when /(Could not find .*):/ + output_error $1 + when /(Unsupported parameters) for/ + output_error $1 + when /aajdhch/ + output_error $1 + when /non-zero return code|Failed to install some of the specified packages/ + return + else + output_error ansible_failed_json['msg'] + end +end + +def parse_stderr(ansible_failed_json) + case ansible_failed_json['stderr'] + when /^$/ + return + else + ansible_failed_json['stderr_lines'].each do |i| + output_error i + end + end +end + +def parse_message(ansible_failed_json) + case ansible_failed_json['message'] + when /(Could not find or access.*)'/ + output_error $1 + else + output_error ansible_failed_json['message'] + end +end + +def parse_failures(ansible_failed_json) + ansible_failed_json['failures'].each do |i| + case i + when /(.*)/ + output_error $1 + end + end +end + while (line = STDIN.gets) - next unless line =~ /(FAILED!|failed:).*=>(.*)/ + case line + when /({'action'.*})/ + @module_name = YAML.load($1) + when /(FAILED!|failed:).*=>(.*)/ - @ansible_failed_info = $2 - next if @ansible_failed_info.empty? + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? - ansible_failed_json = JSON.parse @ansible_failed_info + ansible_failed_json = JSON.parse @ansible_failed_info - output_error ansible_failed_json['msg'],'msg' - output_error ansible_failed_json['message'],'message' - output_error ansible_failed_json['cmd'],'cmd' - output_error ansible_failed_json['failures'],'failures' + parse_msg ansible_failed_json unless ansible_failed_json['msg'].nil? + parse_stderr ansible_failed_json unless ansible_failed_json['stderr'].nil? + parse_message ansible_failed_json unless ansible_failed_json['message'].nil? + parse_failures ansible_failed_json unless ansible_failed_json['failures'].nil? + end end -- 2.23.0

2 1

[PATCH compass-ci] service/scheduler: modify the data saved in the testbox
by Wu Zhende 02 Apr '21

02 Apr '21

1. add name: for search in es 2. add type: vm/dc/physical 3. delete history content: no need this info testbox's history info can find in logging Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 19 ++++++++++++++++++- src/scheduler/elasticsearch_client.cr | 7 ++----- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 305d070..2e4fd30 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -160,18 +160,35 @@ class Sched state = "requesting" end + type = get_type(testbox) queues = JSON.parse(queues.to_json) hash = { "job_id" => job_id, "state" => state, "time" => get_time, "deadline" => deadline, - "queues" => queues + "queues" => queues, + "type" => type, + "name" => testbox } + @redis.update_wtmp(testbox.to_s, hash) @es.update_tbox(testbox.to_s, hash) end + def get_type(testbox) + return unless testbox + + if testbox.includes?("vm") + type = "vm" + elsif testbox.includes?("dc") + type = "dc" + else + type = "physical" + end + type + end + def report_ssh_port testbox = @env.params.query["tbox_name"] ssh_port = @env.params.query["ssh_port"].to_s diff --git a/src/scheduler/elasticsearch_client.cr b/src/scheduler/elasticsearch_client.cr index 219cf3d..f32256b 100644 --- a/src/scheduler/elasticsearch_client.cr +++ b/src/scheduler/elasticsearch_client.cr @@ -117,17 +117,14 @@ class Elasticsearch::Client result = @client.get_source(query) raise result unless result.is_a?(JSON::Any) - history = result["history"].as_a? result = result.as_h else result = wtmp_hash end - history ||= [] of JSON::Any - history << JSON.parse(wtmp_hash.to_json) unless wtmp_hash["state"]?.to_s == "requesting" - history = JSON.parse(history.to_json) - + history = JSON::Any.new([] of JSON::Any) body = { "history" => history} + body.any_merge!(result) body.any_merge!(wtmp_hash) -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/rpmbuild-pkg: add parameters annotation
by Wang Yong 02 Apr '21

02 Apr '21

it's for choosing parameters clearly Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/rpmbuild-pkg | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 7a101b6a0..a91ae4a08 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -3,6 +3,7 @@ # - compat_os # - repo_name # - repo_addr +# Git repo jobs only use the first two parameters, SRPMs jobs use all . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh -- 2.23.0

1 0

[PATCH v2 lkp-tests] stat: optimize to obtain error for build DockerFile
by Liu Shaofei 02 Apr '21

02 Apr '21

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- stats/openeuler_docker.rb | 101 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100755 stats/openeuler_docker.rb diff --git a/stats/openeuler_docker.rb b/stats/openeuler_docker.rb new file mode 100755 index 000000000..706b5b249 --- /dev/null +++ b/stats/openeuler_docker.rb @@ -0,0 +1,101 @@ +#!/usr/bin/env ruby + +def pre_handel(result, file_path) + status = false + repo_set = Set[] + sys_set = Set[] + + File.readlines(file_path).each do |line| + case line.chomp! + # Error: Unable to find a match: docker-registry mock xx + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + repo_set << repo + end + + # RUN yum -y swap -- remove fakesystemd -- install systemd systemd-libs + # yum swap: error: unrecognized arguments: install systemd systemd-libs + when /yum swap: error: .*: install (.+)/ + $1.split.each do |sys| + sys_set << sys + end + + # curl: (22) The requested URL returned error: 404 Not Found + # error: skipping https://dl.fedoraproject.org/pub/epel/bash-latest-7.noarch.rpm - transfer failed + when /.*error: .* (https.*)/ + result['requested-URL-returned.error'] = [1] + result['requested-URL-returned.error.message'] = [line.to_s] + status = true + + # Error: Unknown repo: 'powertools' + when /Error: Unknown repo: (.+)/ + repo = $1.delete!("'") + result["unknown-repo.#{repo}"] = [1] + result["unknown-repo.#{repo}.message"] = [line.to_s] + status = true + + # Error: Module or Group 'convert' does not exist. + when /Error: Module or Group ('[^\s]+')/ + repo = $1.delete!("'") + result["error.not-exist-module-or-group.#{repo}"] = [1] + result["error.not-exist-module-or-group.#{repo}.message"] = [line.to_s] + status = true + # /bin/sh: passwd: command not found + when /\/bin\/sh: (.+): command not found/ + result["sh.command-not-found.#{$1}"] = [1] + result["sh.command-not-found.#{$1}.message"] = [line.to_s] + status = true + end + + repo_set.each do |repo| + result["yum.error.Unable-to-find-a-match.#{repo}"] = [1] + result["yum.error.Unable-to-find-a-match.#{repo}.message"] = ["Error: Unable to find a match #{repo}"] + status = true + end + + sys_set.each do |sys| + result["yum.swap.error.unrecognized-arguments-install.#{sys}"] = [1] + result["yum.swap.error.unrecognized-arguments.#{sys}.message"] = + ["yum swap: error: unrecognized arguments install #{sys}"] + status = true + end + end + status +end + +def handle_unknown_error(_result, file_path) + line_num = %x(cat #{file_path} | grep -n 'Step ' | tail -1 | awk -F: '{print $1}') + + index = 1 + message = '' + File.readlines(file_path).each do |line| + if index == Integer(line_num) + message += line + else + index += 1 + end + end + + message = $1 if message =~ %r(\u001b\[91m(.+)) + message +end + +def openeuler_docker(log_lines) + result = Hash.new { |hash, key| hash[key] = [] } + + log_lines.each do |line| + next unless line =~ %r(([^\s]+).(build|run)\.fail) + + key, value = line.split(':') + key.chomp! + result[key] << value.to_i + + file_path = "#{RESULT_ROOT}/#{$1}" # $1 named by docker-image name + next unless File.exist?(file_path) + next if pre_handel(result, file_path) + + result["#{key}.message"] << handle_unknown_error(result, file_path) + end + + result +end -- 2.23.0

3 3

[PATCH compass-ci] sched: stop ectd watcher before close etcd client
by Cao Xueliang 01 Apr '21

01 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/scheduler/find_job_boot.cr | 43 +++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 6e8d5d1..74a27af 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -129,13 +129,26 @@ class Sched jobs += job.kvs end + ec.close + return jobs, revisions.min end def consume_by_watch(queues, revision) ready_queues = split_ready_queues(queues) - channel = watch_queues(ready_queues, revision) - loop_handle_event(channel) + + channel = Channel(Array(Etcd::Model::WatchEvent)).new + ech = Hash(EtcdClient, Etcd::Watch::Watcher).new + ready_queues.each do |queue| + ec = EtcdClient.new + watcher = ec.watch_prefix(queue, start_revision: revision.to_i64, progress_notify: false, filters: [Etcd::Watch::Filter::NODELETE]) do |events| + channel.send(events) + end + ech[ec] = watcher + end + + watchers = start_watcher(ech) + loop_handle_event(channel, ech) end def split_ready_queues(queues) @@ -148,25 +161,25 @@ class Sched ready_queues.uniq end - def watch_queues(queues, revision) - puts "watch #{queues}, revision #{revision}" - channel = Channel(Array(Etcd::Model::WatchEvent)).new - queues.each do |queue| - watcher = EtcdClient.new.watch_prefix(queue, start_revision: revision.to_i64, progress_notify: false ,filters: [Etcd::Watch::Filter::NODELETE]) do |events| - channel.send(events) - end + def start_watcher(ech) + ech.each do |ec, watcher| spawn { watcher.start } Fiber.yield end - - return channel end - def loop_handle_event(channel) + def loop_handle_event(channel, ech) while true events = channel.receive events.each do |event| - return event.kv if ready2process(event.kv) + if ready2process(event.kv) + ech.each do |ec, watcher| + watcher.stop + ec.close + end + + return event.kv + end end end end @@ -176,7 +189,9 @@ class Sched f_queue = job.key t_queue = f_queue.gsub("/ready/", "/in_process/") value = job.value - ec.move(f_queue, t_queue, value) + res = ec.move(f_queue, t_queue, value) + ec.close + return res end def get_job_boot(host, boot_type) -- 2.23.0

1 0

[PATCH compass-ci] sched: close etcd client in every http request
by Cao Xueliang 01 Apr '21

01 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/etcd_client.cr | 4 ++++ src/lib/sched.cr | 4 ++++ src/scheduler/scheduler.cr | 4 ++++ 3 files changed, 12 insertions(+) diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr index 7891eb3..e0d4239 100644 --- a/src/lib/etcd_client.cr +++ b/src/lib/etcd_client.cr @@ -13,6 +13,10 @@ class EtcdClient @etcd = Etcd.client(host, port, version) end + def close + @etcd.close + end + def put(queue, content) queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) @etcd.kv.put_not_exists(queue, content) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6fd6657..7e01221 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -49,6 +49,10 @@ class Sched @log.info(%({"from": "#{(a)env.request.remote_address}", "response": #{response.to_json}})) end + def etcd_close + @etcd.close + end + def alive(version) debug_message("Env= {\n#{`export`}}") "LKP Alive! The time is #{Time.local}, version = #{version}" diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 6b032ee..ee13fd3 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -41,6 +41,10 @@ module Scheduler env.create_sched end + after_all do |env| + env.sched.etcd_close + end + # echo alive get "/" do |env| env.sched.alive(VERSION) -- 2.23.0

1 0

[PATCH compass-ci] providers/my-docker: use dc-8g for universality
by Lin Jiaxin 01 Apr '21

01 Apr '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- providers/my-docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/my-docker b/providers/my-docker index c8420c7..0a9f8b0 100755 --- a/providers/my-docker +++ b/providers/my-docker @@ -5,7 +5,7 @@ require_relative './docker/docker' -tbox_group = 'dc-1g' +tbox_group = 'dc-8g' hostname = "#{tbox_group}.#{ENV['USER']}-#{Process.pid}" # specify which queues will be request, use "," to separate more than 2 values queues = "#{tbox_group}~#{ENV['USER']}" -- 2.23.0

1 0

[PATCH v3 compass-ci] doc: update sparrow README.md
by Liu Yinsi 01 Apr '21

01 Apr '21

Locally deploy compass-ci just for sparrow, if users want to use other services, prompt users to apply for account and borrow machine. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/README.md | 60 ++++++++++++++++++----------------------------- 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/sparrow/README.md b/sparrow/README.md index d7b8e81..532b7f9 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -41,7 +41,7 @@ 3. 创建工作目录并克隆 compass-ci 项目代码 ```bash mkdir /c/ && ln -s /c/compass-ci /c/cci - git clone https://gitee.com/wu_fengguang/compass-ci.git + git clone https://gitee.com/wu_fengguang/compass-ci.git ``` 4. 编辑setup.yaml配置用户名和邮箱 @@ -57,53 +57,39 @@ cd compass-ci/sparrow && ./install-tiny ``` -#### 提交测试任务前的准备 - -1. 测试环境是否可以提交job测试 +#### 提交测试任务 +本文以/c/lkp-tests/jobs/目录下已有的通用测试用例host-info.yaml为例 +- 使环境变量生效 ```bash - submit iperf.yaml - ``` - - 执行上述命令正常情况下会提示信息如下: + source /etc/profile.d/compass-ci ``` - submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 - submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - ``` - compass-ci搭建完毕，下面就可以开始进行测试了。 - -#### 提交测试任务到本地compass-ci -本文以测试用例iperf.yaml为例 -1. [使用 compass-ci 平台测试开源项目](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… - -2. 根据测试需要[编写测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add… - -3. 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… +- 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… ```bash - submit iperf.yaml + submit host-info.yaml ``` -#### 运行测试任务并查看任务结果 - -1. 运行测试任务 - ```bash - cd /c/compass-ci/providers/ && ./my-qemu.sh + 执行上述命令会打印提示信息如下: + ``` + submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 + submit /c/lkp-tests/jobs/host-info.yaml, got job id=nolab.1 ``` -2. 在本地/srv/result/目录下根据测试用例名称/日期/[testbox](https://gitee.com/wu_fengguang/lab-… [查看任务结果](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/br… +- 查看任务结果 +等待约1分钟，可根据上一步骤中打印的job id查看任务结果。 ```bash - cd /srv/result/iperf/2020-12-29/vm-2p8g/openeuler-20.03-aarch64/nolab.1 - cat output + cd $(es-find id=nolab.1 |grep result_root|awk -F '"' '{print "/srv/"$4}') && ls ``` - >**说明：** - >[登陆测试环境调测任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + 结果文件介绍 + job.yaml文件 + job.yaml 文件中部分字段是用户提交上来的，其他字段是平台根据提交的 job 自动添加进来的。此文件包含了测试任务需要的所有参数。 -# FAQ + output文件 + output 文件记录了用例的执行过程，文件最后部分一般会有 check_exit_code 这个状态码，非 0 代表测试用例错误。 -* 选择下载os rootfs + stats.json + 测试用例执行完成会生成一个与测试用例同名的文件，记录它们的测试命令及标准化输出结果。compass-ci 会对这些文件进行解析，生成后缀名是 .json 的文件。 + stats.json 是所有的 json 文件的汇总，所有测试命令的关键结果都会统计到这个文件中，便于后续的比较和分析。 - 启动qemu测试机需要[os rootfs文件](http://api.compass-ci.openeuler.org:11304/os/)， - 一键部署默认下载os rootfs为openeuler aarch64 20.03，当部署完毕后，可在该目录下查看: /srv/os/openeuler/aarch64/20.03 - 如需要其他os rootfs，可使用该脚本下载: /c/compass-ci/sbin/download-rootfs +体验更多功能例如[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-project.zh.md)、[调测环境登录](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/log-in-machine-debug.md)、[测试结果分析](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/compare-results.zh.md)等，请[申请账号](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.zh.md)、[申请测试机](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/borrow-machine.zh.md)。 -- 2.23.0

1 0

[PATCH compass-ci] service/scheduler: add queues to testbox
by Wu Zhende 01 Apr '21

01 Apr '21

Save data like this "queues":["sched/dc-1g~wuzhende/ready"] to ES. So we can collect testbox information of a queue. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 6 ++++-- src/scheduler/find_job_boot.cr | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6fd6657..c84b3f3 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -143,7 +143,7 @@ class Sched @log.warn(e) end - def set_lifecycle(job, testbox) + def set_lifecycle(job, testbox, queues) if job deadline = job.get_deadline job["deadline"] = deadline @@ -156,11 +156,13 @@ class Sched state = "requesting" end + queues = JSON.parse(queues.to_json) hash = { "job_id" => job_id, "state" => state, "time" => get_time, - "deadline" => deadline + "deadline" => deadline, + "queues" => queues } @redis.update_wtmp(testbox.to_s, hash) @es.update_tbox(testbox.to_s, hash) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 6e8d5d1..ff88d32 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -182,7 +182,7 @@ class Sched def get_job_boot(host, boot_type) queues = get_queues(host) job = get_job_from_queues(queues, host) - set_lifecycle(job, host) + set_lifecycle(job, host, queues) if job @es.set_job_content(job) -- 2.23.0

1 0

[PATCH lkp-tests] tests/rpmbuild-pkg: add parameters annotation
by Wang Yong 01 Apr '21

01 Apr '21

it's for choosing parameters clearly Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/rpmbuild-pkg | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 7a101b6a0..a3868d54a 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -1,6 +1,7 @@ #!/bin/bash # - upstream_repo # - compat_os +# Git repo jobs only use the above parameters, SRPMs jobs use all # - repo_name # - repo_addr -- 2.23.0

1 0

[PATCH v3 lkp-tests] stat: optimize to obtain error for build DockerFile
by Liu Shaofei 01 Apr '21

01 Apr '21

[output] catch errors will be like this: { "openeuler_docker.wordpress_1.build.fail": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.php-mysql": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.php-mysql.message": [ "Error: Unable to find a match php-mysql" ], "openeuler_docker.tools_2.build.fail": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.netsniff-ng": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.netsniff-ng.message": [ "Error: Unable to find a match netsniff-ng" ], "openeuler_docker.yum.error.Unable-to-find-a-match.rpm-ostree": [ 1 ], ...... ...... } Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- stats/openeuler_docker.rb | 103 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100755 stats/openeuler_docker.rb diff --git a/stats/openeuler_docker.rb b/stats/openeuler_docker.rb new file mode 100755 index 000000000..5a9365938 --- /dev/null +++ b/stats/openeuler_docker.rb @@ -0,0 +1,103 @@ +#!/usr/bin/env ruby + +def pre_handel(result, file_path) + status = false + repo_set = Set[] + sys_set = Set[] + + File.readlines(file_path).each do |line| + case line.chomp! + + # Error: Unable to find a match: docker-registry mock xx + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + repo_set << repo + end + + # RUN yum -y swap -- remove fakesystemd -- install systemd systemd-libs + # yum swap: error: unrecognized arguments: install systemd systemd-libs + when /yum swap: error: .*: install (.+)/ + $1.split.each do |sys| + sys_set << sys + end + + # curl: (22) The requested URL returned error: 404 Not Found + # error: skipping https://dl.fedoraproject.org/pub/epel/bash-latest-7.noarch.rpm - transfer failed + when /.*error: .* (https.*)/ + result['requested-URL-returned.error'] = [1] + result['requested-URL-returned.error.message'] = [line.to_s] + status = true + + # Error: Unknown repo: 'powertools' + when /Error: Unknown repo: (.+)/ + repo = $1.delete!("'") + result["unknown-repo.#{repo}"] = [1] + result["unknown-repo.#{repo}.message"] = [line.to_s] + status = true + + # Error: Module or Group 'convert' does not exist. + when /Error: Module or Group ('[^\s]+')/ + repo = $1.delete!("'") + result["error.not-exist-module-or-group.#{repo}"] = [1] + result["error.not-exist-module-or-group.#{repo}.message"] = [line.to_s] + status = true + + # /bin/sh: passwd: command not found + when /\/bin\/sh: (.+): command not found/ + result["sh.command-not-found.#{$1}"] = [1] + result["sh.command-not-found.#{$1}.message"] = [line.to_s] + status = true + end + + repo_set.each do |repo| + result["yum.error.Unable-to-find-a-match.#{repo}"] = [1] + result["yum.error.Unable-to-find-a-match.#{repo}.message"] = ["Error: Unable to find a match #{repo}"] + status = true + end + + sys_set.each do |sys| + result["yum.swap.error.unrecognized-arguments-install.#{sys}"] = [1] + result["yum.swap.error.unrecognized-arguments.#{sys}.message"] = + ["yum swap: error: unrecognized arguments install #{sys}"] + status = true + end + end + status +end + +def handle_unknown_error(_result, file_path) + line_num = %x(cat #{file_path} | grep -n 'Step ' | tail -1 | awk -F: '{print $1}') + + index = 1 + message = '' + File.readlines(file_path).each do |line| + if index == Integer(line_num) + message += line + else + index += 1 + end + end + + message = $1 if message =~ %r(\u001b\[91m(.+)) + message +end + +def openeuler_docker(log_lines) + result = Hash.new { |hash, key| hash[key] = [] } + + log_lines.each do |line| + next unless line =~ %r(([^\s]+).(build|run)\.fail) + + key, value = line.split(':') + key.chomp! + result[key] << value.to_i + + file_path = "#{RESULT_ROOT}/#{$1}" # $1 named by docker-image name + next unless File.exist?(file_path) + next if pre_handel(result, file_path) + + result["#{key}.message"] << handle_unknown_error(result, file_path) + end + + result +end -- 2.23.0

1 0

[PATCH v3 lkp-tests] stat: optimize to obtain error for build DockerFile
by Liu Shaofei 01 Apr '21

01 Apr '21

[output] catch errors will be like this: { "openeuler_docker.wordpress_1.build.fail": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.php-mysql": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.php-mysql.message": [ "Error: Unable to find a match php-mysql" ], "openeuler_docker.tools_2.build.fail": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.netsniff-ng": [ 1 ], "openeuler_docker.yum.error.Unable-to-find-a-match.netsniff-ng.message": [ "Error: Unable to find a match netsniff-ng" ], "openeuler_docker.yum.error.Unable-to-find-a-match.rpm-ostree": [ 1 ], } Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- stats/openeuler_docker.rb | 103 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100755 stats/openeuler_docker.rb diff --git a/stats/openeuler_docker.rb b/stats/openeuler_docker.rb new file mode 100755 index 000000000..5a9365938 --- /dev/null +++ b/stats/openeuler_docker.rb @@ -0,0 +1,103 @@ +#!/usr/bin/env ruby + +def pre_handel(result, file_path) + status = false + repo_set = Set[] + sys_set = Set[] + + File.readlines(file_path).each do |line| + case line.chomp! + + # Error: Unable to find a match: docker-registry mock xx + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + repo_set << repo + end + + # RUN yum -y swap -- remove fakesystemd -- install systemd systemd-libs + # yum swap: error: unrecognized arguments: install systemd systemd-libs + when /yum swap: error: .*: install (.+)/ + $1.split.each do |sys| + sys_set << sys + end + + # curl: (22) The requested URL returned error: 404 Not Found + # error: skipping https://dl.fedoraproject.org/pub/epel/bash-latest-7.noarch.rpm - transfer failed + when /.*error: .* (https.*)/ + result['requested-URL-returned.error'] = [1] + result['requested-URL-returned.error.message'] = [line.to_s] + status = true + + # Error: Unknown repo: 'powertools' + when /Error: Unknown repo: (.+)/ + repo = $1.delete!("'") + result["unknown-repo.#{repo}"] = [1] + result["unknown-repo.#{repo}.message"] = [line.to_s] + status = true + + # Error: Module or Group 'convert' does not exist. + when /Error: Module or Group ('[^\s]+')/ + repo = $1.delete!("'") + result["error.not-exist-module-or-group.#{repo}"] = [1] + result["error.not-exist-module-or-group.#{repo}.message"] = [line.to_s] + status = true + + # /bin/sh: passwd: command not found + when /\/bin\/sh: (.+): command not found/ + result["sh.command-not-found.#{$1}"] = [1] + result["sh.command-not-found.#{$1}.message"] = [line.to_s] + status = true + end + + repo_set.each do |repo| + result["yum.error.Unable-to-find-a-match.#{repo}"] = [1] + result["yum.error.Unable-to-find-a-match.#{repo}.message"] = ["Error: Unable to find a match #{repo}"] + status = true + end + + sys_set.each do |sys| + result["yum.swap.error.unrecognized-arguments-install.#{sys}"] = [1] + result["yum.swap.error.unrecognized-arguments.#{sys}.message"] = + ["yum swap: error: unrecognized arguments install #{sys}"] + status = true + end + end + status +end + +def handle_unknown_error(_result, file_path) + line_num = %x(cat #{file_path} | grep -n 'Step ' | tail -1 | awk -F: '{print $1}') + + index = 1 + message = '' + File.readlines(file_path).each do |line| + if index == Integer(line_num) + message += line + else + index += 1 + end + end + + message = $1 if message =~ %r(\u001b\[91m(.+)) + message +end + +def openeuler_docker(log_lines) + result = Hash.new { |hash, key| hash[key] = [] } + + log_lines.each do |line| + next unless line =~ %r(([^\s]+).(build|run)\.fail) + + key, value = line.split(':') + key.chomp! + result[key] << value.to_i + + file_path = "#{RESULT_ROOT}/#{$1}" # $1 named by docker-image name + next unless File.exist?(file_path) + next if pre_handel(result, file_path) + + result["#{key}.message"] << handle_unknown_error(result, file_path) + end + + result +end -- 2.23.0

1 1

[PATCH compass-ci] doc: README: replace invalid url
by Liu Yinsi 01 Apr '21

01 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- README.en.md | 8 ++++---- README.zh.md | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.en.md b/README.en.md index fa88305..7449696 100644 --- a/README.en.md +++ b/README.en.md @@ -12,7 +12,7 @@ Compass-CI monitors git repos of a large amount of open source software. Once Co **Login to the Test Environment** -The SSH is used to [log in to the test environment for commissioning](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma…. +The SSH is used to [log in to the test environment for commissioning](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma…. **Test Result Analysis** @@ -38,15 +38,15 @@ If a new error ID is generated, the bisect is automatically triggered to locate **Manually Submitting a Test Task** -1. [Install the Compass-CI client](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E…. +1. [Install the Compass-CI client](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/in…. 2. [Compile test cases and manually submit test tasks](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcas…. 3. You can [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/brow… and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/c… test results on the web page: https://compass-ci.openeuler.org/jobs **Logging in to the Test Environment** 1. Send an email to the compass-ci-robot(a)qq.com to [apply for an account](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/a…. -2. [Complete the environment configuration](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma… as instructed by the email. -3. Add the **sshd** field to the test task, submit the task, and [log in to the test environment](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu…. +2. Complete the environment configuration. +3. Add the **sshd** field to the test task, submit the task, and [log in to the test environment](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu…. ## Contributing to Compass-CI diff --git a/README.zh.md b/README.zh.md index 0e6cfb6..f897ae0 100644 --- a/README.zh.md +++ b/README.zh.md @@ -14,7 +14,7 @@ Compass-CI 监控很多开源软件 git repos，一旦检测到代码更新， **调测环境登录** -使用 SSH [登录测试环境进行调测](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +使用 SSH [登录测试环境进行调测](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… **测试结果分析** @@ -41,15 +41,15 @@ Compass-CI 监控很多开源软件 git repos，一旦检测到代码更新， **手动提交测试任务** -1. [安装 Compass-CI 客户端](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E6%9… +1. [安装 Compass-CI 客户端](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/insta… 2. [编写测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testca…, [手动提交测试任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… 3. 在网页中[查看](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… web: https://compass-ci.openeuler.org/jobs **登录测试环境** 1. 向 compass-ci-robot(a)qq.com 发送邮件[申请账号](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual… -2. 根据邮件反馈内容[完成环境配置](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/… -3. 在测试任务中添加 sshd 字段，提交相应的任务，[登录测试环境](https://gitee.com/wu_fengguang/compass-ci/blob/master/d… +2. 根据邮件反馈内容完成环境配置。 +3. 在测试任务中添加 sshd 字段，提交相应的任务，[登录测试环境](https://gitee.com/wu_fengguang/compass-ci/blob/master/d… ## Contributing to Compass-CI -- 2.23.0

1 0

[PATCH v3 compass-ci] container/ssh-r/start: rename variable name from vm_name to docker_name
by Wei Jihui 31 Mar '21

31 Mar '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- container/ssh-r/start | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/ssh-r/start b/container/ssh-r/start index 68a63f2..47a29eb 100755 --- a/container/ssh-r/start +++ b/container/ssh-r/start @@ -4,14 +4,14 @@ . $CCI_SRC/container/defconfig.sh -vm_name=ssh_r +docker_name=ssh_r -docker_rm ${vm_name} +docker_rm ${docker_name} cmd=( docker run --restart=always - --name ${vm_name} + --name ${docker_name} -e GATEWAY_PORTS=true -e TCP_FORWARDING=true -d -- 2.23.0

1 0

[PATCH lkp-tests] test/ansible_test: modify ansible code to output the module name
by Wang Chenglong 31 Mar '21

31 Mar '21

[Why] modify ansible code to output the name of the module for tasks execution. [How] print("{'action':'%s'}" % task_result.__dict__['_task_fields']['action']) {'action':'package'} Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/ansible_test | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/tests/ansible_test b/tests/ansible_test index 56926bbed..8c3e68c59 100755 --- a/tests/ansible_test +++ b/tests/ansible_test @@ -58,7 +58,17 @@ ansible_playbook_run() EOF [ $? -eq 0 ] && { echo "Playbook has been generated!" - } + } + [ $(command -v ansible-playbook) ] || { + echo "ansible-playbook: command not found" + exit 1 + } + + sed -i "536a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + ansible-playbook "$ansible_path"/site.yml + + sed -i "550a\ print$\"{\'action\':\'%s\'}\" \% task_result.__dict__[\'_task_fields\'][\'action\']$" \ + /usr/local/lib/python3.7/site-packages/ansible/plugins/strategy/__init__.py ansible-playbook "$ansible_path"/site.yml if [ $? -eq 0 ]; then -- 2.23.0

1 1

[PATCH compass-ci] replace alpine mirrors
by Liu Yinsi 31 Mar '21

31 Mar '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/result-webdav/Dockerfile | 1 + container/srv-http/Dockerfile | 2 ++ container/upload-libvirt-xml/Dockerfile | 1 + 3 files changed, 4 insertions(+) diff --git a/container/result-webdav/Dockerfile b/container/result-webdav/Dockerfile index db48bcb..2edd2a7 100644 --- a/container/result-webdav/Dockerfile +++ b/container/result-webdav/Dockerfile @@ -5,6 +5,7 @@ FROM openresty/openresty:alpine +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk add bash RUN mkdir -p /var/log/nginx/ RUN adduser lkp -u 1090 -g 1090 -D diff --git a/container/srv-http/Dockerfile b/container/srv-http/Dockerfile index b7d0677..4c010b9 100644 --- a/container/srv-http/Dockerfile +++ b/container/srv-http/Dockerfile @@ -18,6 +18,8 @@ FROM nginx:alpine ADD root / +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories + RUN apk add --no-cache php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo RUN sed -i '/\[global\]/a daemonize = no' /etc/php7/php-fpm.conf diff --git a/container/upload-libvirt-xml/Dockerfile b/container/upload-libvirt-xml/Dockerfile index db48bcb..2edd2a7 100644 --- a/container/upload-libvirt-xml/Dockerfile +++ b/container/upload-libvirt-xml/Dockerfile @@ -5,6 +5,7 @@ FROM openresty/openresty:alpine +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk add bash RUN mkdir -p /var/log/nginx/ RUN adduser lkp -u 1090 -g 1090 -D -- 2.23.0

1 0

[PATCH compass-ci] service/scheduler: add event time
by Wu Zhende 31 Mar '21

31 Mar '21

example: "submit_time": "2021-03-31T16:01:07+0800" "boot_time": "2021-03-31T16:01:33+0800" "close_time": "2021-03-31T16:02:47+0800" Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 5 +++++ src/scheduler/close_job.cr | 5 ++++- src/scheduler/find_job_boot.cr | 1 + 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 5767f61..2d15aae 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -170,6 +170,11 @@ class Job set_sshr_info() set_queue() set_subqueue() + set_time("submit_time") + end + + def set_time(key) + self[key] = Time.local.to_s("%Y-%m-%dT%H:%M:%S+0800") end private def set_docker_os diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index 746f70a..a112a38 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -15,6 +15,9 @@ class Sched job["job_state"] = job_state if job_state job["job_state"] = "complete" if job["job_state"] == "boot" + job.set_time("close_time") + @env.set "close_time", job["close_time"] + response = @es.set_job_content(job) if response["_id"] == nil # es update fail, raise exception @@ -34,7 +37,7 @@ class Sched mq_msg = { "job_id" => @env.get?("job_id").to_s, "job_state" => "close", - "time" => get_time + "time" => @env.get?("close_time").to_s } spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 2596f83..6e8d5d1 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -92,6 +92,7 @@ class Sched if job job.update({"testbox" => testbox}) job.set_result_root + job.set_time("boot_time") @log.info(%({"job_id": "#{job_id}", "result_root": "/srv#{job.result_root}", "job_state" : "set result root"})) set_id2job(job) -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] lib/git_mirror.rb: fix es update fail
by Li Yuanchao 31 Mar '21

31 Mar '21

As the es update to 7.11, some compatibility error occurs. [error msg] git-mirror: warning: 299 Elasticsearch-7.11.1-ff17057114c2199c9c1bbecc727003a907c0db7a "[types removal] Specifying types in doc ument update requests is deprecated, use the endpoint /{index}/_update/{id} instead." Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 7f36be3..1f4e5b5 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -308,11 +308,8 @@ class MirrorMain def es_repo_update(git_repo) repo_info = { 'git_repo' => git_repo, 'url' => @git_info[git_repo]['url'] } repo_info = repo_info.merge(@fork_stat[git_repo]) - body = { - "doc": repo_info, - "doc_as_upsert": true - } - @es_client.update(index: 'repo', type: '_doc', id: git_repo, body: body) + body = repo_info + @es_client.index(index: 'repo', type: '_doc', id: git_repo, body: body) end def update_stat_fetch(git_repo) -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/git_mirror.rb: fix no positive? method error
by Li Yuanchao 31 Mar '21

31 Mar '21

The result of query es changed, the result['hits']['total'] changes from a number to a hash, so the positive? method can not be used. Now use the count method which contained by es_client. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index c967c8b..7f36be3 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -414,9 +414,9 @@ class MirrorMain new_refs_count: {} } query = { query: { match: { _id: git_repo } } } - result = @es_client.search(index: 'repo', body: query)['hits'] - return fork_stat unless result['total'].positive? + return fork_stat unless @es_client.count(index: 'repo', body: query)['count'].positive? + result = @es_client.search(index: 'repo', body: query)['hits'] fork_stat.each_key do |key| fork_stat[key] = result['hits'][0]['_source'][key.to_s] || fork_stat[key] end -- 2.23.0

1 0

[PATCH lkp-tests] add self-test.yaml
by Liu Yinsi 31 Mar '21

31 Mar '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- jobs/self-test.yaml | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 jobs/self-test.yaml diff --git a/jobs/self-test.yaml b/jobs/self-test.yaml new file mode 100644 index 000000000..ad2dcd2a2 --- /dev/null +++ b/jobs/self-test.yaml @@ -0,0 +1,6 @@ +suite: self-test +category: functional + +deploy-cci: +multi-qemu: +multi-docker: -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow: add 8-testbox/multi-docker
by Liu Yinsi 31 Mar '21

31 Mar '21

For user locally deploy compass-ci, execute multi-docker in the background, it is convenient for users to submit jobs. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/8-testbox/multi-docker | 5 +++++ sparrow/install-tiny | 1 + 2 files changed, 6 insertions(+) create mode 100755 sparrow/8-testbox/multi-docker diff --git a/sparrow/8-testbox/multi-docker b/sparrow/8-testbox/multi-docker new file mode 100755 index 0000000..25cb82d --- /dev/null +++ b/sparrow/8-testbox/multi-docker @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +nohup $CCI_SRC/providers/multi-docker -n dc-8g -c 10 -q dc-8g.$(arch) & diff --git a/sparrow/install-tiny b/sparrow/install-tiny index e48bd25..40358a4 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -23,3 +23,4 @@ cd $CCI_SRC/sparrow || exit 5-build/os-ready 5-build/register-account 7-systemd/systemd-setup +8-testbox/multi-docker -- 2.23.0

1 0