- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/4] container/assistant: refactor assistant container
by Cao Xueliang 16 Apr '21

16 Apr '21

1. Refactor assistant container. 2. Add get_mail_list function. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assistant/Dockerfile | 17 ++++---- container/assistant/build | 4 +- container/assistant/routes.rb | 20 ++++++--- container/assistant/start | 51 +++++++++++++++------- container/assistant/views/get_mail_list.rb | 9 ++++ 5 files changed, 69 insertions(+), 32 deletions(-) create mode 100644 container/assistant/views/get_mail_list.rb diff --git a/container/assistant/Dockerfile b/container/assistant/Dockerfile index 04d2da0..3299858 100644 --- a/container/assistant/Dockerfile +++ b/container/assistant/Dockerfile @@ -1,16 +1,15 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM debian +FROM alpine:3.11 MAINTAINER Cao Xueliang <caoxl78320(a)163.com> -ENV DEBIAN_FRONTEND noninteractive +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ + apk update && \ + apk add --no-cache 'ruby-dev' 'ruby-etc' \ + 'g++' 'gcc' 'pcre' 'libevent' 'make' 'git' 'bash' 'grep' 'coreutils' 'curl' 'util-linux' -RUN apt-get update && \ - apt-get install -y ruby-json ruby-sinatra curl - -COPY routes.rb /usr/local/bin/ -COPY views /usr/local/bin/views/ - -CMD ["/usr/local/bin/routes.rb"] +RUN umask 002 && \ + gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ + gem install rest-client activesupport json yaml elasticsearch sinatra puma terminal-table diff --git a/container/assistant/build b/container/assistant/build index 89a8a97..6f739d6 100755 --- a/container/assistant/build +++ b/container/assistant/build @@ -4,6 +4,6 @@ . ../defconfig.sh -docker_skip_rebuild "debian:assistant" +docker_skip_rebuild "assistant" -docker build -t debian:assistant . +docker build -t assistant . diff --git a/container/assistant/routes.rb b/container/assistant/routes.rb index 54d3670..232ec8c 100755 --- a/container/assistant/routes.rb +++ b/container/assistant/routes.rb @@ -8,24 +8,32 @@ require 'json' require 'open3' require_relative './views/locate_files' +require_relative './views/get_mail_list' -set :bind, '0.0.0.0' -set :port, 8101 +configure do + set :bind, '0.0.0.0' + set :port, ENV['ASSISTANT_PORT'] +end post '/locate_files' do request.body.rewind begin data = JSON.parse request.body.read - rescue JSON::ParserError - return [400, 'parse json params error'] + result = locate_files(data) + rescue StandardError => e + return [400, e.backtrace.inspect] end + [200, result.to_json] +end + +get '/get_mail_list/:type' do begin - result = locate_files(data) + result = get_mail_list(params[:type]) rescue StandardError => e return [400, e.backtrace.inspect] end - [200, result.to_json] + return [200, result.to_json] end diff --git a/container/assistant/start b/container/assistant/start index 5f7822e..87c1801 100755 --- a/container/assistant/start +++ b/container/assistant/start @@ -1,21 +1,42 @@ -#!/bin/bash +#!/usr/bin/env ruby # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true -. $CCI_SRC/container/defconfig.sh +require 'set' +require_relative '../defconfig.rb' -docker_rm assistant +names = Set.new %w[ + ASSISTANT_HOST + ASSISTANT_PORT +] -cmd=( - docker run - --restart=always - --name assistant - -u nobody - -d - -p 8101:8101 - -v /srv/initrd:/srv/initrd - -v /etc/localtime:/etc/localtime:ro - debian:assistant -) +defaults = relevant_defaults(names) +env = docker_env(defaults) -"${cmd[@]}" +DEFAULT_LKP = '/c/lkp-tests' +DEFAULT_CCI = '/c/compass-ci' +DEFAULT_CONFIG_DIR = '/etc/compass-ci' +ASSISTANT_PORT = defaults['ASSISTANT_PORT'] +docker_rm 'assistant' + +cmd = %w[ + docker run + --name assistant + --restart=always + -d +] + env + %W[ + -e LKP_SRC=#{DEFAULT_LKP} + -e CCI_SRC=#{DEFAULT_CCI} + -p #{ASSISTANT_PORT}:#{ASSISTANT_PORT} + -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} + -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} + -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro + -v /etc/localtime:/etc/localtime:ro + -w #{DEFAULT_CCI}/container/assistant/ + assistant +] + +cmd += ['sh', '-c', 'umask 002 && ruby ./routes.rb'] + +system(*cmd) diff --git a/container/assistant/views/get_mail_list.rb b/container/assistant/views/get_mail_list.rb new file mode 100644 index 0000000..66a1872 --- /dev/null +++ b/container/assistant/views/get_mail_list.rb @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative "#{ENV['CCI_SRC']}/lib/parse_mail_list" + +def get_mail_list(type) + return parse_mail_list(type) +end -- 2.23.0

1 0

[PATCH compass-ci 1/4] delimiter: consume job by the bad job queue
by Cao Xueliang 16 Apr '21

16 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 - 1 file changed, 1 deletion(-) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index dbc42ef..a0d4196 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -137,7 +137,6 @@ module Utils job['my_email'] = ENV['my_email'] job['my_token'] = ENV['secrets_my_token'] job['bad_job_id'] = job_id - job['queue'] = "#{job['testbox'].split('.')[0]}-bisect" job.delete('id') -- 2.23.0

1 0

[PATCH compass-ci] container/crystal-shards: fix executing command: shards version got exit 1
by Cao Xueliang 16 Apr '21

16 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/crystal-shards/shard.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/container/crystal-shards/shard.yml b/container/crystal-shards/shard.yml index 3466bc5..b9662af 100644 --- a/container/crystal-shards/shard.yml +++ b/container/crystal-shards/shard.yml @@ -35,3 +35,4 @@ dependencies: branch: master amqp-client: github: cloudamqp/amqp-client.cr + version: 1.0.0 -- 2.23.0

1 0

[PATCH compass-ci 2/2] lib/sched.cr: add tbox_group to testbox
by Wu Zhende 16 Apr '21

16 Apr '21

It can be used to display statistics on a certain type of machine. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 1359f95..7a9e4d8 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -174,10 +174,12 @@ class Sched job["job_state"] = "boot" state = "booting" job_id = job["id"] + tbox_group = job["tbox_group"] else deadline = nil job_id = "" state = "requesting" + tbox_group = JobHelper.match_tbox_group(testbox.to_s) end type = get_type(testbox) @@ -189,7 +191,8 @@ class Sched "deadline" => deadline, "queues" => queues, "type" => type, - "name" => testbox + "name" => testbox, + "tbox_group" => tbox_group } @redis.update_wtmp(testbox.to_s, hash) -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/job.cr: fix tbox_group generation rule
by Wu Zhende 16 Apr '21

16 Apr '21

[Why] The tbox generated by this method is different from the tbox generated by the lkp submit method. testbox: dc-8g.wuzhende-3455481 this method get: dc-8g.wuzhende lkp mothed get: dc-8g Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index b6bb730..200ba2b 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -26,11 +26,7 @@ end module JobHelper def self.match_tbox_group(testbox : String) - tbox_group = testbox - if testbox =~ /(.*)-\d+$/ - tbox_group = $1 - end - return tbox_group + testbox.split(/\.|--/)[0] end def self.service_path(path) -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: set pre-mount service to be wanted by kernel param local
by Xu Xijian 15 Apr '21

15 Apr '21

The local mount is dealt in dracut-pre-mount stage in dracut, however sometime this stage is skipped. So add "ConditionKernelCommandLine=|local" to make sure kernel param local can trigger dracut-pre-mount service. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index a57d681..f149556 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -20,6 +20,9 @@ sed -i "/install() {/a\ inst /usr/bin/awk" /usr/lib/dracut/modules.d/40networ tools_00bash="/sbin/e2fsck /sbin/mke2fs /usr/bin/basename /sbin/lvm /sbin/reboot" sed -i "/install() {/a\ inst $tools_00bash" /usr/lib/dracut/modules.d/00bash/module-setup.sh -pre_mount_file="/usr/lib/dracut/modules.d/98dracut-systemd/dracut-pre-mount.sh" +dracut_systemd_dir="/usr/lib/dracut/modules.d/98dracut-systemd" +pre_mount_service="${dracut_systemd_dir}/dracut-pre-mount.service" +sed -i "/Description/aConditionKernelCommandLine=|local" "$pre_mount_service" +pre_mount_file="${dracut_systemd_dir}/dracut-pre-mount.sh" [ "$(sed -n '$p' $pre_mount_file)" = "exit 0" ] && sed -i '$d' "$pre_mount_file" cat set-local-sysroot.sh >> "$pre_mount_file" -- 2.23.0

1 0

help star
by Wu Fengguang 15 Apr '21

15 Apr '21

Wei Jihui is our help star, thanks Wei Jihui! baijing@crystal ~/todo/bin% ./help-star 5.9 Wei Jihui 4.8 Liu Shaofei 3.1 Xu Xijian 2.8 Yu Chuan 2.7 Zhang Dewan 2.3 Wang Yong 2.0 Wu Fengguang 1.8 Cao Xueliang 1.8 Luan Shengde 1.7 Liu Yinsi 1.3 Wu Zhende 0.0 Bai Jing 0.0 Du Kaitian 0.0 Lu Weitao 0.0 Zhang Yale 0.0 Cui Yili 0.0 Wang Chenglong 0.0 Li Yuanchao 0.0 Lin Jiaxin 0.0 Li Ping

1 0

review star
by Wu Fengguang 15 Apr '21

15 Apr '21

: wqUser-Agent: Mutt/1.10.1 (2018-07-13) User-Agent: Mutt/1.10.1 (2018-07-13) Cao Xueliang is our review star, thanks Cao Xueliang! baijing@crystal ~/todo/bin% ./review-star Review email records in the last 7 days total review paragraph point people 75 4 4 15 Cao Xueliang 15 4 10 12 Yu Chuan 21 5 5 8 Wei Jihui 24 6 11 6 Liu Shaofei 11 7 16 4 Luan Shengde 3 1 3 3 Xu Xijian 13 3 3 1 Lin Jiaxin 4 3 6 1 Li Yuanchao 12 1 1 1 Bai Jing 9 0 0 0 Wu Zhende 14 0 0 0 Cui Yili 9 1 8 0 Wang Yong 3 0 0 0 Li Ping 10 2 3 0 Lu Weitao 5 0 0 0 Wu Fengguang 21 0 0 0 Liu Yinsi 11 0 0 0 Wang Chenglong

1 0

git count
by Wu Fengguang 15 Apr '21

15 Apr '21

Cao Xueliang is our code star, thanks Cao Xueliang! baijing@crystal ~/todo/bin% ./git-count 13 Cao Xueliang 7 Wu Zhende 6 Liu Shaofei 6 Lu Weitao 6 Liu Yinsi 4 Wei Jihui 3 Bai Jing 2 Xu Xijian 2 Cui Yili 2 Wang Yong 2 Li Yuanchao 2 Lin Jiaxin 1 Yu Chuan 1 Wang Chenglong 1 Luan Shengde 0 Du Kaitian 0 Wu Fengguang 0 Zhang Yale 0 Zhang Dewan 0 Li Ping

1 0

[PATCH lkp-tests] setup cluster for multi server(3 or more)
by Luan Shengde 15 Apr '21

15 Apr '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- cluster/cs-s1-a122-c2 | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 cluster/cs-s1-a122-c2 diff --git a/cluster/cs-s1-a122-c2 b/cluster/cs-s1-a122-c2 new file mode 100644 index 000000000..a69179eab --- /dev/null +++ b/cluster/cs-s1-a122-c2 @@ -0,0 +1,14 @@ +switch: Switch-P12 +ip0: 1 +nodes: + taishan200-2280-2s64p-256g--a122: + roles: [ node1 ] + macs: [ "44:a1:91:9d:d8:25" ] + + taishan200-2280-2s64p-256g--a123: + roles: [ node2 ] + macs: [ "44:a1:91:9d:f5:4a" ] + + taishan200-2280-2s64p-256g--a125: + roles: [ node3 ] + macs: [ "44:67:47:85:f4:65" ] -- 2.23.0

1 0