December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci] 本地安装compass-ci客户端.md: text format adapt to markdown
by Xu Xijian 01 Dec '20

01 Dec '20

In markdown documents, text for bash should be contained in block as below: ```bash content ``` Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- ...50\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 ++ 1 file changed, 2 insertions(+) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 43a6bf5..9b89d27 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -8,6 +8,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 ## Getting started +```bash 1. setup default config run the following command to add the below setup to default config file mkdir -p ~/.config/compass-ci/defaults/ @@ -23,6 +24,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 cd lkp-tests make install source ~/.bashrc && source ~/.bash_profile +``` 3. submit job Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

1 0

[PATCH v2 compass-ci] 本地安装compass-ci客户端.md: text format adapt to markdown
by Xu Xijian 01 Dec '20

01 Dec '20

In markdown documents, text for bash should be contained in block as below: ```bash content ``` Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- ...50\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 ++ 1 file changed, 2 insertions(+) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 43a6bf5..46fde13 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -8,6 +8,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 ## Getting started +```bash 1. setup default config run the following command to add the below setup to default config file mkdir -p ~/.config/compass-ci/defaults/ @@ -26,3 +27,4 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 3. submit job Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +``` -- 2.23.0

1 1

[PATCH compass-ci] 本地安装compass-ci客: text format adapt to markdown
by Xu Xijian 01 Dec '20

01 Dec '20

In markdown documents, text for bash should be contained in block as below: ```bash content ``` Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- ...50\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 ++ 1 file changed, 2 insertions(+) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 43a6bf5..46fde13 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -8,6 +8,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 ## Getting started +```bash 1. setup default config run the following command to add the below setup to default config file mkdir -p ~/.config/compass-ci/defaults/ @@ -26,3 +27,4 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 3. submit job Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +``` -- 2.23.0

3 5

[PATCH lkp-tests] lib/job.rb: fix submit matrix args by cmd args
by Wei Jihui 01 Dec '20

01 Dec '20

[why] submit "os os_arch os_version"="1 2 3" this submit will split this arg to: os, os_arch, os_version=1, 2, 3 [how] change space to '|', like: submit "os|os_arch|os_version"="1|2|3" ... Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f9e244a9..2869dcb3 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -288,13 +288,13 @@ class Job def multi_args(hash) jobs_array = [] hash.each { |key, value| - next unless key =~ /^\w.*\s+.*\w$/ + next unless key =~ /^\w.*\|.*\w$/ - next unless value.class == Array + key_array = key.split('|') + [value].flatten.each do |v| + next unless v =~ /^\w.*\|.*\w$/ - key_array = key.split - value.each do |v| - v_array = v.split + v_array = v.split('|') next unless key_array.size == v_array.size hash_job = hash.clone -- 2.23.0

1 0

[PATCH v2 compass-ci 4/4] container: add locate_files in assistant container
by Cao Xueliang 01 Dec '20

01 Dec '20

An interface is provided to support obtaining the real path of the /srv/initrd/deps and pkg program. Example: Input: curl -H 'Content-Type: Application/json' -XPOST '172.17.0.1:8101/locate_files' -d '{"deps_files": ["/srv/initrd/deps/nfs/openeuler/aarch64/20.03/perf/perf.cgz"], "pkg_files": ["/srv/initrd/pkg/nfs/openeuler/aarch64/20.03/plzip/latest.cgz", "/srv/initrd/pkg/nfs/openeuler/aarch64/20.03/stream/latest.cgz"]}' Output: {"deps_files":["/srv/initrd/deps/nfs/openeuler/aarch64/20.03/perf/perf_20201104.cgz"], "pkg_files":["/srv/initrd/pkg/nfs/openeuler/aarch64/20.03/plzip/1-4.cgz", "/srv/initrd/pkg/nfs/openeuler/aarch64/20.03/stream/1-1.cgz"]} Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assistant/routes.rb | 31 +++++++++++++++++++++++ container/assistant/views/locate_files.rb | 19 ++++++++++++++ 2 files changed, 50 insertions(+) create mode 100755 container/assistant/routes.rb create mode 100644 container/assistant/views/locate_files.rb diff --git a/container/assistant/routes.rb b/container/assistant/routes.rb new file mode 100755 index 0000000..54d3670 --- /dev/null +++ b/container/assistant/routes.rb @@ -0,0 +1,31 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'open3' + +require_relative './views/locate_files' + +set :bind, '0.0.0.0' +set :port, 8101 + +post '/locate_files' do + request.body.rewind + + begin + data = JSON.parse request.body.read + rescue JSON::ParserError + return [400, 'parse json params error'] + end + + begin + result = locate_files(data) + rescue StandardError => e + return [400, e.backtrace.inspect] + end + + [200, result.to_json] +end diff --git a/container/assistant/views/locate_files.rb b/container/assistant/views/locate_files.rb new file mode 100644 index 0000000..9699128 --- /dev/null +++ b/container/assistant/views/locate_files.rb @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def locate_files(data) + result = {} + + data.each do |key, value| + temp = [] + value.each do |val| + val = val.strip + temp << File.realpath(val) if File.exist?(val) + end + + result.merge!({ "#{key}": temp }) + end + + return result +end -- 2.23.0

1 0

[PATCH v2 compass-ci 3/4] container: start script for assistant container
by Cao Xueliang 01 Dec '20

01 Dec '20

This is a assistant container, it provides some APIs. For now, it provides the locate_files function to give the real path for programs in /srv/initrd/deps and pkg. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assistant/start | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100755 container/assistant/start diff --git a/container/assistant/start b/container/assistant/start new file mode 100755 index 0000000..5f7822e --- /dev/null +++ b/container/assistant/start @@ -0,0 +1,21 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +docker_rm assistant + +cmd=( + docker run + --restart=always + --name assistant + -u nobody + -d + -p 8101:8101 + -v /srv/initrd:/srv/initrd + -v /etc/localtime:/etc/localtime:ro + debian:assistant +) + +"${cmd[@]}" -- 2.23.0

1 0

[PATCH v2 compass-ci 2/4] container: build script for assistant container
by Cao Xueliang 01 Dec '20

01 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assistant/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/assistant/build diff --git a/container/assistant/build b/container/assistant/build new file mode 100755 index 0000000..7778c9f --- /dev/null +++ b/container/assistant/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t debian:assistant . -- 2.23.0

1 0

[PATCH v2 compass-ci 1/4] container: assistant container Dockerfile
by Cao Xueliang 01 Dec '20

01 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assistant/Dockerfile | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 container/assistant/Dockerfile diff --git a/container/assistant/Dockerfile b/container/assistant/Dockerfile new file mode 100644 index 0000000..f1ad7a2 --- /dev/null +++ b/container/assistant/Dockerfile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER caoxl78320(a)163.com + +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-json ruby-sinatra curl + +COPY routes.rb /usr/local/bin/ +COPY views /usr/local/bin/views/ + +CMD ["/usr/local/bin/routes.rb"] -- 2.23.0

1 0

[PATCH compass-ci 3/3] scheduler/scheduler.cr: add start_time in before_all
by Wu Zhende 01 Dec '20

01 Dec '20

Record the interface access time for calculating the interface invoking time Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/scheduler/scheduler.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 98b8e2e..9f5e670 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -31,8 +31,10 @@ module Scheduler VERSION = "0.2.0" add_context_storage_type(Sched) + add_context_storage_type(Time::Span) before_all do |env| + env.set "start_time", Time.monotonic env.set "sched", Sched.new(env) env.response.headers["Connection"] = "close" end -- 2.23.0

1 0

[PATCH compass-ci 2/3] lib/sched.cr: initialize JSONLogger
by Wu Zhende 01 Dec '20

01 Dec '20

initialize JSONLogger as @log, so can use @log in each interface in Shced. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 26ddd92..a9f893f 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -5,6 +5,7 @@ require "kemal" require "yaml" require "./job" +require "./json_logger" require "./block_helper" require "./taskqueue_api" require "./remote_git_client" @@ -32,6 +33,8 @@ class Sched @block_helper = BlockHelper.new @rgc = RemoteGitClient.new @env = env + @log = JSONLogger.new + @log.set_env(env) end def normalize_mac(mac : String) -- 2.23.0

1 0