October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v6 compass-ci 6/6] container/mail-robot: mail-robot.md
by Luan Shengde 13 Oct '20

13 Oct '20

guidance file for mail-robot Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/mail-robot.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 doc/mail-robot.md diff --git a/doc/mail-robot.md b/doc/mail-robot.md new file mode 100644 index 0000000..e248b41 --- /dev/null +++ b/doc/mail-robot.md @@ -0,0 +1,40 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ + +# email monitor robot + +## purpose + +monitor received mails + check if the mail is for applying uuid/account + +## steps overview + +1. monitor mailbox dir + - check for new email file + +2. check email + - check email subject + - return if unmatched subject + - call answerback-email + - check email available + - check base_url for the oss_url in upstream-repos list + if not, it will raise "url not in upstream-repos list" error + - check if the email has commit for the url + - if gitee, git clone the repo and check + if non-gitee, curl url to check + - if not, raise "no commit found" error + - check pub_key + - check if there is a pub_key in the email + if not, it will raise 'no pub_key' error + - apply ssh account + - apply account uuid + - write account info to es + - send uuid to user + - mv mail file to drafts + +3. continue the monitor + - after 1 and 2, continue to monitor the mailbox + +4. service log + - use the following command to get the logs + docker logs -f --tail=100 fluentd |grep mail-robot -- 2.23.0

1 0

[PATCH v6 compass-ci 4/6] container/mail-robot: Dockerfile
by Luan Shengde 13 Oct '20

13 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/Dockerfile | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 container/mail-robot/Dockerfile diff --git a/container/mail-robot/Dockerfile b/container/mail-robot/Dockerfile new file mode 100644 index 0000000..57307bf --- /dev/null +++ b/container/mail-robot/Dockerfile @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER shdluan(a)163.com + +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y uuid-runtime curl ruby-listen ruby-json ruby-mail && \ + gem install fileutils elasticsearch activesupport && \ + ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ + echo 'Asia/Shanghai' > /etc/timezone + +COPY mail-robot.rb /usr/local/bin/ + +COPY answerback-email.rb /usr/local/bin/ + +WORKDIR /usr/local/bin/ -- 2.23.0

1 0

[PATCH v6 compass-ci 3/6] container/mail-robot: build file
by Luan Shengde 13 Oct '20

13 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/mail-robot/build diff --git a/container/mail-robot/build b/container/mail-robot/build new file mode 100755 index 0000000..c564add --- /dev/null +++ b/container/mail-robot/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t mail-robot:latest . -- 2.23.0

1 0

[PATCH v6 compass-ci 1/6] container/mail-robot: mail-robot.rb
by Luan Shengde 13 Oct '20

13 Oct '20

mail robot for application email monitor mailbox check new added email file in new check new mail's subject return if unmatched subject call answerback-email check to apply uuid/account Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/mail-robot.rb | 43 ++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100755 container/mail-robot/mail-robot.rb diff --git a/container/mail-robot/mail-robot.rb b/container/mail-robot/mail-robot.rb new file mode 100755 index 0000000..393b033 --- /dev/null +++ b/container/mail-robot/mail-robot.rb @@ -0,0 +1,43 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' +require_relative 'answerback-email.rb' + +mail_inbox = '/Maildir/.compass-ci/new/' +mail_drafts = '/Maildir/.compass-ci/cur/' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + unless added.empty? + added.each do |mail_file| + begin + check_to_send_account(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + puts e.backtrace + end + end + end + end + listener.start + sleep +end + +def check_to_send_account(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + subject = mail_content.subject + return unless subject =~ /apply account/i + + assign_uuid = AssignUuid.new(mail_content) + assign_uuid.send_account + FileUtils.mv(mail_file, mail_drafts) +end + +monitor_dir(mail_inbox, mail_drafts) -- 2.23.0

1 0

[PATCH v4 compass-ci] feat: build centos-7 initramfs image with one command
by Wang Chenglong 13 Oct '20

13 Oct '20

Use the tool to create a new centos-7 initramfs image with one command. and we also can customize the pre-installed software by configuring ./bin/install-list. --- container/osimage/centos-7/README.md | 13 +++++++ container/osimage/centos-7/bin/create-image | 38 +++++++++++++++++++ container/osimage/centos-7/bin/lib | 30 +++++++++++++++ .../osimage/centos-7/bin/packages-to-install | 2 + .../osimage/centos-7/bin/packages-to-remove | 8 ++++ container/osimage/centos-7/build | 29 ++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 container/osimage/centos-7/README.md create mode 100755 container/osimage/centos-7/bin/create-image create mode 100755 container/osimage/centos-7/bin/lib create mode 100644 container/osimage/centos-7/bin/packages-to-install create mode 100644 container/osimage/centos-7/bin/packages-to-remove create mode 100755 container/osimage/centos-7/build diff --git a/container/osimage/centos-7/README.md b/container/osimage/centos-7/README.md new file mode 100644 index 0000000..d1a0317 --- /dev/null +++ b/container/osimage/centos-7/README.md @@ -0,0 +1,13 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + cd ${CCI_SRC/rootfs/initramfs/centos/aarch64/7} + ./build + +Some configuration items: +./bin/packages-to-install + If you want to pre-install the software, you can write the package names in ./bin/packages-to-install. + +./bin/packages-to-remove + If you want remove some unnecessary files, you can write the names in ./bin/packages-to-remove + diff --git a/container/osimage/centos-7/bin/create-image b/container/osimage/centos-7/bin/create-image new file mode 100755 index 0000000..214204f --- /dev/null +++ b/container/osimage/centos-7/bin/create-image @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/packages-to-install) + yum clean all + rm -rf $(</root/bin/packages-to-remove) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/container/osimage/centos-7/bin/lib b/container/osimage/centos-7/bin/lib new file mode 100755 index 0000000..c3c470f --- /dev/null +++ b/container/osimage/centos-7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(basename $(dirname $(realpath $0)))" +name="$(echo $DIR |awk -F "-" '{print $1}')" +version="$(echo $DIR |awk -F "-" '{print $2}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/container/osimage/centos-7/bin/packages-to-install b/container/osimage/centos-7/bin/packages-to-install new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/container/osimage/centos-7/bin/packages-to-install @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/container/osimage/centos-7/bin/packages-to-remove b/container/osimage/centos-7/bin/packages-to-remove new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/container/osimage/centos-7/bin/packages-to-remove @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/container/osimage/centos-7/build b/container/osimage/centos-7/build new file mode 100755 index 0000000..5e46306 --- /dev/null +++ b/container/osimage/centos-7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos-7/bin/lib +. ${CCI_SRC}/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/create-image +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

3 12

[PATCH compass-ci] sparrow: add docker test
by Liu Yinsi 12 Oct '20

12 Oct '20

1. submit job 2. run docker test Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/6-test/docker | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/sparrow/6-test/docker b/sparrow/6-test/docker index 3d6796d..ab825d3 100755 --- a/sparrow/6-test/docker +++ b/sparrow/6-test/docker @@ -1,7 +1,13 @@ -#!/bin/bash -e +#!/usr/bin/env ruby # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true -## TODO -# 1.submit a job -# 2.run docker.sh +require_relative '../../providers/docker/docker' + +def run_job(hostname) + system "submit -s 'testbox: #{hostname}' #{ENV['CCI_SRC']}/user-client/jobs/iperf-sparrow.yaml" + main "#{hostname}" +end + +run_job "dc-1g" -- 2.23.0

2 4

[PATCH compass-ci] use local docker repository
by Xiao Shenwei 12 Oct '20

12 Oct '20

add DOCKER_REGISTRY_HOST, DOCKER_REGISTRY_PORT field into /etc/compass-ci/defaults/*.yaml, or $HOME/.config/compass-ci/defaults/*.yaml also, if failed from local registry, will pull from docker hub Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/docker.rb | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/providers/docker/docker.rb b/providers/docker/docker.rb index c75fc2c..eea0336 100755 --- a/providers/docker/docker.rb +++ b/providers/docker/docker.rb @@ -11,14 +11,17 @@ require_relative '../../container/defconfig' BASE_DIR = '/srv/dc' +names = Set.new %w[ + SCHED_HOST + SCHED_PORT + DOCKER_REGISTRY_HOST + DOCKER_REGISTRY_PORT +] +DEFAULTS = relevant_defaults(names) + def get_url(hostname) - names = Set.new %w[ - SCHED_HOST - SCHED_PORT - ] - defaults = relevant_defaults(names) - host = defaults['SCHED_HOST'] || '172.17.0.1' - port = defaults['SCHED_PORT'] || 3000 + host = DEFAULTS['SCHED_HOST'] || '172.17.0.1' + port = DEFAULTS['SCHED_PORT'] || 3000 "http://#{host}:#{port}/boot.container/hostname/#{hostname}" end @@ -59,7 +62,7 @@ def clean_dir(path) end def load_initrds(load_path, hash) - clean_dir(load_path) if Dir.exist?(load_path) + clean_dir(load_path) unless Dir.empty?(load_path) arch = RUBY_PLATFORM.split('-')[0] job_url = hash['job'] lkp_url = hash['lkp'] @@ -67,9 +70,18 @@ def load_initrds(load_path, hash) wget_cmd(load_path, lkp_url, "lkp-#{arch}.cgz") end +def pull(image_name) + host = DEFAULTS['DOCKER_REGISTRY_HOST'] || '172.17.0.1' + port = DEFAULTS['DOCKER_REGISTRY_PORT'] || '5001' + flag = system "docker pull #{host}:#{port}/#{image_name}" + unless flag + system "docker pull #{image_name}" + end +end + def run(hostname, load_path, hash) docker_image = hash['docker_image'] - system "docker pull #{docker_image}" + pull docker_image system( { 'hostname' => hostname, 'docker_image' => docker_image, 'load_path' => load_path }, ENV['CCI_SRC'] + '/providers/docker/run.sh' -- 2.23.0

2 4

[PATCH compass-ci] fix: scheduler: update job root_result error
by Tong Qunfeng 12 Oct '20

12 Oct '20

1) after job finished, we save update job field back to es. 2) if the job is consumed at "20201010", but finished at "20201011". then the result_root will be update to "/result/$suite/$tbox_group/20201011/$id". 3) but the result is really saved at "/result/$suite/$tbox_group/20201010/$id", 20201010 is the date when it is consumed. no need to set_default for an initialized job (when is is consumed). Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/job.cr | 40 +++++++++++++++++++++++++++++++++++++--- src/lib/sched.cr | 1 + 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 504bc19..be3446e 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -50,10 +50,13 @@ class Job @hash = job_content.as_h # init job with "-1", or use the original job_content["id"] - if "#{id}" == "" - @hash["id"] = JSON::Any.new("-1") + id == "-1" if "#{id}" == "" + + if initialized? + return if @hash["id"] == "#{id}" end + @hash["id"] = JSON::Any.new("#{id}") check_required_keys() set_defaults() end @@ -165,7 +168,7 @@ class Job self["os_dir"] = "#{os}/#{os_arch}/#{os_version}" end - private def set_result_root + def set_result_root update_tbox_group_from_testbox # id must exists, need update tbox_group date = Time.local.to_s("%F") self["result_root"] = "/result/#{suite}/#{tbox_group}/#{date}/#{id}" @@ -234,6 +237,37 @@ class Job end end + private def initialized? + initialized_keys = [] of String + + REQUIRED_KEYS.each do |key| + initialized_keys << key.to_s + end + + METHOD_KEYS.each do |key| + initialized_keys << key.to_s + end + + INIT_FIELD.each do |key, _value| + initialized_keys << key.to_s + end + + initialized_keys += ["result_service", + "LKP_SERVER", + "LKP_CGI_PORT", + "SCHED_HOST", + "SCHED_PORT"] + + initialized_keys.each do |key| + if @hash.has_key?(key) == false + return false + end + end + + return false if "#{@hash["id"]}" == "" + return true + end + private def vmlinuz if os_mount == "initramfs" return "#{INITRD_HTTP_PREFIX}" + diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 23332df..2fbf05d 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -451,6 +451,7 @@ class Sched if job job.update({"testbox" => testbox}) + job.set_result_root @redis.set_job(job) end return job -- 2.23.0

1 0

[PATCH compass-ci] container/remote-git: disable encode
by Lin Jiaxin 12 Oct '20

12 Oct '20

encode change binary file's md5sums. error: ==> Validating source files with md5sums... VUE.jar ... Passed vue ... Passed vue.png ... FAILED vue.desktop ... Passed ==> ERROR: One or more files did not pass the validity check! md5sum vue.png 7253b39732630ba58186d7b79263b2a3 vue.png correct: ==> Validating source files with md5sums... VUE.jar ... Passed vue ... Passed vue.png ... Passed vue.desktop ... Passed md5sum vue.png 9369da34c69baefc77a7baeaceb4e476 vue.png Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- container/remote-git/remote-git.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/remote-git/remote-git.rb b/container/remote-git/remote-git.rb index 085f4db..44a8e18 100755 --- a/container/remote-git/remote-git.rb +++ b/container/remote-git/remote-git.rb @@ -41,7 +41,7 @@ post '/git_command' do # execute git command Dir.chdir(repo_path) _stdin, stdout, _stderr, wait_thr = Open3.popen3(*data['git_command']) - out = stdout.read.force_encoding('ISO-8859-1').encode('UTF-8') + out = stdout.read exit_code = wait_thr.value.to_i [200, headers.update({ 'errcode' => '0', 'exit_code' => exit_code.to_s }), out] -- 2.23.0

1 0

[PATCH v5 compass-ci 4/6] container/mail-robot: Dockerfile
by Luan Shengde 12 Oct '20

12 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/mail-robot/Dockerfile diff --git a/container/mail-robot/Dockerfile b/container/mail-robot/Dockerfile new file mode 100644 index 0000000..0eddbc4 --- /dev/null +++ b/container/mail-robot/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y uuid-runtime curl ruby-listen ruby-json ruby-mail && \ + gem install fileutils elasticsearch activesupport && \ + ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ + echo 'Asia/Shanghai' > /etc/timezone + +COPY mail-robot.rb /usr/local/bin/ + +COPY answerback-email.rb /usr/local/bin/ + +WORKDIR /usr/local/bin/ -- 2.23.0

2 1

[PATCH v5 compass-ci 3/6] container/mail-robot: build file
by Luan Shengde 12 Oct '20

12 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/build | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100755 container/mail-robot/build diff --git a/container/mail-robot/build b/container/mail-robot/build new file mode 100755 index 0000000..e742191 --- /dev/null +++ b/container/mail-robot/build @@ -0,0 +1,6 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t mail-robot:latest . + -- 2.23.0

2 1

[PATCH compass-ci] src: remove hi1620 from vm-hi1620
by Bai Jing 12 Oct '20

12 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- src/delimiter/constants.rb | 2 +- src/features/jobs/right_iperf.yaml | 2 +- src/features/scheduler.feature | 2 +- src/features/step_definitions/scheduler_step.rb | 2 +- src/lib/web_backend.rb | 4 ++-- src/scheduler/README.md | 2 +- src/taskqueue/queue.cr | 2 +- src/taskqueue/redis_client.cr | 2 +- 8 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index 7a17b36..dafc2c8 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -9,4 +9,4 @@ MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" -DELIMITER_TBOX_GROUP = 'vm-hi1620-2p8g--delimiter' +DELIMITER_TBOX_GROUP = 'vm-2p8g--delimiter' diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index 843564d..f81af38 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -3,7 +3,7 @@ #! jobs/iperf.yaml suite: iperf testcase: iperf -testbox: vm-hi1620-2p8g-chief +testbox: vm-2p8g--chief category: benchmark runtime: 30 cluster: cs-localhost diff --git a/src/features/scheduler.feature b/src/features/scheduler.feature index c335fb4..7c1a804 100644 --- a/src/features/scheduler.feature +++ b/src/features/scheduler.feature @@ -6,7 +6,7 @@ Feature: Scheduler Given prepared a job "right_iperf.yaml" When call with API: post "submit_job" job from add_job.sh Then return with job id - When call with API: put "set_host_mac" "vm-hi1620-2p8g-chief => ef-01-02-03-04-05" + When call with API: put "set_host_mac" "vm-2p8g--chief => ef-01-02-03-04-05" And call with API: get "boot.ipxe/mac/ef-01-02-03-04-05" Then return with basic ipxe boot parameter and initrd and kernel diff --git a/src/features/step_definitions/scheduler_step.rb b/src/features/step_definitions/scheduler_step.rb index 882ddd8..2872bc7 100644 --- a/src/features/step_definitions/scheduler_step.rb +++ b/src/features/step_definitions/scheduler_step.rb @@ -22,7 +22,7 @@ Then('return with job id') do end # set_host_mac -# host_mac => "vm-hi1620-2p8g-chief => ef-01-02-03-04-05" +# host_mac => "vm-2p8g--chief => ef-01-02-03-04-05" Given('call with API: put {string} {string}') do |url, host_mac| host_mac_params = host_mac.split(' ') url_with_params = "#{url}?hostname=#{host_mac_params[0]}\\&mac=#{host_mac_params[2]}" diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 4544690..31aa0c3 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -41,8 +41,8 @@ def es_count(query) ES_CLIENT.count(index: 'jobs*', body: query)['count'] end -# "vm-hi1620-2p8g-212" remove "-212" -# "vm-hi1620-2p8g-zzz" remove "-zzz" +# "vm-2p8g--212" remove "--212" +# "vm-2p8g--zzz" remove "--zzz" # "vm-git-bisect" don't remove "-bisect" def filter_tbox_group(es_result) result = Set.new diff --git a/src/scheduler/README.md b/src/scheduler/README.md index 5da1bde..fdc7027 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -413,7 +413,7 @@ you can simply call the scheduler's build and run script. vim $CCI_SRC/user-client/helper/iperf.yaml runtime: 30 # unit in seconds, the time can be shortened arch: aarch64 - testbox: vm-hi1620-2p8g-$USER # echo $USER for your own user name + testbox: vm-2p8g--$USER # echo $USER for your own user name ## 2.Submit job submit-job $_CCI_SRC/user-client/helper/iperf.yaml diff --git a/src/taskqueue/queue.cr b/src/taskqueue/queue.cr index 02236f5..674606e 100644 --- a/src/taskqueue/queue.cr +++ b/src/taskqueue/queue.cr @@ -183,7 +183,7 @@ class TaskQueue private def move_uuid_task2ready(queue_name) # get uuid lists - # uuid_keys ["queues/sched/vm-hi1620-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451", ] + # uuid_keys ["queues/sched/vm-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451", ] uuid_keys = get_uuid_keys("#{queue_name}") return unless uuid_keys # nil return unless uuid_keys.size > 0 diff --git a/src/taskqueue/redis_client.cr b/src/taskqueue/redis_client.cr index 3bac924..27d46cf 100644 --- a/src/taskqueue/redis_client.cr +++ b/src/taskqueue/redis_client.cr @@ -236,7 +236,7 @@ class TaskQueue private def get_uuid_keys(queue_name) return nil unless queue_name[0..5] == "sched/" - # search = "queues/sched/vm-hi1620-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451" + # search = "queues/sched/vm-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451" # only 3 matchs keyword: * [] ? search = "#{QUEUE_NAME_BASE}/#{queue_name}/[0-9a-eA-F\-]*" lua_script = "return redis.call('keys', KEYS[1])" -- 2.23.0

1 0

[PATCH v5 compass-ci 2/6] container/mail-robot: answerback-email.rb
by Luan Shengde 12 Oct '20

12 Oct '20

read email content check email available check if base url in upstream-repos return if base url not in upstream-repos check if commit url valid return if no match commit info check pub_key return if no pub_key assign account generate uuid write account/uuid to es send uuid to user Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/answerback-email.rb | 207 +++++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100755 container/mail-robot/answerback-email.rb diff --git a/container/mail-robot/answerback-email.rb b/container/mail-robot/answerback-email.rb new file mode 100755 index 0000000..7664f37 --- /dev/null +++ b/container/mail-robot/answerback-email.rb @@ -0,0 +1,207 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require_relative '/compass-ci/lib/es_client.rb' + +def build_message(sum_infos) + # To: #{sum_infos['email']} + message = <<~EMAIL_MESSAGE + To: shdluan(a)163.com + Message-ID: #{sum_infos['message_id']} + Subject: Account Ready + + Dear #{sum_infos['email'].split('@')[0]} + + Thank you for joining us. + You can use the following info to submit jobs: + + First add the info to: ~/.config/compass-ci/defaults/{USER}.yaml + + uuid: #{sum_infos['account_uuid']} + lab: z9 + + Second: download lkp-tests and dependencies + git clone https://gitee.com_/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source {HOME}/.bashrc && source {HOME}/.bash_profile + + Third: submit job + reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md + how to write job yaml + submit jobs/netperf.yaml + + Repalce the fields {...} with your local environment variable. + + regards + compass-ci + EMAIL_MESSAGE + + return message +end + +def email_addr(mail_content) + email = mail_content.from[0] + + return email +end + +def url_addr(mail_content) + url_line = mail_content.part[0].body.decoded.split(/\n/).find { |line| line =~ %r{https?://} } + message = 'No commit url found' + + raise message unless url_line + + url = url_line.split[-1] + + return url +end + +def base_url_addr(url) + url_a = url.split('/') + url_a.pop 2 + base_url = url_a.join('/') + + return base_url +end + +def check_email_available(mail_content, email) + url = url_addr(mail_content) + + base_url = base_url_addr(url) + + upstream_dir = '/upstream-repos' + check_url_in_upstream_repos(upstream_dir, base_url) + check_url_commit(url, base_url, email) + + return url +end + +def check_url_in_upstream_repos(upstream_dir, base_url) + Dir.chdir(upstream_dir) + match_out = %x(grep -rn #{base_url}) + + message = 'The url is not in upstream-repo list' + + raise StandardError, message, base_url if match_out.empty? +end + +def check_url_commit(url, base_url, email) + hub_name = url.split('/')[2] + check_gitee_commit(url, base_url, email) if hub_name.eql? 'gitee.com' + check_non_gitee_commit(url, email) unless hub_name.eql? 'gitee.com' +end + +def check_non_gitee_commit(url, email) + url_fdback = %x(curl #{url}) + email_index = url_fdback.index email + + message = 'No email matched from the commit_id feedback.' + raise StandardError, message, url if email_index.nil? +end + +def check_gitee_commit(url, base_url, email) + repo_info = { + 'repos_dir' => '/repos_dir', + 'repo_dir' => url.split('/')[-3], + 'repo_url' => [base_url, 'git'].join('.'), + 'repo_path' => File.join('/repos_dir', url.split('/')[-3]), + 'commit_id' => url.split('/')[-1] + } + check_gitee_repo_commit(url, email, repo_info) +end + +def check_gitee_repo_commit(url, email, repo_info) + Dir.chdir repo_info['repos_dir'] + %x(/usr/bin/git clone #{repo_info['repo_url']}) + Dir.chdir repo_info['repo_path'] + email_index = %x(/usr/bin/git show #{repo_info['commit_id']}).index email + + Dir.chdir repo_info['repos_dir'] + FileUtils.rm_rf repo_info['repo_dir'] + + message = 'No email matched from commit_id feedback.' if email_index.nil? + raise StandardError, message, url if email_index.nil? +end + +def email_message_id(mail_content) + message_id = mail_content.message_id + return message_id +end + +def pub_key_value(mail_content) + pub_key = mail_content.part[1].body.decoded.split(/\r|\n/).join if mail_content.part[1].filename == 'id_rsa.pub' + + message = 'No pub_key found' + + raise message unless pub_key + + return pub_key +end + +def account_info(pub_key) + account_info_str = %x(curl -XGET "#{ENV['JUMPER_IP']}:#{ENV['JUMPER_PORT']}/assign_account" -d "pub_key: #{pub_key}") + account_info = JSON.parse account_info_str + + message = 'No available jumper account returned.' + raise StandardError, message, account_info if account_info['account'].empty? + + return account_info +end + +def setup_account_uuid + account_uuid = %x(uuidgen).chomp + + return account_uuid +end + +def sum_infos(mail_content) + email = email_addr(mail_content) + pub_key = pub_key_value(mail_content) + sum_infos = { + 'email' => email, + 'message_id' => email_message_id(mail_content), + 'url' => check_email_available(mail_content, email), + 'acct_infos' => account_info(pub_key), + 'account_uuid' => setup_account_uuid, + 'pub_key' => pub_key + } + store_account_info(sum_infos) + sum_infos +end + +def send_uuid(sum_infos) + message = build_message(sum_infos) + + send_mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp + %x(curl -XPOST "#{send_mail_server}:#{ENV['SEND_MAIL_PORT']}/send_mail_text" -d "#{message}") +end + +def send_account(mail_content) + sum_infos = sum_infos(mail_content) + + send_uuid(sum_infos) +end + +def store_account_info(sum_infos) + account_uuid_info = { + 'email' => sum_infos['email'], + 'name' => sum_infos['email'].split('@')[0], + 'uuid' => sum_infos['account_uuid'], + 'user' => sum_infos['acct_infos']['account'], + 'my_commit_url' => sum_infos['url'] + } + + es = ESClient.new(index: 'accounts') + es.put_source_by_id(sum_infos['email'], account_uuid_info) +end + +def read_mail_content(mail_file) + mail_content = Mail.read(mail_file) + + return mail_content +end -- 2.23.0

2 1

[PATCH v4 compass-ci] feat: build centos initramfs image with one command
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-$version initramfs image with one command. and we also can customize the pre-installed software by configuring ./bin/packages-to-install. --- container/osimage/centos/README.md | 15 +++++++ container/osimage/centos/bin/create-image | 38 +++++++++++++++++ container/osimage/centos/bin/lib | 41 +++++++++++++++++++ .../osimage/centos/bin/packages-to-install | 2 + .../osimage/centos/bin/packages-to-remove | 8 ++++ container/osimage/centos/run | 27 ++++++++++++ 6 files changed, 131 insertions(+) create mode 100644 container/osimage/centos/README.md create mode 100755 container/osimage/centos/bin/create-image create mode 100755 container/osimage/centos/bin/lib create mode 100644 container/osimage/centos/bin/packages-to-install create mode 100644 container/osimage/centos/bin/packages-to-remove create mode 100755 container/osimage/centos/run diff --git a/container/osimage/centos/README.md b/container/osimage/centos/README.md new file mode 100644 index 0000000..000ca05 --- /dev/null +++ b/container/osimage/centos/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new centos-${version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/centos/aarch64/${version}/ + ./build + +Some configuration items: +./bin/packages-to-install + If you want to pre-install the software, you can write the package names in ./bin/packages-to-install. + +./bin/packages-to-remove + If you want remove some unnecessary files, you can write the names in ./bin/packages-to-remove. + +$HOME/.config/compass-ci/rootfs.passwd + Set the password for the image into this file. diff --git a/container/osimage/centos/bin/create-image b/container/osimage/centos/bin/create-image new file mode 100755 index 0000000..214204f --- /dev/null +++ b/container/osimage/centos/bin/create-image @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/packages-to-install) + yum clean all + rm -rf $(</root/bin/packages-to-remove) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/container/osimage/centos/bin/lib b/container/osimage/centos/bin/lib new file mode 100755 index 0000000..b4a59e2 --- /dev/null +++ b/container/osimage/centos/bin/lib @@ -0,0 +1,41 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(pwd)" +name="$(echo $DIR |awk -F "/" '{print $(NF - 2)}')" +version="$(echo $DIR |awk -F "/" '{print $NF}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker_image() +{ + docker pull $image + if [ $? = 0 ]; then + echo "finish downloading image" + else + echo "[INFO] Run command in false path ." + exit + fi +} + +# cp image package to host +cp_package() +{ + docker cp -a init_docker:/${image_name} $HOME/ + echo "result: $(ls $HOME/${image_name})" +} diff --git a/container/osimage/centos/bin/packages-to-install b/container/osimage/centos/bin/packages-to-install new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/container/osimage/centos/bin/packages-to-install @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/container/osimage/centos/bin/packages-to-remove b/container/osimage/centos/bin/packages-to-remove new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/container/osimage/centos/bin/packages-to-remove @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/container/osimage/centos/run b/container/osimage/centos/run new file mode 100755 index 0000000..47817b9 --- /dev/null +++ b/container/osimage/centos/run @@ -0,0 +1,27 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos/bin/lib +. ${CCI_SRC}/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/create-image +) + +pull_docker_image +${cmd[@]} +cp_package +docker_rm init_docker &> /dev/null +echo "build finished" + -- 2.23.0

2 2

[PATCH v5 compass-ci 6/6] container/mail-robot: mail-robot.md
by Luan Shengde 12 Oct '20

12 Oct '20

guidance file for mail-robot Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/mail-robot.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 doc/mail-robot.md diff --git a/doc/mail-robot.md b/doc/mail-robot.md new file mode 100644 index 0000000..e248b41 --- /dev/null +++ b/doc/mail-robot.md @@ -0,0 +1,40 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ + +# email monitor robot + +## purpose + +monitor received mails + check if the mail is for applying uuid/account + +## steps overview + +1. monitor mailbox dir + - check for new email file + +2. check email + - check email subject + - return if unmatched subject + - call answerback-email + - check email available + - check base_url for the oss_url in upstream-repos list + if not, it will raise "url not in upstream-repos list" error + - check if the email has commit for the url + - if gitee, git clone the repo and check + if non-gitee, curl url to check + - if not, raise "no commit found" error + - check pub_key + - check if there is a pub_key in the email + if not, it will raise 'no pub_key' error + - apply ssh account + - apply account uuid + - write account info to es + - send uuid to user + - mv mail file to drafts + +3. continue the monitor + - after 1 and 2, continue to monitor the mailbox + +4. service log + - use the following command to get the logs + docker logs -f --tail=100 fluentd |grep mail-robot -- 2.23.0

1 0

[PATCH v5 compass-ci 1/6] container/mail-robot: mail-robot.rb
by Luan Shengde 12 Oct '20

12 Oct '20

mail robot for application email monitor mailbox check new added email file in new check new mail's subject return if unmatched subject call answerback-email apply account apply uuid write account/uuid etc. to es send uuid to user Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/mail-robot.rb | 41 ++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100755 container/mail-robot/mail-robot.rb diff --git a/container/mail-robot/mail-robot.rb b/container/mail-robot/mail-robot.rb new file mode 100755 index 0000000..2de0ac4 --- /dev/null +++ b/container/mail-robot/mail-robot.rb @@ -0,0 +1,41 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' +require_relative 'answerback-email.rb' + +mail_inbox = '/Maildir/.compass-ci/new/' +mail_drafts = '/Maildir/.compass-ci/cur/' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + unless added.empty? + added.each do |mail_file| + begin + check_to_send_account(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + end + end + end + end + listener.start + sleep +end + +def check_to_send_account(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + subject = mail_content.subject + return unless subject =~ /apply account/i + + send_account(mail_content) + FileUtils.mv(mail_file, mail_drafts) +end + +monitor_dir(mail_inbox, mail_drafts) -- 2.23.0

1 0

[PATCH v5 compass-ci 5/6] container/mail-robot: start file
by Luan Shengde 12 Oct '20

12 Oct '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..a2e183d --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,29 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative '../defconfig.rb' +docker_rm 'mail-robot' + +JUMPER_IP = '192.168.132.2' +JUMPER_PORT = '29999' +SEND_MAIL_PORT = '11311' + +cmd = %W[ + docker run + --restart=always + --name=mail-robot + -d + -e JUMPER_IP=#{JUMPER_IP} + -e JUMPER_PORT=#{JUMPER_PORT} + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -v /c/upstream-repos:/upstream-repos:ro + -v /srv/cci/Maildir:/Maildir:rw + -v /c/compass-ci:/compass-ci:ro + mail-robot +] + +cmd += ['bash', '-c', 'umask 002 && ruby ./mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci] feat: a link for one command to build centos initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-$version initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..2d22759 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos/run \ No newline at end of file -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/srv-http allow access file with level 4 path
by Lu Weitao 12 Oct '20

12 Oct '20

problem: can not access file with level 4 path like: http://localhost:11300/result/unixbench/crystal.82998/boot-time.json cause analysis: we had forbidden access level 4 catalog and file solution: keep forbidden access level 4 catalog, allow access file with level 4 path. Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 403b0dc..9c15d99 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -22,7 +22,7 @@ server { } } - location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/?)$ { + location ~* ^/result/(([^/]+)?/?|[^/]+/[^/]+/[^/]+/)$ { deny all; } } -- 2.23.0

1 0

[PATCH compass-ci 1/2] container/srv-http fix Chinese garbled
by Lu Weitao 12 Oct '20

12 Oct '20

problem: Chinese file name and content display garbled characters Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/srv-http/root/etc/nginx/conf.d/default.conf | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/srv-http/root/etc/nginx/conf.d/default.conf b/container/srv-http/root/etc/nginx/conf.d/default.conf index 2bf55b6..403b0dc 100644 --- a/container/srv-http/root/etc/nginx/conf.d/default.conf +++ b/container/srv-http/root/etc/nginx/conf.d/default.conf @@ -6,6 +6,8 @@ server { root /usr/share/nginx/html; index index.html; + charset utf-8,gbk; + location /favicon.ico { log_not_found off; } -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: update_id need set_initrds_uri
by Tong Qunfeng 12 Oct '20

12 Oct '20

Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/job.cr | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index bf959c2..504bc19 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -382,9 +382,13 @@ class Job def update_id(id) @hash["id"] = JSON::Any.new(id) - # "result_root" is based on "id" + # "result_root" => "/result/#{suite}/#{tbox_group}/#{date}/#{id}" + # set_initrds_uri -> get_initrds -> common_initrds => ".../#{id}/job.cgz" + # + # "result_root, common_initrds" is associate with "id" # so when update id, we need redo set_ set_result_root() + set_initrds_uri() end def get_uuid_tag -- 2.23.0

1 0

[PATCH v4 compass-ci] feat: build centos-7 initramfs image with one command
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with one command. and we also can customize the pre-installed software by configuring ./bin/install-list. --- container/osimage/centos-7/README.md | 13 +++++++ container/osimage/centos-7/bin/create-image | 38 +++++++++++++++++++++ container/osimage/centos-7/bin/delete-list | 8 +++++ container/osimage/centos-7/bin/install-list | 2 ++ container/osimage/centos-7/bin/lib | 30 ++++++++++++++++ container/osimage/centos-7/build | 29 ++++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 container/osimage/centos-7/README.md create mode 100755 container/osimage/centos-7/bin/create-image create mode 100644 container/osimage/centos-7/bin/delete-list create mode 100644 container/osimage/centos-7/bin/install-list create mode 100755 container/osimage/centos-7/bin/lib create mode 100755 container/osimage/centos-7/build diff --git a/container/osimage/centos-7/README.md b/container/osimage/centos-7/README.md new file mode 100644 index 0000000..2c2f956 --- /dev/null +++ b/container/osimage/centos-7/README.md @@ -0,0 +1,13 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + cd ${CCI_SRC/rootfs/initramfs/centos/aarch64/7} + ./build + +Some configuration items: +./bin/install-list + If you want to pre-install the software, you can write the package names in ./bin/install-list. + +./bin/delete-list + If you want remove some unnecessary files, you can write the names in ./bin/delete-list + diff --git a/container/osimage/centos-7/bin/create-image b/container/osimage/centos-7/bin/create-image new file mode 100755 index 0000000..ad51986 --- /dev/null +++ b/container/osimage/centos-7/bin/create-image @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/install-list) + yum clean all + rm -rf $(</root/bin/delete-list) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/container/osimage/centos-7/bin/delete-list b/container/osimage/centos-7/bin/delete-list new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/container/osimage/centos-7/bin/delete-list @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/container/osimage/centos-7/bin/install-list b/container/osimage/centos-7/bin/install-list new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/container/osimage/centos-7/bin/install-list @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/container/osimage/centos-7/bin/lib b/container/osimage/centos-7/bin/lib new file mode 100755 index 0000000..c3c470f --- /dev/null +++ b/container/osimage/centos-7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(basename $(dirname $(realpath $0)))" +name="$(echo $DIR |awk -F "-" '{print $1}')" +version="$(echo $DIR |awk -F "-" '{print $2}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/container/osimage/centos-7/build b/container/osimage/centos-7/build new file mode 100755 index 0000000..5e46306 --- /dev/null +++ b/container/osimage/centos-7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos-7/bin/lib +. ${CCI_SRC}/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/create-image +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH lkp-tests] /lkp-tests/jobs: add test-tools.yaml
by Li Ping 12 Oct '20

12 Oct '20

--- jobs/test-tools.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 jobs/test-tools.yaml diff --git a/jobs/test-tools.yaml b/jobs/test-tools.yaml new file mode 100644 index 00000000..43ca1343 --- /dev/null +++ b/jobs/test-tools.yaml @@ -0,0 +1,8 @@ +suite: test-tools +category: functional +os_mount: initramfs + +test-tools: + add: 1 + testsuite: smoke-testing + testcase: oe_test_dd_001 -- 2.23.0

1 0

[PATCH compass-ci] fix: scheduler: output exception backtrace
by Tong Qunfeng 12 Oct '20

12 Oct '20

we catch exception at submit_job and prepare_job, and just output the exception message. it is not easy to find out detail reason, so add backtrace. Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- src/lib/sched.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2f432a9..23332df 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -188,6 +188,7 @@ class Sched return submit_single_job(job) rescue ex + puts ex.inspect_with_backtrace return [{ "job_id" => "0", "message" => ex.to_s, @@ -444,6 +445,7 @@ class Sched job = @es.get_job(job_id.to_s) rescue ex puts "Invalid job (id=#{job_id}) in es. Info: #{ex}" + puts ex.inspect_with_backtrace end end -- 2.23.0

1 0

[PATCH compass-ci] mount busybox into container
by Xiao Shenwei 12 Oct '20

12 Oct '20

some container may missing basic commands, such as curl wget ... use busybox to replace. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/run.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 6798f48..3c3b0ec 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -18,6 +18,7 @@ host=${tbox_group%%--*} create_yaml_variables "$LKP_SRC/hosts/${host}" DIR=$(dirname $(realpath $0)) +busybox_path=$(command -v busybox) cmd=( docker run --rm @@ -28,6 +29,7 @@ cmd=( -v ${DIR}/bin:/root/bin:ro -v $CCI_SRC:/c/compass-ci -v /srv/git:/srv/git:ro + -v ${busybox_path}:/bin/busybox --oom-score-adj="-1000" ${docker_image} /root/bin/entrypoint.sh -- 2.23.0

2 1

[PATCH v3 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 12 Oct '20

12 Oct '20

Use the tool to create a new centos-7 initramfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initramfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initramfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initramfs/centos/aarch64/7/bin/lib | 30 +++++++++++++++ .../initramfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initramfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 rootfs/initramfs/centos/aarch64/7/README.md create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initramfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initramfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/README.md b/rootfs/initramfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..1e47e11 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/creat b/rootfs/initramfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..4dacbdf --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config password +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # Change the password in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Configure ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initramfs/centos/aarch64/7/bin/dellist b/rootfs/initramfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initramfs/centos/aarch64/7/bin/lib b/rootfs/initramfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..1bf0b5d --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/lib @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $(NF - 2)}')" +version="$(echo $DIR |awk -F "/" '{print $NF}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# Check password file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# Pull docker image +pull_docker() +{ + docker pull $image || return + echo "finish downloading image" +} + diff --git a/rootfs/initramfs/centos/aarch64/7/bin/repolist b/rootfs/initramfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 4

[PATCH compass-ci] feat: a link for one command to build centos-7 initramfs image
by Wang Chenglong 12 Oct '20

12 Oct '20

This link is used to one command to build centos-7 initramfs image Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/centos/aarch64/7/build | 1 + 1 file changed, 1 insertion(+) create mode 120000 rootfs/initramfs/centos/aarch64/7/build diff --git a/rootfs/initramfs/centos/aarch64/7/build b/rootfs/initramfs/centos/aarch64/7/build new file mode 120000 index 0000000..e3adb9c --- /dev/null +++ b/rootfs/initramfs/centos/aarch64/7/build @@ -0,0 +1 @@ +../../../../../container/osimage/centos-7/build \ No newline at end of file -- 2.23.0

1 0

[PATCH compass-ci] fix rubocop warnings on *.cr
by Wu Fengguang 12 Oct '20

12 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- src/extract-stats/constants.cr | 2 +- src/extract-stats/extract_stats.cr | 10 +++++----- src/monitoring/monitoring.cr | 1 + src/scheduler/jobfile_operate.cr | 21 +++++++++++---------- src/spec/scheduler/tools_spec.cr | 8 ++++---- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/src/extract-stats/constants.cr b/src/extract-stats/constants.cr index 332962a..70c66ad 100644 --- a/src/extract-stats/constants.cr +++ b/src/extract-stats/constants.cr @@ -5,4 +5,4 @@ EXTRACT_STATS_QUEUE_PATH = "extract_stats" STATS_WORKER_COUNT = 10 DELIMITER_TASK_QUEUE = "delimiter" -ERROR_ID_FILES = %w(build-pkg.json) +ERROR_ID_FILES = %w[build-pkg.json] diff --git a/src/extract-stats/extract_stats.cr b/src/extract-stats/extract_stats.cr index d3f3203..45310ad 100644 --- a/src/extract-stats/extract_stats.cr +++ b/src/extract-stats/extract_stats.cr @@ -7,15 +7,15 @@ require "./stats_worker" module ExtractStats # Consume scheduler queue def self.in_extract_stats - self.back_fill_task + back_fill_task STATS_WORKER_COUNT.times do - Process.fork { - self.consume_task - } + Process.fork do + consume_task + end end # keep main-process alive - sleep() + sleep end def self.consume_task diff --git a/src/monitoring/monitoring.cr b/src/monitoring/monitoring.cr index 178e4c9..e665d5f 100644 --- a/src/monitoring/monitoring.cr +++ b/src/monitoring/monitoring.cr @@ -24,6 +24,7 @@ module Monitoring socket.on_close do next unless query.as_h? + filter.remove_filter_rule(query, socket) end end diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 0533346..e0e298f 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -15,7 +15,7 @@ end module Jobfile::Operate def self.prepare_dir(file_path : String) file_path_dir = File.dirname(file_path) - if !File.exists?(file_path_dir) + unless File.exists?(file_path_dir) FileUtils.mkdir_p(file_path_dir) end end @@ -59,11 +59,12 @@ module Jobfile::Operate if val.as_h? return false end - if val.as_a? - value = shell_escape(val.as_a) - else - value = shell_escape(val.to_s) - end + + value = if val.as_a? + shell_escape(val.as_a) + else + shell_escape(val.to_s) + end script_lines << "\texport #{key}=" + value if value end end @@ -78,7 +79,7 @@ module Jobfile::Operate script_lines << "\texport LKP_SRC=/lkp/${user:-lkp}/src" script_lines << "}\n\n" - script_lines = "#{script_lines}" + script_lines = script_lines.to_s script_lines = JSON.parse(script_lines) end @@ -121,7 +122,7 @@ module Jobfile::Operate if job_sh_array.empty? lkp_src = prepare_lkp_tests(job_content["lkp_initrd_user"], - job_content["os_arch"]) + job_content["os_arch"]) cmd = "#{lkp_src}/sbin/create-job-cpio.sh #{temp_yaml}" idd = `#{cmd}` @@ -158,8 +159,8 @@ module Jobfile::Operate # update lkp-xxx.cgz if they are different target_path = update_lkp_when_different(expand_dir_base, - lkp_initrd_user, - os_arch) + lkp_initrd_user, + os_arch) # delete oldest lkp, if exists too much del_lkp_if_too_much(expand_dir_base) diff --git a/src/spec/scheduler/tools_spec.cr b/src/spec/scheduler/tools_spec.cr index 0a71af0..4c3a6aa 100644 --- a/src/spec/scheduler/tools_spec.cr +++ b/src/spec/scheduler/tools_spec.cr @@ -8,16 +8,16 @@ require "file_utils" describe Public do describe "hash replace" do it "can replace k:v" do - hash_old = {"name" => "OldValue"} - hash_new = {"name" => "NewValue"} + hash_old = { "name" => "OldValue" } + hash_new = { "name" => "NewValue" } hash_result = Public.hash_replace_with(hash_old, hash_new) (hash_result["name"]).should eq("NewValue") end it "can add k:v" do - hash_old = {"name2" => "OldValue"} - hash_new = {"name" => "NewValue"} + hash_old = { "name2" => "OldValue" } + hash_new = { "name" => "NewValue" } hash_result = Public.hash_replace_with(hash_old, hash_new) (hash_result["name"]).should eq("NewValue") -- 2.23.0

1 0

[PATCH lkp-tests] tests: tests: add get_ipmi_ip function to host-info
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..3cabebbf 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

2 1

[PATCH lkp-tests] tests: add get_ipmi_ip function to host-info
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..11cbed38 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip() } main -- 2.23.0

1 0

[PATCH lkp-tests] tests: fix 'memory' field more precise
by Zhang Yu 12 Oct '20

12 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..a57855dc 100755 --- a/tests/host-info +++ b/tests/host-info @@ -5,8 +5,8 @@ get_node() echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') + memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -- 2.23.0

1 0

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initramfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..1e47e11 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initramfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

2 3

[PATCH lkp-tests] tests: fix 'memory' field more precise and add ipmi_ip
by Zhang Yu 10 Oct '20

10 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 9b8bfc50..6ee8df21 100755 --- a/tests/host-info +++ b/tests/host-info @@ -5,8 +5,8 @@ get_node() echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') + memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } @@ -80,6 +80,11 @@ get_partitions() fi } +get_ipmi_ip() +{ + echo "ipmi_ip: $(ipmitool lan print |grep -E "IP Address\s*:" | awk '{print $4}')" +} + main() { get_node @@ -87,6 +92,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

2 1

[PATCH compass-ci] mount busybox into container
by Xiao Shenwei 10 Oct '20

10 Oct '20

some container may missing basic commands, such as curl wget ... use busybox to replace. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/run.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 6798f48..f8aba13 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -28,6 +28,7 @@ cmd=( -v ${DIR}/bin:/root/bin:ro -v $CCI_SRC:/c/compass-ci -v /srv/git:/srv/git:ro + -v /usr/bin/busybox:/usr/bin/busybox --oom-score-adj="-1000" ${docker_image} /root/bin/entrypoint.sh -- 2.23.0

3 8

[PATCH compass-ci] feat: build initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 +++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 46 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 32 +++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 28 +++++++++++ 6 files changed, 128 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..c503925 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +#Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package name in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the name in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..8cd0d2c --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,46 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +#Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idmv >/dev/null + + +} + +#config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + echo $passwd_md5 + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" + } + +} + +#pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" |cpio -o -Hnewc |gzip -9 >/${image_name} + chmod 644 /${image_name} +} + + +make_rootfs +pre_config_rootfs +cat $ROOTFS_DIR/etc/shadow +pack_cgz + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..da0cdaa --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,32 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +image="centos:7" +name="$(echo "$image" | cut -d '/' -f 2 | cut -d ':' -f 1)" +version="$(echo "$image" | cut -d '/' -f 2 | cut -d ':' -f 2)" +image_name="${name}-${version}.cgz" + +#check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + + +#pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh down image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..36a8216 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker rm -f init_docker 2&> /dev/null +. ./bin/lib + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

4 13

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 29 ++++++++++++++ 6 files changed, 120 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..ff79352 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +# Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..5e1a0a0 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,29 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib +. $CCI_SRC/container/defconfig.sh + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker_rm init_docker +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

1 1

[PATCH compass-ci] fix(dnsmasq): DHCP lease record will be lost after restart container
by Yu Chuan 10 Oct '20

10 Oct '20

Persist the lease file to /etc/compass-ci/dnsmasq/lease_dir so that each restart of the container won't cause the lease file to be lost. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/start | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/container/dnsmasq/start b/container/dnsmasq/start index 7230c9cf6188..ff4247046eb1 100755 --- a/container/dnsmasq/start +++ b/container/dnsmasq/start @@ -9,10 +9,15 @@ docker_rm dnsmasq kill $(ps -ef| grep dnsmasq| grep libvirt| awk '{print $2}') 2> /dev/null WORKSPACE=$(dirname "$(realpath "$0")") +DNSMASQ_CONF_DIR="/etc/compass-ci/dnsmasq" # load custom dnsmasq.conf -[ -d "/etc/compass-ci/dnsmasq.d/" ] && - cp -a /etc/compass-ci/dnsmasq.d/* ${WORKSPACE}/dnsmasq.d/ +[ -d "${DNSMASQ_CONF_DIR}/dnsmasq.d/" ] && + cp -a ${DNSMASQ_CONF_DIR}/dnsmasq.d/* ${WORKSPACE}/dnsmasq.d/ + +# persist the dhcp lease file +[ -d "${DNSMASQ_CONF_DIR}/lease_dir" ] || + mkdir -p ${DNSMASQ_CONF_DIR}/lease_dir cmd=( docker run @@ -22,6 +27,7 @@ cmd=( --publish 67:67/udp --publish 69:69/udp -v ${WORKSPACE}/dnsmasq.d:/etc/dnsmasq.d + -v ${DNSMASQ_CONF_DIR}/lease_dir:/var/lib/misc # persist the dhcp lease file -v /etc/localtime:/etc/localtime:ro -v /tftpboot:/tftpboot:ro --detach -- 2.23.0

1 0

[PATCH v2 compass-ci] feat: build centos-7 initramfs image with on click
by Wang Chenglong 10 Oct '20

10 Oct '20

Use the tool to create a new centos-7 initfamfs image with on click. and we also can customize the pre-installed software by configuring ./bin/repolist. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initfamfs/centos/aarch64/7/README.md | 12 ++++++ rootfs/initfamfs/centos/aarch64/7/bin/creat | 38 +++++++++++++++++++ rootfs/initfamfs/centos/aarch64/7/bin/dellist | 8 ++++ rootfs/initfamfs/centos/aarch64/7/bin/lib | 31 +++++++++++++++ .../initfamfs/centos/aarch64/7/bin/repolist | 2 + rootfs/initfamfs/centos/aarch64/7/build | 28 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 rootfs/initfamfs/centos/aarch64/7/README.md create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/creat create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/dellist create mode 100755 rootfs/initfamfs/centos/aarch64/7/bin/lib create mode 100644 rootfs/initfamfs/centos/aarch64/7/bin/repolist create mode 100755 rootfs/initfamfs/centos/aarch64/7/build diff --git a/rootfs/initfamfs/centos/aarch64/7/README.md b/rootfs/initfamfs/centos/aarch64/7/README.md new file mode 100644 index 0000000..f6af3ca --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/README.md @@ -0,0 +1,12 @@ +#Use the tool to create a new centos-7 initfamfs image. + +Usage: + ./build + +Some configuration items: +./bin/repolist + If you want to pre-install the software, you can write the package names in ./bin/repolist. + +./bin/dellist + If you want remove some unnecessary files, you can write the names in ./bin/dellist + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/creat b/rootfs/initfamfs/centos/aarch64/7/bin/creat new file mode 100755 index 0000000..a9947e2 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/creat @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# Configure rootfs +make_rootfs() +{ + yum repolist + yum -y install --skip-broken $(</root/bin/repolist) + yum clean all + rm -rf $(</root/bin/dellist) + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + cd /root/modules + zcat modules-5.8.0.1.cgz | cpio -idm +} + +# Config passwd +pre_config_rootfs() { + [ -n "$ROOT_NEW_PASSWD" ] && { + echo "Changing root password" + passwd_md5=$(openssl passwd -1 "$ROOT_NEW_PASSWD") + sed -i -r "s/^root:[^:]*:(.*)/root:${passwd_md5//\//\\/}:\1/" "$ROOTFS_DIR/etc/shadow" # change the passwd in shadow + sed -i 's/[# ]PermitRootLogin.*/PermitRootLogin yes/' "$ROOTFS_DIR/etc/ssh/sshd_config" # Config ssh service + } +} + +# Pack cgz package +pack_cgz() +{ + echo "Packing package. Please wait." + cd / + find ./ ! -path "./${image_name}" ! -path "./root/modules/modules-5.8.0.1.cgz" | cpio -o -Hnewc | gzip -9 > /${image_name} + chmod 644 /${image_name} +} + +make_rootfs +pre_config_rootfs +pack_cgz diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/dellist b/rootfs/initfamfs/centos/aarch64/7/bin/dellist new file mode 100644 index 0000000..ce079b9 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/dellist @@ -0,0 +1,8 @@ +/.dockerenv +/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/lib b/rootfs/initfamfs/centos/aarch64/7/bin/lib new file mode 100755 index 0000000..dec8bb3 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/lib @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + + +DIR="$(dirname $(realpath $0))" +name="$(echo $DIR |awk -F "/" '{print $7}')" +version="$(echo $DIR |awk -F "/" '{print $9}')" +image="${name}:${version}" +image_name="${name}-${version}-$(date +"%Y%m%d").cgz" + +# check passwd file +check_passwd_file() { + root_pwd_file="$HOME/.config/compass-ci/rootfs.passwd" + export ROOT_NEW_PASSWD= + [ -f "$root_pwd_file" ] || { + echo "[INFO] Please set the password file." + echo "$HOME/.config/compass-ci/rootfs.passwd" + exit 1 + } + + export ROOT_NEW_PASSWD=$(cat "$root_pwd_file") +} + +# pull_docker +pull_docker() +{ + docker pull $image || return + echo "finsh downloading image" +} + diff --git a/rootfs/initfamfs/centos/aarch64/7/bin/repolist b/rootfs/initfamfs/centos/aarch64/7/bin/repolist new file mode 100644 index 0000000..5d31c40 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/bin/repolist @@ -0,0 +1,2 @@ +openssh-server +openssl diff --git a/rootfs/initfamfs/centos/aarch64/7/build b/rootfs/initfamfs/centos/aarch64/7/build new file mode 100755 index 0000000..f47a701 --- /dev/null +++ b/rootfs/initfamfs/centos/aarch64/7/build @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ./bin/lib + +check_passwd_file "$root_pwd_file" + +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/bin/:/root/bin + -v /srv/initrd/modules/:/root/modules:ro + -e ROOT_NEW_PASSWD=$ROOT_NEW_PASSWD + -e image_name=$image_name + $image + /root/bin/creat +) + +pull_docker + +docker rm -f init_docker 2&> /dev/null +${cmd[@]} + +docker cp -a init_docker:/${image_name} ./ +echo "build finished!!!" + -- 2.23.0

1 1

[PATCH compass-ci] get busybox file from debian container
by Xiao Shenwei 10 Oct '20

10 Oct '20

add apt-get install -y busybox then copy it to host Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/qemu-efi/Dockerfile | 2 +- container/qemu-efi/install | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/container/qemu-efi/Dockerfile b/container/qemu-efi/Dockerfile index e1087c5..ced7f66 100644 --- a/container/qemu-efi/Dockerfile +++ b/container/qemu-efi/Dockerfile @@ -8,4 +8,4 @@ ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y qemu-efi-aarch64 + apt-get install -y qemu-efi-aarch64 busybox diff --git a/container/qemu-efi/install b/container/qemu-efi/install index 96be9aa..bc61b9a 100755 --- a/container/qemu-efi/install +++ b/container/qemu-efi/install @@ -12,4 +12,7 @@ cmd=( "${cmd[@]}" docker cp qemu-efi:/usr/share/qemu-efi-aarch64 /usr/share/qemu-efi-aarch64 +[ -f "/usr/bin/busybox" ] || { + docker cp qemu-efi:/bin/busybox /usr/bin/busybox +} docker_rm qemu-efi -- 2.23.0

3 3

[PATCH compass-ci] sparrow: install busybox
by Wu Fengguang 10 Oct '20

10 Oct '20

For use by docker provider. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 8700ac5..261592c 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -19,6 +19,7 @@ pkgs=( redis liblockfile lockfile-progs + busybox ) yum install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 68423d7..020c34a 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -17,6 +17,7 @@ pkgs=( dmidecode lockfile-progs bridge-utils + busybox ) apt-get install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 3573006..5c64fe8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -18,6 +18,7 @@ pkgs=( dmidecode redis wget + busybox ) yum install -y "${pkgs[@]}" -- 2.23.0

2 1

[PATCH lkp-tests] add depends for multi-docker
by Wu Fengguang 10 Oct '20

10 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- distro/depends/multi-docker | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 distro/depends/multi-docker diff --git a/distro/depends/multi-docker b/distro/depends/multi-docker new file mode 100644 index 00000000..b7b0bacf --- /dev/null +++ b/distro/depends/multi-docker @@ -0,0 +1,4 @@ +docker.io +busybox +ruby +ruby-json -- 2.23.0

1 0

[PATCH lab-z9 2/2] fix memory size, and the 128G's host will be added to 256G as soon
by Zhang Dewan 10 Oct '20

10 Oct '20

--- hosts/taishan200-2280-2s48p-256g--a13 | 2 +- hosts/taishan200-2280-2s48p-256g--a14 | 2 +- hosts/taishan200-2280-2s48p-256g--a15 | 2 +- hosts/taishan200-2280-2s48p-256g--a17 | 2 +- hosts/taishan200-2280-2s48p-256g--a18 | 2 +- hosts/taishan200-2280-2s48p-256g--a19 | 2 +- hosts/taishan200-2280-2s48p-256g--a21 | 2 +- hosts/taishan200-2280-2s48p-256g--a22 | 2 +- hosts/taishan200-2280-2s48p-256g--a24 | 2 +- hosts/taishan200-2280-2s48p-256g--a25 | 2 +- hosts/taishan200-2280-2s48p-256g--a26 | 2 +- hosts/taishan200-2280-2s48p-256g--a27 | 2 +- hosts/taishan200-2280-2s48p-256g--a28 | 2 +- hosts/taishan200-2280-2s48p-256g--a29 | 2 +- hosts/taishan200-2280-2s48p-256g--a30 | 2 +- hosts/taishan200-2280-2s48p-256g--a31 | 2 +- hosts/taishan200-2280-2s48p-256g--a32 | 2 +- hosts/taishan200-2280-2s48p-256g--a33 | 2 +- hosts/taishan200-2280-2s48p-256g--a34 | 2 +- hosts/taishan200-2280-2s48p-256g--a52 | 2 +- hosts/taishan200-2280-2s48p-256g--a53 | 2 +- hosts/taishan200-2280-2s48p-256g--a60 | 2 +- hosts/taishan200-2280-2s48p-256g--a61 | 2 +- hosts/taishan200-2280-2s48p-256g--a62 | 2 +- hosts/taishan200-2280-2s48p-256g--a66 | 2 +- hosts/taishan200-2280-2s48p-256g--a67 | 2 +- hosts/taishan200-2280-2s48p-256g--a68 | 2 +- hosts/taishan200-2280-2s48p-256g--a69 | 2 +- hosts/taishan200-2280-2s48p-256g--a70 | 2 +- hosts/taishan200-2280-2s48p-256g--a71 | 2 +- hosts/taishan200-2280-2s64p-256g--a10 | 2 +- hosts/taishan200-2280-2s64p-256g--a40 | 2 +- hosts/taishan200-2280-2s64p-256g--a41 | 2 +- hosts/taishan200-2280-2s64p-256g--a42 | 2 +- 34 files changed, 34 insertions(+), 34 deletions(-) diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 index 1c8f554..fe05233 100644 --- a/hosts/taishan200-2280-2s48p-256g--a13 +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c500c2799bb3 diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 index ae84bd5..bd5962b 100644 --- a/hosts/taishan200-2280-2s48p-256g--a14 +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4475 diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 index 7fa0b67..9b0cd96 100644 --- a/hosts/taishan200-2280-2s48p-256g--a15 +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b841fff5 diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 index 8be93de..cbbf268 100644 --- a/hosts/taishan200-2280-2s48p-256g--a17 +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b44cd diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 index e86bdeb..648a1db 100644 --- a/hosts/taishan200-2280-2s48p-256g--a18 +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b891b4d1 diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 index 0c4a60f..2aec477 100644 --- a/hosts/taishan200-2280-2s48p-256g--a19 +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8923939 diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 index 51b7709..9cbb5b5 100644 --- a/hosts/taishan200-2280-2s48p-256g--a21 +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf2d5 diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 index 4de0ad0..6af83a7 100644 --- a/hosts/taishan200-2280-2s48p-256g--a22 +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919069 diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index f47947c..dd9b501 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8917d59 diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index 2f270e8..5d075b9 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4499 diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 index b6de96e..fe395ec 100644 --- a/hosts/taishan200-2280-2s48p-256g--a26 +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf131 diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 index b6d9945..63b94f3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a27 +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918ea9 diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 index ecebafc..df048f7 100644 --- a/hosts/taishan200-2280-2s48p-256g--a28 +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdf1eb07 diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 index bdb13ca..849be55 100644 --- a/hosts/taishan200-2280-2s48p-256g--a29 +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918425 diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 index 810859b..2be9047 100644 --- a/hosts/taishan200-2280-2s48p-256g--a30 +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b881 diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 index 8f9d13b..8af7d18 100644 --- a/hosts/taishan200-2280-2s48p-256g--a31 +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b929 diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 index 66d9b68..fb52e7f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a32 +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf141 diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 index ef3b148..4095419 100644 --- a/hosts/taishan200-2280-2s48p-256g--a33 +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919555 diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 index 528ec82..bae3e58 100644 --- a/hosts/taishan200-2280-2s48p-256g--a34 +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be219 diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 index 4353d4c..654b021 100644 --- a/hosts/taishan200-2280-2s48p-256g--a52 +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdefd89b diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 index 0df1cd9..a90b8eb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a53 +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 2 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6aa4e3 diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 index 6986a8f..b87d962 100644 --- a/hosts/taishan200-2280-2s48p-256g--a60 +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd67640b diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 index 1c198b8..b33712c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a61 +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399886a7869 diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 index 26ab646..5c3b04d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a62 +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b892d131 diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index c7b2083..88ef791 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c5009a671773 diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index f207003..97ea498 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88aed21 diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index 054105c..f3d50f4 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8909f81 diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index e96e34c..9e5734e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be92d diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 61f99a3..0dd8208 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b890931d diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 index 4896475..dde6205 100644 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ b/hosts/taishan200-2280-2s48p-256g--a71 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 252G +memory: 256G mac_addr: - 44:67:47:d7:6d:e6 - 44:67:47:d7:6d:e7 diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index a66bff0..4687d1a 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 128 -memory: 252G +memory: 256G nr_hdd_partitions: 1 nr_ssd_partitions: 1 hdd_partitions: diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 index 91f5619..c0ce550 100644 --- a/hosts/taishan200-2280-2s64p-256g--a40 +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 128 -memory: 252G +memory: 256G nr_hdd_partitions: 2 hdd_partitions: - /dev/disk/by-id/scsi-3500003999812ed81 diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 index 9525cfd..9e01396 100644 --- a/hosts/taishan200-2280-2s64p-256g--a41 +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 128 -memory: 252G +memory: 256G nr_hdd_partitions: 2 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919495 diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 index 40c1c4b..1a953df 100644 --- a/hosts/taishan200-2280-2s64p-256g--a42 +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 128 -memory: 252G +memory: 256G nr_hdd_partitions: 2 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918a11 -- 2.23.0

2 1

[PATCH compass-ci] providers: change vm-hi1620 to vm-aarch64
by Bai Jing 10 Oct '20

10 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- providers/multi-qemu | 2 +- providers/my-qemu.sh | 2 +- providers/qemu.sh | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/providers/multi-qemu b/providers/multi-qemu index 72453fb..76e39ee 100755 --- a/providers/multi-qemu +++ b/providers/multi-qemu @@ -8,7 +8,7 @@ require 'fileutils' PWD = Dir.pwd # Run multiple QEMU in parallel -HOSTNAME = ARGV[0] || "vm-hi1620-2p8g--#{ENV['USER']}" +HOSTNAME = ARGV[0] || "vm-aarch64-2p8g--#{ENV['USER']}" NR_VM = ARGV[1] || 1 def run(seqno) diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 2f20f67..f52b2ba 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. [[ $tbox_group ]] || -tbox_group=vm-hi1620-2p8g +tbox_group=vm-aarch64-2p8g export hostname=$tbox_group--$USER-$$ $CCI_SRC/providers/qemu.sh diff --git a/providers/qemu.sh b/providers/qemu.sh index 786a7aa..aed8c3d 100755 --- a/providers/qemu.sh +++ b/providers/qemu.sh @@ -8,7 +8,7 @@ load_cci_defaults -: ${hostname:="vm-hi1620-1p1g-1"} +: ${hostname:="vm-aarch64-1p1g-1"} # unicast prefix: x2, x6, xA, xE export mac=$(echo $hostname | md5sum | sed 's/^$..$$..$$..$$..$$..$.*$/0a-\1-\2-\3-\4-\5/') echo hostname: $hostname -- 2.23.0

2 1

[PATCH v3 compass-ci] sched: refactor get_boot_ipxe to job class
by Cao Xueliang 10 Oct '20

10 Oct '20

The function get_boot_ipxe use initrds_uri, kernel_uri, kernel_params from job class as response. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 121 +++++++++++++++++++++++++++++++++---- src/lib/sched.cr | 65 ++------------------ src/scheduler/constants.cr | 4 ++ 3 files changed, 117 insertions(+), 73 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 9786cd9..bf959c2 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -72,11 +72,14 @@ class Job lab initrd_pkg initrd_deps + initrds_uri result_root access_key access_key_file lkp_initrd_user + kernel_uri kernel_append_root + kernel_params docker_image ) @@ -123,8 +126,11 @@ class Job set_result_root() set_result_service() set_os_mount() - set_kernel_append_root() set_pp_initrd() + set_initrds_uri() + set_kernel_uri() + set_kernel_append_root() + set_kernel_params() set_lkp_server() end @@ -228,27 +234,117 @@ class Job end end - private def set_kernel_append_root - os_real_path = JobHelper.service_path("#{SRV_OS}/#{os_dir}") - lkp_real_path = JobHelper.service_path("#{SRV_OS}/#{os_dir}/initrd.lkp") - lkp_basename = File.basename(lkp_real_path) + private def vmlinuz + if os_mount == "initramfs" + return "#{INITRD_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{os_dir}/vmlinuz")}" + else + return "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + end + end + + private def set_kernel_uri + self["kernel_uri"] = "kernel #{vmlinuz()}" + end + + private def kernel_common_params + return "user=lkp job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" + end + + private def common_initrds + temp_initrds = [] of String + + temp_initrds << "#{INITRD_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_INITRD}/lkp/#{lkp_initrd_user}/lkp-#{os_arch}.cgz")}" + temp_initrds << "#{SCHED_HTTP_PREFIX}/job_initrd_tmpfs/#{id}/job.cgz" + + return temp_initrds + end + + private def initramfs_initrds + temp_initrds = [] of String + + temp_initrds << "#{INITRD_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{os_dir}/current")}" + temp_initrds << "#{INITRD_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{os_dir}/run-ipconfig.cgz")}" + + temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? + temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? + + return temp_initrds + end + + private def nfs_cifs_initrds + temp_initrds = [] of String + + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/initrd.lkp")}" + + return temp_initrds + end + + private def get_initrds + temp_initrds = [] of String - current_basename = "" if "#{os_mount}" == "initramfs" - current_real_path = JobHelper.service_path("#{SRV_INITRD}/osimage/#{os_dir}/current") - current_basename = File.basename(current_real_path) + temp_initrds.concat(initramfs_initrds()) + else + temp_initrds.concat(nfs_cifs_initrds()) + end + + temp_initrds.concat(common_initrds()) + + return temp_initrds + end + + private def initrds_basename + basenames = "" + + get_initrds().each do |initrd| + basenames += "initrd=#{File.basename(initrd)} " end + return basenames + end + + private def set_initrds_uri + uris = "" + + get_initrds().each do |initrd| + uris += "initrd #{initrd}\n" + end + + self["initrds_uri"] = uris + end + + private def set_kernel_append_root + os_real_path = JobHelper.service_path("#{SRV_OS}/#{os_dir}") + fs2root = { - "nfs" => "root=#{OS_HTTP_HOST}:#{os_real_path} initrd=#{lkp_basename}", + "nfs" => "root=#{OS_HTTP_HOST}:#{os_real_path} #{initrds_basename()}", "cifs" => "root=cifs://#{OS_HTTP_HOST}#{os_real_path}" + - ",guest,ro,hard,vers=1.0,noacl,nouser_xattr initrd=#{lkp_basename}", - "initramfs" => "rdinit=/sbin/init prompt_ramdisk=0 initrd=#{current_basename}", - "container" => "" + ",guest,ro,hard,vers=1.0,noacl,nouser_xattr #{initrds_basename()}", + "initramfs" => "rdinit=/sbin/init prompt_ramdisk=0 #{initrds_basename()}", + "container" => "", } + self["kernel_append_root"] = fs2root[os_mount] end + private def kernel_console + if os_arch == "x86_64" + return "console=ttyS0,115200 console=tty0" + else + return "" + end + end + + private def set_kernel_params + self["kernel_params"] = " #{kernel_common_params()} #{kernel_append_root} #{kernel_console()}" + end + private def set_pp_initrd initrd_deps_arr = Array(String).new initrd_pkg_arr = Array(String).new @@ -270,6 +366,7 @@ class Job end end end + self["initrd_deps"] = initrd_deps_arr.join(" ") self["initrd_pkg"] = initrd_pkg_arr.join(" ") end diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 47a2e54..2f432a9 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -480,69 +480,12 @@ class Sched extra_job_fields) if Dir.glob(full_path_patterns).size > 0 end - private def add_kernel_console_param(arch_tmp) - returned = "" - if arch_tmp == "x86_64" - returned = " console=ttyS0,115200 console=tty0" - end - return returned - end - - private def get_pp_initrd(job : Job) - initrd_deps = "" - initrd_pkg = "" - if job.os_mount == "initramfs" - initrd_deps += job.initrd_deps.split.join() { |item| "initrd #{item}\n" } - initrd_pkg += job.initrd_pkg.split.join() { |item| "initrd #{item}\n" } - end - return initrd_deps, initrd_pkg - end - private def get_boot_ipxe(job : Job) - initrd_lkp_cgz = "lkp-#{job.os_arch}.cgz" - - initrd_deps, initrd_pkg = get_pp_initrd(job) - - initrd_http_prefix = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" - sched_http_prefix = "http://#{SCHED_HOST}:#{SCHED_PORT}" - os_http_prefix = "http://#{OS_HTTP_HOST}:#{OS_HTTP_PORT}" - response = "#!ipxe\n\n" - if job.os_mount == "initramfs" - response += "initrd #{initrd_http_prefix}" + - "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/current")}\n" - response += "initrd #{initrd_http_prefix}" + - "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/run-ipconfig.cgz")}\n" - else - response += "initrd #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/initrd.lkp")}\n" - end - response += "initrd #{initrd_http_prefix}" + - "#{JobHelper.service_path("#{SRV_INITRD}/lkp/#{job.lkp_initrd_user}/#{initrd_lkp_cgz}")}\n" - response += "initrd #{sched_http_prefix}/job_initrd_tmpfs/#{job.id}/job.cgz\n" - response += initrd_deps - response += initrd_pkg - if job.os_mount == "initramfs" - response += "kernel #{initrd_http_prefix}" + - "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/vmlinuz")}" - else - response += "kernel #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" - end - response += " user=lkp" - response += " job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" - response += " #{job.kernel_append_root}" - response += add_kernel_console_param(job.os_arch) - if job.os_mount == "initramfs" - response += " initrd=#{initrd_lkp_cgz} initrd=job.cgz" - job.initrd_deps.split.each do |initrd_dep| - response += " initrd=#{File.basename(initrd_dep)}" - end - response += " initrd=#{File.basename(JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/run-ipconfig.cgz"))}\n" - else - response += " initrd=#{initrd_lkp_cgz} initrd=job.cgz\n" - end - response += "boot\n" + response += job.initrds_uri + response += job.kernel_uri + response += job.kernel_params + response += "\nboot\n" puts %({"job_id": "#{job.id}", "job_state": "boot"}) return response diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index 7ce49a1..839c241 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -24,4 +24,8 @@ OS_HTTP_PORT = (ENV.has_key?("OS_HTTP_PORT") ? ENV["OS_HTTP_PORT"] : 8000).to_i3 SRV_OS = "/srv/os" SRV_INITRD = "/srv/initrd" +INITRD_HTTP_PREFIX = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" +OS_HTTP_PREFIX = "http://#{OS_HTTP_HOST}:#{OS_HTTP_PORT}" +SCHED_HTTP_PREFIX = "http://#{SCHED_HOST}:#{SCHED_PORT}" + DEMO_JOB = %({"suite":"pixz","testcase":"pixz","category":"benchmark","nr_threads":1,"pixz":null,"job_origin":"jobs/pixz.yaml","testbox":"wfg-e595","arch":"x86_64","tbox_group":"wfg-e595","id":"100","kmsg":null,"boot-time":null,"uptime":null,"iostat":null,"heartbeat":null,"vmstat":null,"numa-numastat":null,"numa-vmstat":null,"numa-meminfo":null,"proc-vmstat":null,"proc-stat":null,"meminfo":null,"slabinfo":null,"interrupts":null,"kconfig":"x86_64-rhel-7.6","compiler":"gcc-7"}) -- 2.23.0

1 0

[PATCH v4 compass-ci 1/7] container/mail-robot: mail-robot.rb
by Luan Shengde 10 Oct '20

10 Oct '20

mail robot for application email monitor mailbox check new added email file in new check new mail's subject return if unmatched subject call answerback-email apply account apply uuid write account/uuid etc. to es send uuid to user Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/mail-robot.rb | 40 ++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 container/mail-robot/mail-robot.rb diff --git a/container/mail-robot/mail-robot.rb b/container/mail-robot/mail-robot.rb new file mode 100755 index 0000000..f91b89d --- /dev/null +++ b/container/mail-robot/mail-robot.rb @@ -0,0 +1,40 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' +require_relative 'answerback-email.rb' + +mail_inbox = '/Maildir/.compass-ci/new/' +mail_drafts = '/Maildir/.compass-ci/cur/' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |modified, added, _removed| + unless added.empty? + added.each do |mail_file| + begin + check_to_send_account(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + end + end + end + end + listener.start + sleep +end + +def check_to_send_account(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + subject = mail_content.subject + return unless subject =~ /apply account/i + + send_account(mail_content) + FileUtils.mv(mail_file, mail_drafts) +end + +monitor_dir(mail_inbox, mail_drafts) -- 2.23.0

3 4

[PATCH compass-ci] code-spec: refactor friendly principles
by Wu Fengguang 10 Oct '20

10 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/code-spec.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index 9805182..a859239 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -7,6 +7,23 @@ code注意事项 - 一个函数, 不要超过10行代码 - 起准确和有意义的变量/函数名字, 名字要让人看出赋予他们的确切意义 +重构友好编码 +============ + +基本原则: +- 预防自己的代码造成他人的几大困难 + - 难以理解不敢下手 + - 不知道怎么找到*所有*关联逻辑 + - 不知道自己不知道 (unknown unknown) +- 设想将来做重构, 或者需要理解任何一个高层次的功能/流程, + 一个新人应当可以通过 git grep 一两个关键字或者pattern, + 就能找到所有相关底层代码与文档, 而不必担心有所遗漏. + +基本方法: +- 隐式场景显式化: 加注释明确代码的各类隐含目的, 意义, 场景, 数据示例等等 +- 隐式依赖显式化: 定义相同的ID, 把所有的关联文档与各处代码串起来 +- 分散依赖集中化: 定义类/函数/assert, 封装/强制各类设计中的假设, 约定与规则 + PATCH注意事项 ============= -- 2.23.0

1 0

[PATCH compass-ci] src: change vm-hi1620 to vm-aarch64
by Bai Jing 10 Oct '20

10 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- src/delimiter/constants.rb | 2 +- src/features/jobs/right_iperf.yaml | 2 +- src/features/scheduler.feature | 2 +- src/features/step_definitions/scheduler_step.rb | 2 +- src/lib/web_backend.rb | 4 ++-- src/scheduler/README.md | 2 +- src/taskqueue/queue.cr | 2 +- src/taskqueue/redis_client.cr | 2 +- 8 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index 7a17b36..70196c1 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -9,4 +9,4 @@ MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" -DELIMITER_TBOX_GROUP = 'vm-hi1620-2p8g--delimiter' +DELIMITER_TBOX_GROUP = 'vm-aarch64-2p8g--delimiter' diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index 843564d..60654d9 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -3,7 +3,7 @@ #! jobs/iperf.yaml suite: iperf testcase: iperf -testbox: vm-hi1620-2p8g-chief +testbox: vm-aarch64-2p8g-chief category: benchmark runtime: 30 cluster: cs-localhost diff --git a/src/features/scheduler.feature b/src/features/scheduler.feature index c335fb4..e22ad6a 100644 --- a/src/features/scheduler.feature +++ b/src/features/scheduler.feature @@ -6,7 +6,7 @@ Feature: Scheduler Given prepared a job "right_iperf.yaml" When call with API: post "submit_job" job from add_job.sh Then return with job id - When call with API: put "set_host_mac" "vm-hi1620-2p8g-chief => ef-01-02-03-04-05" + When call with API: put "set_host_mac" "vm-aarch64-2p8g-chief => ef-01-02-03-04-05" And call with API: get "boot.ipxe/mac/ef-01-02-03-04-05" Then return with basic ipxe boot parameter and initrd and kernel diff --git a/src/features/step_definitions/scheduler_step.rb b/src/features/step_definitions/scheduler_step.rb index 882ddd8..cb49837 100644 --- a/src/features/step_definitions/scheduler_step.rb +++ b/src/features/step_definitions/scheduler_step.rb @@ -22,7 +22,7 @@ Then('return with job id') do end # set_host_mac -# host_mac => "vm-hi1620-2p8g-chief => ef-01-02-03-04-05" +# host_mac => "vm-aarch64-2p8g-chief => ef-01-02-03-04-05" Given('call with API: put {string} {string}') do |url, host_mac| host_mac_params = host_mac.split(' ') url_with_params = "#{url}?hostname=#{host_mac_params[0]}\\&mac=#{host_mac_params[2]}" diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 4544690..7271605 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -41,8 +41,8 @@ def es_count(query) ES_CLIENT.count(index: 'jobs*', body: query)['count'] end -# "vm-hi1620-2p8g-212" remove "-212" -# "vm-hi1620-2p8g-zzz" remove "-zzz" +# "vm-aarch64-2p8g-212" remove "-212" +# "vm-aarch64-2p8g-zzz" remove "-zzz" # "vm-git-bisect" don't remove "-bisect" def filter_tbox_group(es_result) result = Set.new diff --git a/src/scheduler/README.md b/src/scheduler/README.md index 5da1bde..66d7e49 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -413,7 +413,7 @@ you can simply call the scheduler's build and run script. vim $CCI_SRC/user-client/helper/iperf.yaml runtime: 30 # unit in seconds, the time can be shortened arch: aarch64 - testbox: vm-hi1620-2p8g-$USER # echo $USER for your own user name + testbox: vm-aarch64-2p8g-$USER # echo $USER for your own user name ## 2.Submit job submit-job $_CCI_SRC/user-client/helper/iperf.yaml diff --git a/src/taskqueue/queue.cr b/src/taskqueue/queue.cr index 02236f5..565af7d 100644 --- a/src/taskqueue/queue.cr +++ b/src/taskqueue/queue.cr @@ -183,7 +183,7 @@ class TaskQueue private def move_uuid_task2ready(queue_name) # get uuid lists - # uuid_keys ["queues/sched/vm-hi1620-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451", ] + # uuid_keys ["queues/sched/vm-aarch64-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451", ] uuid_keys = get_uuid_keys("#{queue_name}") return unless uuid_keys # nil return unless uuid_keys.size > 0 diff --git a/src/taskqueue/redis_client.cr b/src/taskqueue/redis_client.cr index 3bac924..35ee31e 100644 --- a/src/taskqueue/redis_client.cr +++ b/src/taskqueue/redis_client.cr @@ -236,7 +236,7 @@ class TaskQueue private def get_uuid_keys(queue_name) return nil unless queue_name[0..5] == "sched/" - # search = "queues/sched/vm-hi1620-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451" + # search = "queues/sched/vm-aarch64-2p8g/ee44b164-90e3-49a7-9798-5e7cc9bc7451" # only 3 matchs keyword: * [] ? search = "#{QUEUE_NAME_BASE}/#{queue_name}/[0-9a-eA-F\-]*" lua_script = "return redis.call('keys', KEYS[1])" -- 2.23.0

2 1

[PATCH compass-ci] fix: change nfsroot path from /os to /srv/os
by Xu Xijian 10 Oct '20

10 Oct '20

In z9 the path /os is a link to /srv/os, so when we get the realpath, the nfsroot path will be /srv/os instead of /os. Here change the default nfsroot path as /srv/os to adapt it. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- container/update-os-docker/lib/common.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/update-os-docker/lib/common.rb b/container/update-os-docker/lib/common.rb index eb694b1..23c4fdc 100755 --- a/container/update-os-docker/lib/common.rb +++ b/container/update-os-docker/lib/common.rb @@ -6,14 +6,14 @@ require_relative 'dc-image' require 'pathname' -OS_PATH ||= ENV['OS_PATH'] || '/os/' +OS_PATH ||= ENV['OS_PATH'] || '/srv/os/' # Parse the parameters and make use of them class ParseParam attr_reader :dc_name, :local_dc_img def initialize(rootdir) - _, _, @os_name, _, @os_version = rootdir.split('/') + _, _, _, @os_name, _, @os_version = rootdir.split('/') end def prepare_env -- 2.23.0

1 0

[PATCH lab-z9 1/2] add new hosts file
by Zhang Dewan 10 Oct '20

10 Oct '20

--- hosts/taishan200-2280-2s48p-256g--a13 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a14 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a15 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a17 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a18 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a19 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a21 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a22 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a24 | 7 +++---- hosts/taishan200-2280-2s48p-256g--a25 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a26 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a27 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a28 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a29 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a30 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a31 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a32 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a33 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a34 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a52 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a53 | 17 +++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a60 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a61 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a62 | 19 +++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a66 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a67 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a68 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a69 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a70 | 5 ++--- hosts/taishan200-2280-2s48p-256g--a71 | 2 +- hosts/taishan200-2280-2s64p-256g--a10 | 8 ++++++-- hosts/taishan200-2280-2s64p-256g--a36 | 24 ++++++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a40 | 21 +++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a41 | 21 +++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a42 | 21 +++++++++++++++++++++ hosts/taishan200-2280-2s64p-256g--a9 | 5 ++--- 36 files changed, 527 insertions(+), 28 deletions(-) create mode 100644 hosts/taishan200-2280-2s48p-256g--a13 create mode 100644 hosts/taishan200-2280-2s48p-256g--a14 create mode 100644 hosts/taishan200-2280-2s48p-256g--a15 create mode 100644 hosts/taishan200-2280-2s48p-256g--a17 create mode 100644 hosts/taishan200-2280-2s48p-256g--a18 create mode 100644 hosts/taishan200-2280-2s48p-256g--a19 create mode 100644 hosts/taishan200-2280-2s48p-256g--a21 create mode 100644 hosts/taishan200-2280-2s48p-256g--a22 create mode 100644 hosts/taishan200-2280-2s48p-256g--a26 create mode 100644 hosts/taishan200-2280-2s48p-256g--a27 create mode 100644 hosts/taishan200-2280-2s48p-256g--a28 create mode 100644 hosts/taishan200-2280-2s48p-256g--a29 create mode 100644 hosts/taishan200-2280-2s48p-256g--a30 create mode 100644 hosts/taishan200-2280-2s48p-256g--a31 create mode 100644 hosts/taishan200-2280-2s48p-256g--a32 create mode 100644 hosts/taishan200-2280-2s48p-256g--a33 create mode 100644 hosts/taishan200-2280-2s48p-256g--a34 create mode 100644 hosts/taishan200-2280-2s48p-256g--a52 create mode 100644 hosts/taishan200-2280-2s48p-256g--a53 create mode 100644 hosts/taishan200-2280-2s48p-256g--a60 create mode 100644 hosts/taishan200-2280-2s48p-256g--a61 create mode 100644 hosts/taishan200-2280-2s48p-256g--a62 create mode 100644 hosts/taishan200-2280-2s64p-256g--a36 create mode 100644 hosts/taishan200-2280-2s64p-256g--a40 create mode 100644 hosts/taishan200-2280-2s64p-256g--a41 create mode 100644 hosts/taishan200-2280-2s64p-256g--a42 diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 new file mode 100644 index 0000000..1c8f554 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500c2799bb3 + - /dev/disk/by-id/scsi-35000c500c278b18b + - /dev/disk/by-id/scsi-35000c500c279f96b + - /dev/disk/by-id/scsi-35000c500c2789b4b +mac_addr: + - 44:67:47:97:2b:bf + - 44:67:47:97:2b:c0 + - 44:67:47:97:2b:c1 + - 44:67:47:97:2b:c2 + - 44:67:47:97:2b:c3 + - 44:67:47:97:2b:c4 + - 44:67:47:97:2b:c5 + - 44:67:47:97:2b:c6 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 new file mode 100644 index 0000000..ae84bd5 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88b4475 + - /dev/disk/by-id/scsi-350000399b88b4371 + - /dev/disk/by-id/scsi-350000399b88b432d + - /dev/disk/by-id/scsi-350000399b88b4531 +mac_addr: + - 44:67:47:c9:e9:c2 + - 44:67:47:c9:e9:c3 + - 44:67:47:c9:e9:c4 + - 44:67:47:c9:e9:c5 + - 44:67:47:c9:e9:c6 + - 44:67:47:c9:e9:c7 + - 44:67:47:c9:e9:c8 + - 44:67:47:c9:e9:c9 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 new file mode 100644 index 0000000..7fa0b67 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b841fff5 + - /dev/disk/by-id/scsi-350000399b841fcb9 + - /dev/disk/by-id/scsi-350000399b8420a0d + - /dev/disk/by-id/scsi-350000399b841ff55 +mac_addr: + - 44:67:47:85:d4:9e + - 44:67:47:85:d4:9f + - 44:67:47:85:d4:a0 + - 44:67:47:85:d4:a1 + - 44:67:47:85:d4:a2 + - 44:67:47:85:d4:a3 + - 44:67:47:85:d4:a4 + - 44:67:47:85:d4:a5 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 new file mode 100644 index 0000000..8be93de --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88b44cd + - /dev/disk/by-id/scsi-350000399b88b4541 + - /dev/disk/by-id/scsi-350000399b88b4495 + - /dev/disk/by-id/scsi-350000399b88b48d1 +mac_addr: + - 44:67:47:c9:ea:1c + - 44:67:47:c9:ea:1d + - 44:67:47:c9:ea:1e + - 44:67:47:c9:ea:1f + - 44:67:47:c9:ea:20 + - 44:67:47:c9:ea:21 + - 44:67:47:c9:ea:22 + - 44:67:47:c9:ea:23 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 new file mode 100644 index 0000000..e86bdeb --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b891b4d1 + - /dev/disk/by-id/scsi-350000399b8890d59 + - /dev/disk/by-id/scsi-350000399b8890f65 + - /dev/disk/by-id/scsi-350000399b891b3c5 +mac_addr: + - 84:46:fe:3d:a7:e9 + - 84:46:fe:3d:a7:ea + - 84:46:fe:3d:a7:eb + - 84:46:fe:3d:a7:ec + - 84:46:fe:3d:a7:ed + - 84:46:fe:3d:a7:ee + - 84:46:fe:3d:a7:ef + - 84:46:fe:3d:a7:f0 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 new file mode 100644 index 0000000..0c4a60f --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8923939 + - /dev/disk/by-id/scsi-350000399b8420b2d + - /dev/disk/by-id/scsi-350000399b891b5c5 + - /dev/disk/by-id/scsi-350000399b841f769 +mac_addr: + - 44:67:47:85:d5:48 + - 44:67:47:85:d5:49 + - 44:67:47:85:d5:4a + - 44:67:47:85:d5:4b + - 44:67:47:85:d5:4c + - 44:67:47:85:d5:4d + - 44:67:47:85:d5:4e + - 44:67:47:85:d5:4f +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 new file mode 100644 index 0000000..51b7709 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bf2d5 + - /dev/disk/by-id/scsi-350000399b88bf5a5 + - /dev/disk/by-id/scsi-350000399b88bf279 + - /dev/disk/by-id/scsi-350000399b88bf24d +mac_addr: + - 44:67:47:86:1b:e0 + - 44:67:47:86:1b:e1 + - 44:67:47:86:1b:e2 + - 44:67:47:86:1b:e3 + - 44:67:47:86:1b:e4 + - 44:67:47:86:1b:e5 + - 44:67:47:86:1b:e6 + - 44:67:47:86:1b:e7 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 new file mode 100644 index 0000000..4de0ad0 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8919069 + - /dev/disk/by-id/scsi-350000399b8918c15 + - /dev/disk/by-id/scsi-350000399b8918c79 + - /dev/disk/by-id/scsi-350000399b8918cb9 +mac_addr: + - 44:67:47:d7:6d:14 + - 44:67:47:d7:6d:15 + - 44:67:47:d7:6d:16 + - 44:67:47:d7:6d:17 + - 44:67:47:d7:6d:18 + - 44:67:47:d7:6d:19 + - 44:67:47:d7:6d:1a + - 44:67:47:d7:6d:1b +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index f46bfab..f47947c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8917d59 - - /dev/disk/by-id/scsi-350000399b891790d - /dev/disk/by-id/scsi-350000399b89185d9 -rootfs_partition: + - /dev/disk/by-id/scsi-350000399b891790d - /dev/disk/by-id/scsi-350000399b8917ef1 mac_addr: - 44:67:47:c9:d8:f4 diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index ee5eeb3..2f270e8 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4499 - /dev/disk/by-id/scsi-350000399b88b4455 - /dev/disk/by-id/scsi-350000399b88b442d -rootfs_partition: - /dev/disk/by-id/scsi-350000399b88b44f9 mac_addr: - 44:67:47:c9:ea:08 diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 new file mode 100644 index 0000000..b6de96e --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bf131 + - /dev/disk/by-id/scsi-350000399b88bfd8d + - /dev/disk/by-id/scsi-350000399b88bfe55 + - /dev/disk/by-id/scsi-350000399b88bef31 +mac_addr: + - 44:67:47:c9:e9:40 + - 44:67:47:c9:e9:41 + - 44:67:47:c9:e9:42 + - 44:67:47:c9:e9:43 + - 44:67:47:c9:e9:44 + - 44:67:47:c9:e9:45 + - 44:67:47:c9:e9:46 + - 44:67:47:c9:e9:47 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 new file mode 100644 index 0000000..b6d9945 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8918ea9 + - /dev/disk/by-id/scsi-350000399b8918bad + - /dev/disk/by-id/scsi-350000399b8918d05 + - /dev/disk/by-id/scsi-350000399b8919415 +mac_addr: + - 44:67:47:d7:6d:3c + - 44:67:47:d7:6d:3d + - 44:67:47:d7:6d:3e + - 44:67:47:d7:6d:3f + - 44:67:47:d7:6d:40 + - 44:67:47:d7:6d:41 + - 44:67:47:d7:6d:42 + - 44:67:47:d7:6d:43 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 new file mode 100644 index 0000000..ecebafc --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500cdf1eb07 + - /dev/disk/by-id/scsi-35000c500cdef4c6f + - /dev/disk/by-id/scsi-35000c500cdefac2b + - /dev/disk/by-id/scsi-35000c500cdefa75f +mac_addr: + - 5c:e8:83:31:c8:20 + - 5c:e8:83:31:c8:21 + - 5c:e8:83:31:c8:22 + - 5c:e8:83:31:c8:23 + - 5c:e8:83:31:c8:24 + - 5c:e8:83:31:c8:25 + - 5c:e8:83:31:c8:26 + - 5c:e8:83:31:c8:27 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 new file mode 100644 index 0000000..bdb13ca --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8918425 + - /dev/disk/by-id/scsi-350000399b891841d + - /dev/disk/by-id/scsi-350000399b8917859 + - /dev/disk/by-id/scsi-350000399b8917a51 +mac_addr: + - 44:67:47:c9:d8:ea + - 44:67:47:c9:d8:eb + - 44:67:47:c9:d8:ec + - 44:67:47:c9:d8:ed + - 44:67:47:c9:d8:ee + - 44:67:47:c9:d8:ef + - 44:67:47:c9:d8:f0 + - 44:67:47:c9:d8:f1 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 new file mode 100644 index 0000000..810859b --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b883b881 + - /dev/disk/by-id/scsi-350000399b88bf83d + - /dev/disk/by-id/scsi-350000399b883bbbd + - /dev/disk/by-id/scsi-350000399b88bf48d +mac_addr: + - 44:67:47:86:1b:cc + - 44:67:47:86:1b:cd + - 44:67:47:86:1b:ce + - 44:67:47:86:1b:cf + - 44:67:47:86:1b:d0 + - 44:67:47:86:1b:d1 + - 44:67:47:86:1b:d2 + - 44:67:47:86:1b:d3 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 new file mode 100644 index 0000000..8f9d13b --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b883b929 + - /dev/disk/by-id/scsi-350000399b883bbe1 + - /dev/disk/by-id/scsi-350000399b883c059 + - /dev/disk/by-id/scsi-350000399b883bc9d +mac_addr: + - 44:67:47:86:1b:ae + - 44:67:47:86:1b:af + - 44:67:47:86:1b:b0 + - 44:67:47:86:1b:b1 + - 44:67:47:86:1b:b2 + - 44:67:47:86:1b:b3 + - 44:67:47:86:1b:b4 + - 44:67:47:86:1b:b5 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 new file mode 100644 index 0000000..66d9b68 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bf141 + - /dev/disk/by-id/scsi-350000399b88bee25 + - /dev/disk/by-id/scsi-350000399b88bea59 + - /dev/disk/by-id/scsi-350000399b88be1d9 +mac_addr: + - 44:67:47:c9:db:38 + - 44:67:47:c9:db:39 + - 44:67:47:c9:db:3a + - 44:67:47:c9:db:3b + - 44:67:47:c9:db:3c + - 44:67:47:c9:db:3d + - 44:67:47:c9:db:3e + - 44:67:47:c9:db:3f +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 new file mode 100644 index 0000000..ef3b148 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8919555 + - /dev/disk/by-id/scsi-350000399b8918c4d + - /dev/disk/by-id/scsi-350000399b8919c3d + - /dev/disk/by-id/scsi-350000399b8919389 +mac_addr: + - 44:67:47:d7:6c:c4 + - 44:67:47:d7:6c:c5 + - 44:67:47:d7:6c:c6 + - 44:67:47:d7:6c:c7 + - 44:67:47:d7:6c:c8 + - 44:67:47:d7:6c:c9 + - 44:67:47:d7:6c:ca + - 44:67:47:d7:6c:cb +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 new file mode 100644 index 0000000..528ec82 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88be219 + - /dev/disk/by-id/scsi-350000399b88bdf71 + - /dev/disk/by-id/scsi-350000399b88bdb49 + - /dev/disk/by-id/scsi-350000399b891784d +mac_addr: + - 44:67:47:c9:d9:26 + - 44:67:47:c9:d9:27 + - 44:67:47:c9:d9:28 + - 44:67:47:c9:d9:29 + - 44:67:47:c9:d9:2a + - 44:67:47:c9:d9:2b + - 44:67:47:c9:d9:2c + - 44:67:47:c9:d9:2d +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 new file mode 100644 index 0000000..4353d4c --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500cdefd89b + - /dev/disk/by-id/scsi-35000c500cdefac67 + - /dev/disk/by-id/scsi-35000c500cdedee6f + - /dev/disk/by-id/scsi-35000c500cdedfdaf +mac_addr: + - 5c:e8:83:31:c8:d4 + - 5c:e8:83:31:c8:d5 + - 5c:e8:83:31:c8:d6 + - 5c:e8:83:31:c8:d7 + - 5c:e8:83:31:c8:d8 + - 5c:e8:83:31:c8:d9 + - 5c:e8:83:31:c8:da + - 5c:e8:83:31:c8:db +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 new file mode 100644 index 0000000..0df1cd9 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -0,0 +1,17 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500bd6aa4e3 + - /dev/disk/by-id/scsi-35000c500bd681acf +mac_addr: + - 5c:e8:83:73:25:c9 + - 5c:e8:83:73:25:ca + - 5c:e8:83:73:25:cb + - 5c:e8:83:73:25:cc + - 5c:e8:83:73:25:cd + - 5c:e8:83:73:25:ce + - 5c:e8:83:73:25:cf + - 5c:e8:83:73:25:d0 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 new file mode 100644 index 0000000..6986a8f --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500bd67640b + - /dev/disk/by-id/scsi-35000c500bd6707a7 + - /dev/disk/by-id/scsi-35000c500bd680dc3 + - /dev/disk/by-id/scsi-35000c500bd683a73 +mac_addr: + - e0:00:84:2b:51:7d + - e0:00:84:2b:51:7e + - e0:00:84:2b:51:7f + - e0:00:84:2b:51:80 + - e0:00:84:2b:51:81 + - e0:00:84:2b:51:82 + - e0:00:84:2b:51:83 + - e0:00:84:2b:51:84 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 new file mode 100644 index 0000000..1c198b8 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399886a7869 + - /dev/disk/by-id/scsi-350000399886acc5d + - /dev/disk/by-id/scsi-350000399886b6861 + - /dev/disk/by-id/scsi-350000399886ac70d +mac_addr: + - 44:67:47:c9:e4:b8 + - 44:67:47:c9:e4:b9 + - 44:67:47:c9:e4:ba + - 44:67:47:c9:e4:bb + - 44:67:47:c9:e4:bc + - 44:67:47:c9:e4:bd + - 44:67:47:c9:e4:be + - 44:67:47:c9:e4:bf +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 new file mode 100644 index 0000000..26ab646 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -0,0 +1,19 @@ +nr_node: 4 +nr_cpu: 96 +memory: 252G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b892d131 + - /dev/disk/by-id/scsi-350000399b892d619 + - /dev/disk/by-id/scsi-350000399b892dc1d + - /dev/disk/by-id/scsi-350000399b892dbb5 +mac_addr: + - 44:67:47:c9:e3:3a + - 44:67:47:c9:e3:3b + - 44:67:47:c9:e3:3c + - 44:67:47:c9:e3:3d + - 44:67:47:c9:e3:3e + - 44:67:47:c9:e3:3f + - 44:67:47:c9:e3:40 + - 44:67:47:c9:e3:41 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index 881f2d6..c7b2083 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000c5009a671773 - /dev/disk/by-id/scsi-35000c5009a690c53 - /dev/disk/by-id/scsi-35000c5009a6713fb -rootfs_partition: - /dev/disk/by-id/scsi-35000c5009a65963f mac_addr: - 5c:e8:83:31:c8:8e diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index d23d755..f207003 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88aed21 - /dev/disk/by-id/scsi-350000399b88af325 - /dev/disk/by-id/scsi-350000399b88af699 -rootfs_partition: - /dev/disk/by-id/scsi-350000399b88af421 mac_addr: - 44:67:47:c9:e2:2c diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index 0ceb191..054105c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8909f81 - /dev/disk/by-id/scsi-350000399b8909f79 - /dev/disk/by-id/scsi-350000399b8909605 -rootfs_partition: - /dev/disk/by-id/scsi-350000399b8909b85 mac_addr: - 84:46:fe:89:b6:e6 diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index 5e948ee..e96e34c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be92d - /dev/disk/by-id/scsi-350000399b88befc5 - /dev/disk/by-id/scsi-350000399b88be701 -rootfs_partition: - /dev/disk/by-id/scsi-350000399b88bea5d mac_addr: - 44:67:47:c9:d9:4e diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 760ce0c..61f99a3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -1,12 +1,11 @@ nr_node: 4 nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 3 +memory: 252G +nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b890931d - /dev/disk/by-id/scsi-350000399b8909d79 - /dev/disk/by-id/scsi-350000399b8909ac5 -rootfs_partition: - /dev/disk/by-id/scsi-350000399b8909c99 mac_addr: - 84:46:fe:89:b7:22 diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 index dde6205..4896475 100644 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ b/hosts/taishan200-2280-2s48p-256g--a71 @@ -1,6 +1,6 @@ nr_node: 4 nr_cpu: 96 -memory: 256G +memory: 252G mac_addr: - 44:67:47:d7:6d:e6 - 44:67:47:d7:6d:e7 diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index 15a9315..a66bff0 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -1,7 +1,11 @@ nr_node: 4 nr_cpu: 128 -memory: 256G -rootfs_partition: +memory: 252G +nr_hdd_partitions: 1 +nr_ssd_partitions: 1 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399c8030fd5 +ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB44333 mac_addr: - 44:67:47:86:2c:95 diff --git a/hosts/taishan200-2280-2s64p-256g--a36 b/hosts/taishan200-2280-2s64p-256g--a36 new file mode 100644 index 0000000..d9f1184 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a36 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 128 +memory: 255G +nr_hdd_partitions: 1 +nr_ssd_partitions: 1 +hdd_partitions: + - /dev/disk/by-id/scsi-3500003999813487d +ssd_partitions: + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB42949-part1 + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB42949-part2 +mac_addr: + - 84:46:fe:73:b2:75 + - 84:46:fe:73:b2:76 + - 84:46:fe:73:b2:77 + - 84:46:fe:73:b2:78 + - ac:b3:b5:1e:bd:5b + - ac:b3:b5:1e:bd:5c + - ac:b3:b5:1e:bd:5d + - ac:b3:b5:1e:bd:5e + - ac:b3:b5:1e:bd:bb + - ac:b3:b5:1e:bd:bc + - ac:b3:b5:1e:bd:bd + - ac:b3:b5:1e:bd:be +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 new file mode 100644 index 0000000..91f5619 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -0,0 +1,21 @@ +nr_node: 4 +nr_cpu: 128 +memory: 252G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-3500003999812ed81 + - /dev/disk/by-id/scsi-350000399c8020115 +mac_addr: + - 84:46:fe:89:be:8e + - 84:46:fe:89:be:8f + - 84:46:fe:89:be:90 + - 84:46:fe:89:be:91 + - ac:b3:b5:1e:d7:3b + - ac:b3:b5:1e:d7:3c + - ac:b3:b5:1e:d7:3d + - ac:b3:b5:1e:d7:3e + - ac:b3:b5:1e:d6:cb + - ac:b3:b5:1e:d6:cc + - ac:b3:b5:1e:d6:cd + - ac:b3:b5:1e:d6:ce +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 new file mode 100644 index 0000000..9525cfd --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -0,0 +1,21 @@ +nr_node: 4 +nr_cpu: 128 +memory: 252G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8919495 + - /dev/disk/by-id/scsi-35000039998130a85 +mac_addr: + - 84:46:fe:89:bc:4a + - 84:46:fe:89:bc:4b + - 84:46:fe:89:bc:4c + - 84:46:fe:89:bc:4d + - ac:b3:b5:1e:b6:db + - ac:b3:b5:1e:b6:dc + - ac:b3:b5:1e:b6:dd + - ac:b3:b5:1e:b6:de + - ac:b3:b5:1e:b7:5b + - ac:b3:b5:1e:b7:5c + - ac:b3:b5:1e:b7:5d + - ac:b3:b5:1e:b7:5e +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 new file mode 100644 index 0000000..40c1c4b --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -0,0 +1,21 @@ +nr_node: 4 +nr_cpu: 128 +memory: 252G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8918a11 + - /dev/disk/by-id/scsi-350000399c802ee75 +mac_addr: + - cc:05:77:fe:d8:64 + - cc:05:77:fe:d8:65 + - cc:05:77:fe:d8:66 + - cc:05:77:fe:d8:67 + - 44:a1:91:35:6c:50 + - 44:a1:91:35:6c:51 + - 44:a1:91:35:6c:52 + - 44:a1:91:35:6c:53 + - 44:a1:91:35:68:d0 + - 44:a1:91:35:68:d1 + - 44:a1:91:35:68:d2 + - 44:a1:91:35:68:d3 +model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 index d6c4353..e360e88 100644 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ b/hosts/taishan200-2280-2s64p-256g--a9 @@ -1,10 +1,9 @@ nr_node: 4 nr_cpu: 128 -memory: 128G -nr_hdd_partitions: 1 +memory: 126G +nr_hdd_partitions: 2 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6a1347 -rootfs_partition: - /dev/disk/by-id/scsi-35000c500bd62acb3 mac_addr: - cc:05:77:fe:d4:0e -- 2.23.0

2 1

[PATCH v2 compass-ci] sbin/result2stats add help option and message
by Lu Weitao 10 Oct '20

10 Oct '20

background: follow doc/code_spec.md, command line tool should support help output Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/result2stats | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/sbin/result2stats b/sbin/result2stats index 85d68a7..d24cc7b 100755 --- a/sbin/result2stats +++ b/sbin/result2stats @@ -5,6 +5,7 @@ CCI_SRC = File.dirname(__dir__) +require 'optparse' require "#{CCI_SRC}/lib/matrix2" File.umask 0o002 @@ -16,6 +17,26 @@ def extract_stats(result_root) system job_script, 'extract_stats' end -result_root = ARGV[0] || ENV['RESULT_ROOT'] +opt_parser = OptionParser.new do |opts| + opts.banner = 'Usage: result2stats result_root' + opts.separator 'eg: result2stats /srv/result/iperf/crystal.69948' + opts.separator '' + opts.separator 'options:' + + opts.on_tail('-h', '--help', 'show this message') do + puts opts + exit + end +end + +argv = if ARGV == [] + ['-h'] + else + ARGV + end + +opt_parser.parse!(argv) + +result_root = argv[0] || ENV['RESULT_ROOT'] extract_stats result_root create_stats result_root -- 2.23.0

1 0

2024

2023

2022

2021

2020

Compass-ci October 2020