- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 3/3] container/assign-account: answerback-email.rb
by Luan Shengde 03 Nov '20

03 Nov '20

1. disable use golbal variable apply_info [why] Style/GlobalVars: Do not introduce global variables. [how] use local variables instead of global variables 2. enable transfer user info when execute apply account command [why] when applying account, the assign-account service will write the my info to user's default config file ~/.config/compass-ci/defaults/account.yaml my info: - my_email - my_name - my_uuid [how] transfer the user info along with the pub_key to assign-account service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/answerback-email.rb | 62 ++++++++++++-------- 1 file changed, 38 insertions(+), 24 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index bb8e809..c174302 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -12,6 +12,7 @@ require 'mail' require 'set' require 'optparse' require_relative '../defconfig' +require_relative '../../lib/es_client' names = Set.new %w[ JUMPER_HOST @@ -24,23 +25,25 @@ defaults = relevant_defaults(names) JUMPER_HOST = defaults['JUMPER_HOST'] || 'api.compass-ci.openeuler.org' JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 -SEND_MAIL_HOST = defaults['SEND_MAIL_HOST_INTERNET'] || 'localhost' -SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] || 11312 +SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 -$apply_info = { +apply_info = { 'my_email' => nil, + 'my_name' => nil, + 'my_uuid' => %x(uuidgen).chomp, 'my_ssh_pubkey' => nil } -def init_info(email_file) +def init_info(email_file, apply_info) mail_content = Mail.read(email_file) + apply_info['my_email'] = mail_content.from[0] + apply_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') + apply_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' + mail_content.part[1].body.decoded + end - $apply_info['my_email'] = mail_content.from[0] - $apply_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded.gsub(/\r|\n/, '') - end - - $apply_info + apply_info end options = OptionParser.new do |opts| @@ -52,15 +55,17 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e|--email email_address', 'appoint email address') do |email_address| - $apply_info['my_email'] = email_address + apply_info['my_email'] = email_address + # when apply account with email address, will get no user name + apply_info['my_name'] = '' end opts.on('-s|--ssh-pubkey pub_key_file', 'ssh pub_key file, enable password-less login') do |pub_key_file| - $apply_info['my_ssh_pubkey'] = File.read(pub_key_file) + apply_info['my_ssh_pubkey'] = File.read(pub_key_file) end opts.on('-f|--raw-email email_file', 'email file') do |email_file| - init_info(email_file) + init_info(email_file, apply_info) end opts.on_tail('-h|--help', 'show this message') do @@ -94,26 +99,35 @@ def build_message(email, acct_infos) return message end -def account_info(pub_key) - account_info_str = if pub_key.nil? - %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account') - else - %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d "pub_key: #{pub_key}") - end +def account_info(user_info) + account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{user_info.to_json}') JSON.parse account_info_str end -def send_account +def send_account(apply_info) message = "No email address specified\n" message += "use -e email_address add a email address\n" message += 'or use -f to add a email file' - raise message if $apply_info['my_email'].nil? + raise message if apply_info['my_email'].nil? - acct_info = account_info($apply_info['my_ssh_pubkey']) + acct_info = account_info(apply_info) + my_info = { + 'my_email' => apply_info['my_email'], + 'my_name' => apply_info['my_name'], + 'my_commit_url' => '', + 'my_login_name' => acct_info['account'], + 'my_uuid' => apply_info['my_uuid'] + } - message = build_message($apply_info['my_email'], acct_info) + store_account_info(my_info) + message = build_message(apply_info['my_email'], acct_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end -send_account +def store_account_info(my_info) + es = ESClient.new(index: 'accounts') + es.put_source_by_id(my_info['my_email'], my_info) +end + +send_account(apply_info) -- 2.23.0

2 2

[PATCH v2 compass-ci 2/3] container/assign-account: get_account_info
by Luan Shengde 03 Nov '20

03 Nov '20

add new function to config default yaml file [why]: easier for user to config the default yaml file [how]: when get request data, parse the data and write the user infos to default yaml file: ~/.config/compass-ci/default/account.yaml include: - my_email - my_name - my_uuid Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/get_account_info.rb | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 2f93d5b..ae55dcb 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -44,6 +44,8 @@ the returned data for setup_jumper_account_info like: =end +require 'fileutils' + # get jumper and account info class AccountStorage ACCOUNT_DIR = '/opt/account_data/' @@ -93,7 +95,7 @@ class AccountStorage def setup_jumper_account_info account_info = read_account_info jumper_info = read_jumper_info - pub_key = @data['pub_key'] unless @data.nil? + pub_key = @data['my_ssh_pubkey'] unless @data['my_ssh_pubkey'].nil? jumper_ip = jumper_info[0].chomp jumper_port = jumper_info[1].chomp @@ -111,9 +113,23 @@ class AccountStorage } setup_authorized_key(account, pub_key) + setup_default_yaml(account) + return jumper_account_info end + def setup_default_yaml(account) + default_yaml_dir = File.join('/home', account, '.config/compass-ci/defaults') + FileUtils.mkdir_p default_yaml_dir + + File.open("#{default_yaml_dir}/#{account}.yaml", 'a') do |file| + file.puts "my_email: #{@data['my_email']}" + file.puts "my_name: #{@data['my_name']}" + file.puts "my_uuid: #{@data['my_uuid']}" + end + %x(chown -R #{account}:#{account} "/home/#{account}/.config") + end + def setup_authorized_key(account, pub_key) ssh_dir = File.join('/home/', account, '.ssh') Dir.mkdir ssh_dir, 0o700 -- 2.23.0

2 2

[PATCH v3 compass-ci] lib/job.cr: fix pp dir layout
by Wang Yong 03 Nov '20

03 Nov '20

[Why] due to cci-makepkg and cci-depends dir layout change, pp need update together Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- src/lib/job.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..1e49da6 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -405,8 +405,8 @@ class Job program = $1 end - deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}.cgz" - pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}.cgz" + deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}/#{program}.cgz" + pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}/latest.cgz" if File.exists?(deps_dest_file) initrd_deps_arr << "#{initrd_http_prefix}" + JobHelper.service_path(deps_dest_file) -- 2.23.0

1 0

[PATCH v3 lkp-tests 2/2] tests/*: fix cci job dir layout
by Wang Yong 03 Nov '20

03 Nov '20

[Why] for multi version support, cgz file for cci-makepkg need change dir layout, and change cci-depends dir layout together to easily update scheduler code path: /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/ => /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/${pkgname}/ filename: cci-makepkg ${pkgver}-${pkgrel}.cgz latest.cgz -> ${pkgver}-${pkgrel}.cgz filename: cci-depends stay original ${benchmark}.cgz ${benchmark}.cgz -> ${benchmark}_${date}.cgz E.g. output: cci-depends stay original $ tree iperf ├── iperf_20201102.cgz └── iperf.cgz -> iperf_20201102.cgz output: cci-makepkg $ tree sar ├── git-1.cgz └── latest.cgz -> git-1.cgz Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/cci-depends | 2 +- tests/cci-makepkg | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/cci-depends b/tests/cci-depends index 36923ea4d..efa9a025b 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -18,7 +18,7 @@ DEPS_MNT=/initrd/deps && mkdir -p "$DEPS_MNT" DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} pack_arch=$os_arch [[ "$benchmark" = "${benchmark##*.}" ]] || diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 9d5082629..9551ebfe2 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -26,7 +26,7 @@ mkdir -p "$PKG_MNT" export DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} . $LKP_SRC/distro/${DISTRO} . $LKP_SRC/lib/install.sh @@ -65,8 +65,8 @@ update_shared_pkg() [ "$bm_name" = "$benchmark" ] && return # benchmark is a symlink - ln -sf "$bm_link" "$sync_dest/${benchmark}.cgz" || return - echo "update shared pkg link ${benchmark}.cgz -> $bm_link" + ln -sf latest.cgz "$sync_dest/${benchmark}.cgz" || return + echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } distro_install_depends lkp-dev @@ -86,7 +86,7 @@ date=$(date +"%Y%m%d") pkgver=$(get_pkg_info pkgver) pkgrel=$(get_pkg_info pkgrel) bm_name=$(check_shared_pkg) -cgz_name="${bm_name}-${pkgver:-0}-${pkgrel:-0}_${date}.cgz" +cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy @@ -98,8 +98,8 @@ update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return - ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/${bm_name}.cgz" || return - echo "create package: $sync_dest/${bm_name}.cgz -> $(realpath $sync_dest/$cgz_name)" + ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/latest.cgz" || return + echo "create package: $sync_dest/latest.cgz -> $(realpath $sync_dest/$cgz_name)" update_shared_pkg "${bm_name}.cgz" -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/2] fix(distro/depends): add depends file
by Wang Yong 03 Nov '20

03 Nov '20

[Why] there are pack-deps/makepkg depends file before and use nfs-common, now we use cci-depends/cci-makepkg file and use cifs-utils. Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/cci-depends | 2 ++ distro/depends/cci-makepkg | 10 ++++++++++ 2 files changed, 12 insertions(+) create mode 100644 distro/depends/cci-depends create mode 100644 distro/depends/cci-makepkg diff --git a/distro/depends/cci-depends b/distro/depends/cci-depends new file mode 100644 index 000000000..1f7f73ac3 --- /dev/null +++ b/distro/depends/cci-depends @@ -0,0 +1,2 @@ +cifs-utils +cpio diff --git a/distro/depends/cci-makepkg b/distro/depends/cci-makepkg new file mode 100644 index 000000000..a7ee67537 --- /dev/null +++ b/distro/depends/cci-makepkg @@ -0,0 +1,10 @@ +curl +libarchive-tools +bzip2 +fakeroot +gnupg +gettext +openssl +ncurses-bin +binutils +cifs-utils -- 2.23.0

1 0

[PATCH compass-ci] sparrow/7-systemd: fix failed to resume dnsmasq and es service
by Liu Yinsi 03 Nov '20

03 Nov '20

[why] when execute sparrow/4-docker/buildall reboot, error: ==================== es ======================= [2020-11-02T10:24:23,046][INFO ][o.e.n.Node ] [] initializing ... [2020-11-02T10:24:23,084][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/srv/es/elasticsearch]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])? at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.4.3.jar:6.4.3] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.4.3.jar:6.4.3] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.4.3.jar:6.4.3] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.4.3.jar:6.4.3] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.4.3.jar:6.4.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.4.3.jar:6.4.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.4.3.jar:6.4.3] =================== dnsmasq =================== dnsmasq: failed to bind DHCP server socket: Address in use because sometimes even if 'docker stop&&rm -f $container', there will still be a legacy process running, causing container to fail to start. [how] kill process before start container Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/7-systemd/cci-network | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 6e59dbb..dfe7cc9 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -12,6 +12,9 @@ $CCI_SRC/sparrow/2-network/iptables $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs +# fix failed to start container dnsmasq and es. +kill $(ps -ef| grep -E "dnsmasq|elasticsearch"| awk '{print $2}' ) 2> /dev/null + # --restart=always option is not absolutely reliable # container need to start its dependences firstly. $CCI_SRC/sparrow/4-docker/buildall reboot -- 2.23.0

2 1

[PATCH v2 compass-ci 1/3] container/assign-account: assign-account.rb
by Luan Shengde 03 Nov '20

03 Nov '20

disable assigning account for user if there is no my_email, my_name, my_uuid [why]: my_email, my_name, my_uuid is required when initialize the default config file [how]: check if the data has keys: my_email, my_name, my_uuid when get apply account request. the apply account will return with 'no my info' if missing any of my_email, my_name or my_uuid. Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/assign-account.rb | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/container/assign-account/assign-account.rb b/container/assign-account/assign-account.rb index e356c18..3afcc9f 100755 --- a/container/assign-account/assign-account.rb +++ b/container/assign-account/assign-account.rb @@ -7,7 +7,7 @@ require 'sinatra' require 'open3' require 'json' require 'yaml' -require_relative 'get_account_info.rb' +require_relative 'get_account_info' set :bind, '0.0.0.0' set :port, 29999 @@ -17,8 +17,12 @@ get '/assign_account' do data = YAML.safe_load request.body.read rescue StandardError => e puts e.message + puts e.backtrace end + message = 'lack of my infos(my_email, my_name, my_uuid)' + raise message unless %w[my_email my_name my_uuid].to_set.subset? data.keys.to_set + ref_account_info = AccountStorage.new(data) account_info = ref_account_info.setup_jumper_account_info -- 2.23.0

3 4

[PATCH compass-ci] sched: hand over the job from job.queue to extract queue
by Cao Xueliang 03 Nov '20

03 Nov '20

[why] The job add to job.queue, when lkp finish the job need hand over the job from job.queue to extract queue. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/sched.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 46b06da..a4b12b4 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -650,7 +650,7 @@ class Sched end response = @task_queue.hand_over_task( - "sched/#{job.tbox_group}", "extract_stats", job_id + "sched/#{job.queue}", "extract_stats", job_id ) if response[0] != 201 raise "#{response}" -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code: add ENV TASKQUEUE_PORT
by Liu Yinsi 03 Nov '20

03 Nov '20

need env TASKQUEUE_PORT to start taskqueue service. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f36aaca..64f34f8 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -18,6 +18,7 @@ mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF SCHED_HOST: $sched_host SCHED_PORT: $sched_port +TASKQUEUE_HOST: $sched_host OS_HTTP_HOST: $os_http_host INITRD_HTTP_HOST: $initrd_http_host INITRD_HTTP_PORT: $initrd_http_port -- 2.23.0

1 0

[PATCH v2 compass-ci] lib/job.cr: fix pp dir layout
by Wang Yong 03 Nov '20

03 Nov '20

[Why] due to cci-makepkg and cci-depends dir layout change, pp need update together Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- src/lib/job.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 955bc4b..4076745 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -434,8 +434,8 @@ class Job program = $1 end - deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}/latest.cgz" - pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}/latest.cgz" + deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}/#{program}.cgz" + pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}/#{program}.cgz" if File.exists?(deps_dest_file) initrd_deps_arr << "#{initrd_http_prefix}" + JobHelper.service_path(deps_dest_file) -- 2.23.0

2 2

[PATCH compass-ci] container: fix failed to start git-mirror
by Liu Yinsi 03 Nov '20

03 Nov '20

[why] when start git-mirror container in machine, error: /usr/lib/ruby/2.5.0/net/http.rb:939:in `rescue in block in connect': Failed to open TCP connection to 172.17.0.1:9200 (Connection refused - connect(2) for "172.17.0.1" port 9200) (Faraday::ConnectionFailed) Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/git-mirror/start-depends | 1 + 1 file changed, 1 insertion(+) diff --git a/container/git-mirror/start-depends b/container/git-mirror/start-depends index 5232abf..38fc696 100755 --- a/container/git-mirror/start-depends +++ b/container/git-mirror/start-depends @@ -1 +1,2 @@ rabbitmq +es -- 2.23.0

1 0

[PATCH compass-ci 1/8] lib/mail-robot.rb
by Luan Shengde 03 Nov '20

03 Nov '20

mail robot for applying account monitor mailbox check new added email file in new email directory handle new email file check to apply account only if email's subject match 'apply account' Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-robot.rb | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100755 lib/mail-robot.rb diff --git a/lib/mail-robot.rb b/lib/mail-robot.rb new file mode 100755 index 0000000..9437078 --- /dev/null +++ b/lib/mail-robot.rb @@ -0,0 +1,43 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + next if added.empty? + + added.each do |mail_file| + begin + handle_new_email(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + puts e.backtrace + end + end + end + listener.start + sleep +end + +def handle_new_email(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + check_to_apply_account(mail_content) + + FileUtils.mv(mail_file, mail_drafts) +end + +def check_to_apply_account(mail_content) + return unless mail_content.subject == 'apply account' + + assign_uuid = AssignAccount.new(mail_content) + assign_uuid.check_to_send_account +end + +monitor_dir("#{ENV['MAILDIR']}/new", "#{ENV['MAILDIR']}/cur") -- 2.23.0

2 1

[PATCH lkp-tests] bin/lkp-setup-rootfs: change the script that called when packaging
by Li Ping 02 Nov '20

02 Nov '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- bin/lkp-setup-rootfs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 2890601e..680027e6 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -187,14 +187,14 @@ define_files_pkg_pack() ( local benchmark=${1#pkg/} export benchmark=${benchmark%/PKGBUILD} - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) } # There may be $program-dev or $program-dev-dev in the # "$LKP_SRC/distro/depends" directory, # when the value passed by $define_files is "distro/depends/$program-dev", -# you don't know which script in "makepkg" or "pack-deps" to execute, +# you don't know which script in "cci-makepkg" or "cci-depends" to execute, # so it needs to be handled sparately. # Save the result to # "/srv/initrd/deps/$os/$os_arch/$os_version/$deps_benchmark.cgz", @@ -208,7 +208,7 @@ define_files_deps_pack() [ $suffix == "dev" ] && [ -f "$LKP_SRC/pkg/$pkg_benchmark/PKGBUILD" ] && { ( export benchmark=$pkg_benchmark - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) return 0 } @@ -216,7 +216,7 @@ define_files_deps_pack() ( export benchmark=$deps_benchmark export pack_arch=$os_arch - $LKP_SRC/tests/pack-deps + $LKP_SRC/tests/cci-depends ) } -- 2.23.0

1 0

[PATCH compass-ci 8/8] doc/mail-robot.md
by Luan Shengde 02 Nov '20

02 Nov '20

documentation for mail-robot service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/mail-robot.md | 76 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 doc/mail-robot.md diff --git a/doc/mail-robot.md b/doc/mail-robot.md new file mode 100644 index 0000000..d8ac188 --- /dev/null +++ b/doc/mail-robot.md @@ -0,0 +1,76 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ + +# email monitor robot + +## purpose + +apply account/uuid for user + uuid: used for user to submit jobs + account: used for user to check the test data + +## allowed email format: + +subject: apply account +mail content + my oss commit: commit_url +attahcment + ssh pub_key + - id_rsa.pub + +## steps overview + +1. monitor + - the monitor will listen to the mailbox for new email files + handle new email file: + check to send account + email subject: apply account + +2. assign account + - check email + check commit url + check whether there is a standard commit url + check whether the repo's url in upstream-repos + check whether the commit url valid + if the hub name is gitee.com + # authentication is needed qwhen use curl to check the url + clone the repo and check the commit + if the hub name is not gitee.com + check the comit with curl + + check pub_key + check whether there is a ssh pub_key in the email file + + it will return failure email if any check end with failure + + - generate uuid + - apply account + build user_info + - my_email + - my_name + - my_uuid + - lab + - pub_key + execute command to apply account on jumper and return the account info + curl -XGET "${JUMPER_HOST}:${JUMPER_PORT}/assign_account" -d '${user_info}' + write my_email, my_name, my_uuid, lab to the defaule config file: + + - store apply infos + apply_infos: + - my_email + - my_name + - my_commit_url + - my_login_name + - my_uuid + invoke es to store the apply infos + + - send uuid + build send uuid email message with the apply_infos + send mail + +3. continue the monitor + continue to monitor the mailbox for new email file + cycle run step 1 and 2 if matched email files + +4. service log + use the following command to get the logs + docker logs -f --tail=100 fluentd |grep mail-robot -- 2.23.0

1 0

[PATCH compass-ci 7/8] container/mail-robot: start
by Luan Shengde 02 Nov '20

02 Nov '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/start | 41 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..671eaf2 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,41 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'mail-robot' + +names = Set.new %w[ + JUMPER_HOST + JUMPER_PORT + SEND_MAIL_PORT + MAILDIR + ROBOT_TMP_REPOS_DIR +] + +defaults = relevant_defaults(names) + +defaults['CCI_SRC'] = '/c/compass-ci' + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + --name=mail-robot + -d +] + env + %W[ + -v /c/upstream-repos:/c/upstream-repos:ro + -v #{defaults['MAILDIR']}:#{defaults['MAILDIR']}:rw + -v #{ENV['CCI_SRC']}:#{defaults['CCI_SRC']}:rw + -w #{defaults['CCI_SRC']}/container/mail-robot + --log-driver json-file + mail-robot +] + +cmd += ['ruby', 'run.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci 6/8] container/mail-robot: build
by Luan Shengde 02 Nov '20

02 Nov '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/mail-robot/build diff --git a/container/mail-robot/build b/container/mail-robot/build new file mode 100755 index 0000000..c564add --- /dev/null +++ b/container/mail-robot/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t mail-robot:latest . -- 2.23.0

1 0

[PATCH compass-ci 5/8] container/mail-robot: Dockerfile
by Luan Shengde 02 Nov '20

02 Nov '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/Dockerfile | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/mail-robot/Dockerfile diff --git a/container/mail-robot/Dockerfile b/container/mail-robot/Dockerfile new file mode 100644 index 0000000..5eea701 --- /dev/null +++ b/container/mail-robot/Dockerfile @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER shdluan(a)163.com + +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y git uuid-runtime curl ruby-listen ruby-json ruby-mail && \ + gem install fileutils elasticsearch activesupport + +RUN mkdir -p /repos_dir -- 2.23.0

1 0

[PATCH compass-ci 4/8] container/mail-robot: run.rb
by Luan Shengde 02 Nov '20

02 Nov '20

invoke apply-account-component, apply-account.rb, mail-rebot.rb to run the service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/run.rb | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100755 container/mail-robot/run.rb diff --git a/container/mail-robot/run.rb b/container/mail-robot/run.rb new file mode 100755 index 0000000..b432d7d --- /dev/null +++ b/container/mail-robot/run.rb @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative 'apply-account-component' +require_relative 'apply-account' +require_relative '../../lib/mail-robot' -- 2.23.0

1 0

[PATCH lkp-tests 2/2] tests/iozone: modify parameter parse process
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool need change parse process. [how] modify parse process Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 50 ++++++++++++++++++++++---------------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..19c246f1 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,36 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + [ "$ele" == "write_rewrite" ] && args+=" -i 0" + [ "$ele" == "read_reread" ] && args+=" -i 1" + [ "$ele" == "random_read_write" ] && args+=" -i 2" + [ "$ele" == "read_backwards" ] && args+=" -i 3" + [ "$ele" == "rewrite_record" ] && args+=" -i 4" + [ "$ele" == "stride_read" ] && args+=" -i 5" + [ "$ele" == "fwrite_refwrite" ] && args+=" -i 6" + [ "$ele" == "fread_refread" ] && args+=" -i 7" + [ "$ele" == "random_mix" ] && args+=" -i 8" + [ "$ele" == "pwrite_repwrite" ] && args+=" -i 9" + [ "$ele" == "pread_repread" ] && args+=" -i 10" + [ "$ele" == "pwritev_repwritev" ] && args+=" -i 11" + [ "$ele" == "preadv_repreadv" ] && args+=" -i 12" + done else args+=" -a" fi -- 2.23.0

1 0

[PATCH lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH lkp-tests 2/2] tests/iozone: modify parameter parse process
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool need change parse process. [how] modify parse process Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 50 ++++++++++++++++++++++---------------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..19c246f1 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,36 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + [ "$ele" == "write_rewrite" ] && args+=" -i 0" + [ "$ele" == "read_reread" ] && args+=" -i 1" + [ "$ele" == "random_read_write" ] && args+=" -i 2" + [ "$ele" == "read_backwards" ] && args+=" -i 3" + [ "$ele" == "rewrite_record" ] && args+=" -i 4" + [ "$ele" == "stride_read" ] && args+=" -i 5" + [ "$ele" == "fwrite_refwrite" ] && args+=" -i 6" + [ "$ele" == "fread_refread" ] && args+=" -i 7" + [ "$ele" == "random_mix" ] && args+=" -i 8" + [ "$ele" == "pwrite_repwrite" ] && args+=" -i 9" + [ "$ele" == "pread_repread" ] && args+=" -i 10" + [ "$ele" == "pwritev_repwritev" ] && args+=" -i 11" + [ "$ele" == "preadv_repreadv" ] && args+=" -i 12" + done else args+=" -a" fi -- 2.23.0

1 0

[PATCH lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH v3 lkp-tests] jobs/hackbench-git.yaml: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 lkp-tests] jobs: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..6494bc8 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file=/var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

3 2

[PATCH v3 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..cdba89c 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file = /var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

1 0

[PATCH v2 compass-ci] container: fix failed to build archlinux images
by Liu Yinsi 02 Nov '20

02 Nov '20

[why] when build archlinux images in x86_64 machine error: error: failed retrieving file 'core.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update core (failed to retrieve some files) error: failed retrieving file 'extra.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirror.archlinuxarm.org : Resolving timed out after 10000 milliseconds error: failed to update extra (download library error) error: failed retrieving file 'community.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update community (failed to retrieve some files) error: failed to synchronize all databases The command '/bin/sh -c pacman --needed --noprogressbar --noconfirm -Syu && pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk' returned a non-zero code: 1 becaue archlinux mirror not support x86_64 machine. [how] when deploy in x86_64 machine, change mirror http://mirrors.tuna.tsinghua.edu.cn/archlinuxarm/$arch/$repo => http://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/archlinux/build | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/archlinux/build b/container/archlinux/build index 81feda2..f96ad40 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ "$(arch)" == "x86_64" ] && + sed -i 's#archlinuxarm/$arch/$repo#archlinux/$repo/os/$arch#g' root/etc/pacman.d/mirrorlist + docker build -t archlinux:testbed . -- 2.23.0

3 4

[PATCH v2 compass-ci] src/lib/web_backend.rb: "/compare_candidates" tbox_group regex error
by Zhang Yuhang 02 Nov '20

02 Nov '20

[error info] 1. /\d+$/ will be matched, such as "xxx123". 2. `index = "xxx123".index('--') || "xxx123".rindex('-')` - index equal nil. 3. `r = r[0, nil]` - raise a TypeError. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index c3f6bef..55c5e4a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -47,7 +47,7 @@ end def filter_tbox_group(es_result) result = Set.new es_result.each do |r| - if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$)/ + if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$)/ index = r.index('--') || r.rindex('-') r = r[0, index] end -- 2.23.0

3 4

[PATCH v6 compass-ci 2/3] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 02 Nov '20

02 Nov '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..8bf7a65 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -87,6 +87,10 @@ class Job kernel_append_root kernel_params docker_image + kernel_version + vmlinuz_path + modules_path + headers_path ) macro method_missing(call) @@ -135,6 +139,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,9 +298,22 @@ class Job return true end + private def set_kernel_version + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + if self["kernel_version"]? + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz-#{kernel_version}" + self["modules_path"] = "#{boot_dir}/modules-#{kernel_version}.cgz" + self["headers_path"] = "#{boot_dir}/headers-#{kernel_version}.cgz" + else + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz" + self["modules_path"] = "#{boot_dir}/modules.cgz" + self["headers_path"] = "#{boot_dir}/headers.cgz" + end + end + private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{vmlinuz_path}")}" end private def common_initrds @@ -316,6 +334,10 @@ class Job "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{modules_path}")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{headers_path}")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 2

[PATCH lkp-tests] jobs/iozone-bs.yaml: fix mapper_parsing_exception
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] when key:value of fs: xfs in same line for .yaml file, submit job will cause some error occurred like below: submit /home/lukaiyi/lkp-tests/jobs/iozone-bs.yaml failed, got job_id=z9.144590, error: [{"type" => "mapper_parsing_exception", "reason" => "failed to parse [pp.fs.fs]"}] [how] change key and value of fs: xfs to different line, same to iosched: kyber Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index 868979fc..e2cd9f48 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -14,6 +14,11 @@ block_size: - 16m disk: 1HDD -fs: xfs -iosched: kyber + +fs: +- xfs + +iosched: +- kyber + iozone: -- 2.23.0

1 0

[PATCH compass-ci] based on queue field add the job to specified queue
by Xiao Shenwei 02 Nov '20

02 Nov '20

add queue field into job.yaml, refer to commit-id 912214cf97c125ed5387dc20ce394ee6db2b11b6 then use queue field replace tbox_group to add the job Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 80cf0b0..b4745aa 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -278,24 +278,24 @@ class Sched # success: [{"job_id" => job_id, "message" => "", job_state => "submit"}] # failure: [{"job_id" => "0", "message" => err_msg, job_state => "submit"}] def submit_single_job(job) - tbox_group = job.tbox_group + queue = job.queue return [{ "job_id" => "0", - "message" => "get tbox group failed", + "message" => "get queue failed", "job_state" => "submit", - }] unless tbox_group + }] unless queue # only single job will has "idle job" and "execute rate limiter" if job["idle_job"].empty? - tbox_group += "#{job.get_uuid_tag}" + queue += "#{job.get_uuid_tag}" else - tbox_group = "#{tbox_group}/idle" + queue = "#{queue}/idle" end - job_id = add_task(tbox_group, job.lab) + job_id = add_task(queue, job.lab) return [{ "job_id" => "0", - "message" => "add task queue sched/#{tbox_group} failed", + "message" => "add task queue sched/#{queue} failed", "job_state" => "submit", }] unless job_id @@ -311,9 +311,9 @@ class Sched end # return job_id - def add_task(tbox_group, lab) + def add_task(queue, lab) task_desc = JSON.parse(%({"domain": "compass-ci", "lab": "#{lab}"})) - response = @task_queue.add_task("sched/#{tbox_group}", task_desc) + response = @task_queue.add_task("sched/#{queue}", task_desc) JSON.parse(response[1].to_json)["id"].to_s if response[0] == 200 end -- 2.23.0

1 0

[PATCH lab-z9] modify hosts file
by Zhang Dewan 02 Nov '20

02 Nov '20

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- hosts/taishan200-2280-2s48p-256g--a101 | 5 +---- hosts/taishan200-2280-2s48p-256g--a11 | 5 +---- hosts/taishan200-2280-2s48p-256g--a13 | 5 +---- hosts/taishan200-2280-2s48p-256g--a14 | 5 +---- hosts/taishan200-2280-2s48p-256g--a15 | 5 +---- hosts/taishan200-2280-2s48p-256g--a17 | 5 +---- hosts/taishan200-2280-2s48p-256g--a18 | 5 +---- hosts/taishan200-2280-2s48p-256g--a19 | 5 +---- hosts/taishan200-2280-2s48p-256g--a21 | 5 +---- hosts/taishan200-2280-2s48p-256g--a22 | 5 +---- hosts/taishan200-2280-2s48p-256g--a24 | 5 +---- hosts/taishan200-2280-2s48p-256g--a25 | 5 +---- hosts/taishan200-2280-2s48p-256g--a26 | 5 +---- hosts/taishan200-2280-2s48p-256g--a27 | 5 +---- hosts/taishan200-2280-2s48p-256g--a28 | 5 +---- hosts/taishan200-2280-2s48p-256g--a29 | 5 +---- hosts/taishan200-2280-2s48p-256g--a30 | 5 +---- hosts/taishan200-2280-2s48p-256g--a31 | 5 +---- hosts/taishan200-2280-2s48p-256g--a32 | 5 +---- hosts/taishan200-2280-2s48p-256g--a33 | 5 +---- hosts/taishan200-2280-2s48p-256g--a34 | 5 +---- hosts/taishan200-2280-2s48p-256g--a52 | 5 +---- hosts/taishan200-2280-2s48p-256g--a53 | 5 +---- hosts/taishan200-2280-2s48p-256g--a60 | 5 +---- hosts/taishan200-2280-2s48p-256g--a61 | 5 +---- hosts/taishan200-2280-2s48p-256g--a62 | 5 +---- hosts/taishan200-2280-2s48p-256g--a66 | 5 +---- hosts/taishan200-2280-2s48p-256g--a67 | 5 +---- hosts/taishan200-2280-2s48p-256g--a68 | 5 +---- hosts/taishan200-2280-2s48p-256g--a69 | 5 +---- hosts/taishan200-2280-2s48p-256g--a70 | 5 +---- hosts/taishan200-2280-2s48p-256g--a71 | 5 +---- ...280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} | 5 +---- ...-2280-2s48p-256g--a95 => taishan200-2280-2s48p-512g--a95} | 5 +---- hosts/taishan200-2280-2s64p-256g--a10 | 5 +---- hosts/taishan200-2280-2s64p-256g--a3 | 5 +---- hosts/taishan200-2280-2s64p-256g--a36 | 5 +---- hosts/taishan200-2280-2s64p-256g--a4 | 5 +---- hosts/taishan200-2280-2s64p-256g--a40 | 5 +---- hosts/taishan200-2280-2s64p-256g--a41 | 5 +---- hosts/taishan200-2280-2s64p-256g--a42 | 5 +---- hosts/taishan200-2280-2s64p-256g--a8 | 5 +---- hosts/taishan200-2280-2s64p-256g--a9 | 5 +---- 43 files changed, 43 insertions(+), 172 deletions(-) rename hosts/{taishan200-2280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} (93%) rename hosts/{taishan200-2280-2s48p-256g--a95 => taishan200-2280-2s48p-512g--a95} (89%) diff --git a/hosts/taishan200-2280-2s48p-256g--a101 b/hosts/taishan200-2280-2s48p-256g--a101 index f8a0429..200087c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a101 +++ b/hosts/taishan200-2280-2s48p-256g--a101 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 nr_ssd_partitions: 1 hdd_partitions: diff --git a/hosts/taishan200-2280-2s48p-256g--a11 b/hosts/taishan200-2280-2s48p-256g--a11 index 619dc06..54db822 100644 --- a/hosts/taishan200-2280-2s48p-256g--a11 +++ b/hosts/taishan200-2280-2s48p-256g--a11 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8891431-part1 diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 index 468c175..0c9fe89 100644 --- a/hosts/taishan200-2280-2s48p-256g--a13 +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-35000c500c2799bb3 diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 index 24ea9dc..f4c87bb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a14 +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4475 diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 index 9c28258..4eb4d2e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a15 +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b841fff5 diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 index 16b065d..1f9df96 100644 --- a/hosts/taishan200-2280-2s48p-256g--a17 +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b44cd diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 index 9d1ec01..d570c02 100644 --- a/hosts/taishan200-2280-2s48p-256g--a18 +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b891b4d1 diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 index 217eaa9..bf856a8 100644 --- a/hosts/taishan200-2280-2s48p-256g--a19 +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8923939 diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 index 738738d..8f68f5d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a21 +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf2d5 diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 index bb29d94..8efa398 100644 --- a/hosts/taishan200-2280-2s48p-256g--a22 +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919069 diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index 60ea1d2..4f85b6c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8917d59 diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index 916513e..eab76f0 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4499 diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 index 4a5a0f3..46d4b57 100644 --- a/hosts/taishan200-2280-2s48p-256g--a26 +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf131 diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 index 129589c..7e7692a 100644 --- a/hosts/taishan200-2280-2s48p-256g--a27 +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918ea9 diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 index f8a77f8..fa788e9 100644 --- a/hosts/taishan200-2280-2s48p-256g--a28 +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdf1eb07 diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 index 37814cc..87f1f18 100644 --- a/hosts/taishan200-2280-2s48p-256g--a29 +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918425 diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 index f722edb..5e69e11 100644 --- a/hosts/taishan200-2280-2s48p-256g--a30 +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b881 diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 index 7d6c67c..87448ab 100644 --- a/hosts/taishan200-2280-2s48p-256g--a31 +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b929 diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 index fb17e38..28c9a85 100644 --- a/hosts/taishan200-2280-2s48p-256g--a32 +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf141 diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 index a3a6b03..74be9f5 100644 --- a/hosts/taishan200-2280-2s48p-256g--a33 +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919555 diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 index 5a15949..ca35af0 100644 --- a/hosts/taishan200-2280-2s48p-256g--a34 +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be219 diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 index 2bd7096..f3442dc 100644 --- a/hosts/taishan200-2280-2s48p-256g--a52 +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdefd89b diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 index bf0894b..bffcb38 100644 --- a/hosts/taishan200-2280-2s48p-256g--a53 +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6aa4e3 diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 index 2255f2f..092e0bb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a60 +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd67640b diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 index 73079cb..e496d34 100644 --- a/hosts/taishan200-2280-2s48p-256g--a61 +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399886a7869 diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 index abaac6e..ef5dc25 100644 --- a/hosts/taishan200-2280-2s48p-256g--a62 +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b892d131 diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index fce93d1..a22322d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-35000c5009a671773 diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index 6b32609..f3f92c5 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88aed21 diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index 4bdd8f4..f90f5c1 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8909f81 diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index 3d3be80..a1eb381 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be92d diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 89c3c77..e61655d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g nr_hdd_partitions: 3 hdd_partitions: - /dev/disk/by-id/scsi-350000399b890931d diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 index 3121212..49eff04 100644 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ b/hosts/taishan200-2280-2s48p-256g--a71 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 256G +<<: hosts/taishan200-2280-2s48p-256g mac_addr: - 44:67:47:d7:6d:e6 - 44:67:47:d7:6d:e7 diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-512g--a102 similarity index 93% rename from hosts/taishan200-2280-2s48p-256g--a102 rename to hosts/taishan200-2280-2s48p-512g--a102 index e1d3f10..ea249e6 100644 --- a/hosts/taishan200-2280-2s48p-256g--a102 +++ b/hosts/taishan200-2280-2s48p-512g--a102 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 512G +<<: hosts/taishan200-2280-2s48p-512g nr_hdd_partitions: 2 nr_ssd_partitions: 3 hdd_partitions: diff --git a/hosts/taishan200-2280-2s48p-256g--a95 b/hosts/taishan200-2280-2s48p-512g--a95 similarity index 89% rename from hosts/taishan200-2280-2s48p-256g--a95 rename to hosts/taishan200-2280-2s48p-512g--a95 index 213d10f..bcd135f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a95 +++ b/hosts/taishan200-2280-2s48p-512g--a95 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 96 -memory: 512G +<<: hosts/taishan200-2280-2s48p-512g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-35000039988792e31 diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index cdb32f5..e630c59 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_ssd_partitions: 1 rootfs_partition: - /dev/disk/by-id/scsi-350000399c8030fd5 diff --git a/hosts/taishan200-2280-2s64p-256g--a3 b/hosts/taishan200-2280-2s64p-256g--a3 index 8f34f3f..53f7a81 100644 --- a/hosts/taishan200-2280-2s64p-256g--a3 +++ b/hosts/taishan200-2280-2s64p-256g--a3 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_ssd_partitions: 3 ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part1 diff --git a/hosts/taishan200-2280-2s64p-256g--a36 b/hosts/taishan200-2280-2s64p-256g--a36 index 9757144..eb828b8 100644 --- a/hosts/taishan200-2280-2s64p-256g--a36 +++ b/hosts/taishan200-2280-2s64p-256g--a36 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_ssd_partitions: 1 rootfs_partition: - /dev/disk/by-id/scsi-3500003999813487d diff --git a/hosts/taishan200-2280-2s64p-256g--a4 b/hosts/taishan200-2280-2s64p-256g--a4 index bcfaea7..50161f6 100644 --- a/hosts/taishan200-2280-2s64p-256g--a4 +++ b/hosts/taishan200-2280-2s64p-256g--a4 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_ssd_partitions: 3 ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part1 diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 index d9a9937..e8a7b2e 100644 --- a/hosts/taishan200-2280-2s64p-256g--a40 +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-3500003999812ed81 diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 index 91531fd..3cbda60 100644 --- a/hosts/taishan200-2280-2s64p-256g--a41 +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919495 diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 index ed1f188..61dc01d 100644 --- a/hosts/taishan200-2280-2s64p-256g--a42 +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918a11 diff --git a/hosts/taishan200-2280-2s64p-256g--a8 b/hosts/taishan200-2280-2s64p-256g--a8 index e515306..8dc0734 100644 --- a/hosts/taishan200-2280-2s64p-256g--a8 +++ b/hosts/taishan200-2280-2s64p-256g--a8 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g rootfs_partition: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43325 mac_addr: diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 index ce3aa0a..bdd27f0 100644 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ b/hosts/taishan200-2280-2s64p-256g--a9 @@ -1,7 +1,4 @@ -arch: aarch64 -nr_node: 4 -nr_cpu: 128 -memory: 256G +<<: hosts/taishan200-2280-2s64p-256g nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6a1347 -- 2.23.0

2 1

[PATCH v4 compass-ci] save hostname and queues relation in redis
by Xiao Shenwei 02 Nov '20

02 Nov '20

[why] the test machine has the ability to know which queues should be requested [how] for VM HW, we will register mac2host and host2queues. then they can find queues based on mac. API: boot.ipxe/mac/${mac} for docker, we will register host2queues, then container can find queues based on hostname. API: boot.container/hostname/${hostname Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 8 ++++++++ src/scheduler/scheduler.cr | 24 ++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index b2ef6ce..80cf0b0 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -38,6 +38,14 @@ class Sched @redis.hash_del("sched/mac2host", normalize_mac(mac)) end + def set_host2queues(hostname : String, queues : String) + @redis.hash_set("sched/host2queues", hostname, queues) + end + + def del_host2queues(hostname : String) + @redis.hash_del("sched/host2queues", hostname) + end + # return: # Hash(String, Hash(String, String)) def get_cluster_state(cluster_id) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 371e4ed..a946ea0 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -113,6 +113,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/set_host2queues?queues=vm-2p8g.aarch64&host=vm-2p8g.a…" + put "/set_host2queues" do |env| + env.response.headers["Connection"] = "close" + if (client_queues = env.params.query["queues"]?) && (client_host = env.params.query["host"]?) + sched.set_host2queues(client_host, client_queues) + + "Done" + else + "No yet" + end + end + # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| env.response.headers["Connection"] = "close" @@ -125,6 +137,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/del_host2queues?host=vm-2p8g.aarch64" + put "/del_host2queues" do |env| + env.response.headers["Connection"] = "close" + if client_host = env.params.query["host"]? + sched.del_host2queues(client_host) + + "Done" + else + "No yet!" + end + end + # client(runner) report job's status # /~lkp/cgi-bin/lkp-jobfile-append-var # ?job_file=/lkp/scheduled/job.yaml&job_state=running&job_id=10 -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] auto group jobs with user-defined template
by Lu Weitao 02 Nov '20

02 Nov '20

auto group jobs with user-defined template to support template compare mode background: before compare, create job-groups for all jobs that query from ES with user's filter then, create matrices-groups according job-groups finally, compare each group matrices Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/params_group.rb | 121 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 1 deletion(-) diff --git a/lib/params_group.rb b/lib/params_group.rb index 0f07d26..3c6de7a 100644 --- a/lib/params_group.rb +++ b/lib/params_group.rb @@ -2,7 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -# Exammple: +# Example: # Input: jobs_list. The results of ES query. # # eg: [ jobs1, jobs2, ...... ] @@ -113,3 +113,122 @@ end def remove_singleton(groups) groups.delete_if { |_k, v| v.length < 2 } end + +# -------------------------------------------------------------------------------------------------- +# auto group job_list by user's template +# Example: +# Input: +# 1. jobs_list. +# 2. params from template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# Output: +# eg: +# { +# '4K|1G' => { +# 'openeuler 20.03' => [ +# {'stats' => {'fio.write_iops' => 312821.002387, 'fio.read_iops' => 212821.2387}}, +# {'stats' => {'fio.write_iops' => 289661.878453}}, +# ... +# ], +# 'centos 7.6' => [...] +# }, +# '16K|1G' => {...}, +# ... +# } +# + +def auto_group_by_template(jobs_list, group_params, dimensions, metrics) + job_list = extract_jobs_list(jobs_list) + get_group_by_template(job_list, group_params, dimensions, metrics) +end + +def get_group_by_template(job_list, group_params, dimensions, metrics) + groups = {} + job_list.each do |job| + next unless job['stats'] + + group_key = get_user_group_key(job, group_params) + dimension = get_user_dimension(job, dimensions) + next unless group_key && dimension + + new_job = get_new_job_by_metrics(job, metrics) + next if new_job.empty? + + groups[group_key] ||= {} + groups[group_key][dimension] ||= [] + groups[group_key][dimension] << new_job + end + groups +end + +# @group_params Array(String) +# eg: +# ['block_size', 'package_size'] +# return eg: +# '4K|1G' +def get_user_group_key(job, group_params) + group_key_list = [] + group_params.each do |param| + if job.key?(param) + group_key_list << job[param] + next + end + job.each_value do |v| + if v.is_a?(Hash) && v.key?(param) + group_key_list << v[param] + break + end + end + end + return nil if group_key_list.size < group_params.size || group_key_list.empty? + + group_key_list.join('|') +end + +# @dimension Array(Hash) +# eg: +# [ +# {os => openeuler, os_version => 20.03}, +# {os => centos, os_version => 7.6} +# ] +# return eg: +# 'openeuler 20.03' +def get_user_dimension(job, dimensions) + dimension_list = [] + dimensions.each do |dim| + dim.each do |key, value| + if job[key] == value + dimension_list << value + end + end + return nil if !dimension_list.empty? && dimension_list.size < dim.size + end + return nil if dimension_list.empty? + + dimension_list.join(' ') +end + +# @metrics Array(String) +# eg: +# ["fio.read_iops", "fio.write_iops"] +# return new_job +# eg: +# {'stats' => {'fio.write_iops' => 312821.002387, 'fio.read_iops' => 212821.2387}}, +def get_new_job_by_metrics(job, metrics) + new_job = {} + metrics.each do |metric| + if job['stats'].key?(metric) + new_job['stats'] ||= {} + new_job['stats'][metric] = job['stats'][metric] + end + end + new_job +end -- 2.23.0

3 7

[PATCH v3 compass-ci] LICENSES: add THIRD PARTY OPEN SOURCE SOFTWARE NOTICE
by Lu Kaiyi 02 Nov '20

02 Nov '20

add THIRD PARTY OPEN SOURCE SOFTWARE NOTICE for references. the directory structure of LICENSES as below: . ├── CCBY-4.0 ├── coreutils │ ├── coreutils │ ├── files │ └── GPL-3.0 -> ../GPL-3.0 ├── docker-sshd │ ├── docker-sshd │ ├── files │ └── MIT -> ../MIT ├── dracut │ ├── dracut │ ├── files │ └── GPL-2.0 -> ../GPL-2.0 ├── GPL-2.0 ├── GPL-3.0 ├── kemal │ ├── files │ ├── kemal │ └── MIT -> ../MIT ├── lkp-tests │ ├── files │ ├── GPL-2.0 -> ../GPL-2.0 │ └── lkp-tests ├── MIT ├── mritd.github.io │ ├── files │ ├── MIT -> ../MIT │ └── mritd.github.io ├── MulanPSL-2.0 └── Scout ├── files ├── Scout └── MIT -> ../MIT Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- LICENSES/Scout/MIT | 1 + LICENSES/Scout/Scout | 17 +++++++++++++++++ LICENSES/Scout/files | 1 + LICENSES/coreutils/GPL-3.0 | 1 + LICENSES/coreutils/coreutils | 17 +++++++++++++++++ LICENSES/coreutils/files | 1 + LICENSES/docker-sshd/MIT | 1 + LICENSES/docker-sshd/docker-sshd | 17 +++++++++++++++++ LICENSES/docker-sshd/files | 1 + LICENSES/dracut/GPL-2.0 | 1 + LICENSES/dracut/dracut | 18 ++++++++++++++++++ LICENSES/dracut/files | 1 + LICENSES/kemal/MIT | 1 + LICENSES/kemal/files | 1 + LICENSES/kemal/kemal | 17 +++++++++++++++++ LICENSES/lkp-tests/GPL-2.0 | 1 + LICENSES/lkp-tests/files | 8 ++++++++ LICENSES/lkp-tests/lkp-tests | 18 ++++++++++++++++++ LICENSES/mritd.github.io/MIT | 1 + LICENSES/mritd.github.io/files | 1 + LICENSES/mritd.github.io/mritd.github.io | 17 +++++++++++++++++ 21 files changed, 142 insertions(+) create mode 120000 LICENSES/Scout/MIT create mode 100644 LICENSES/Scout/Scout create mode 100644 LICENSES/Scout/files create mode 120000 LICENSES/coreutils/GPL-3.0 create mode 100644 LICENSES/coreutils/coreutils create mode 100644 LICENSES/coreutils/files create mode 120000 LICENSES/docker-sshd/MIT create mode 100644 LICENSES/docker-sshd/docker-sshd create mode 100644 LICENSES/docker-sshd/files create mode 120000 LICENSES/dracut/GPL-2.0 create mode 100644 LICENSES/dracut/dracut create mode 100644 LICENSES/dracut/files create mode 120000 LICENSES/kemal/MIT create mode 100644 LICENSES/kemal/files create mode 100644 LICENSES/kemal/kemal create mode 120000 LICENSES/lkp-tests/GPL-2.0 create mode 100644 LICENSES/lkp-tests/files create mode 100644 LICENSES/lkp-tests/lkp-tests create mode 120000 LICENSES/mritd.github.io/MIT create mode 100644 LICENSES/mritd.github.io/files create mode 100644 LICENSES/mritd.github.io/mritd.github.io diff --git a/LICENSES/Scout/MIT b/LICENSES/Scout/MIT new file mode 120000 index 0000000..91e6d98 --- /dev/null +++ b/LICENSES/Scout/MIT @@ -0,0 +1 @@ +../MIT \ No newline at end of file diff --git a/LICENSES/Scout/Scout b/LICENSES/Scout/Scout new file mode 100644 index 0000000..6844406 --- /dev/null +++ b/LICENSES/Scout/Scout @@ -0,0 +1,17 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: Scout db170ce4ff160c4a2c30483e9410df1773cf572c + +Copyright notice: Copyright(c) 2010 Menno van Slooten, http://mennovanslooten.nl/ +License: The MIT License +License-Text: please refer to file MIT diff --git a/LICENSES/Scout/files b/LICENSES/Scout/files new file mode 100644 index 0000000..e120df8 --- /dev/null +++ b/LICENSES/Scout/files @@ -0,0 +1 @@ +/compass-ci/lib/themes.rb diff --git a/LICENSES/coreutils/GPL-3.0 b/LICENSES/coreutils/GPL-3.0 new file mode 120000 index 0000000..481ee24 --- /dev/null +++ b/LICENSES/coreutils/GPL-3.0 @@ -0,0 +1 @@ +../GPL-3.0 \ No newline at end of file diff --git a/LICENSES/coreutils/coreutils b/LICENSES/coreutils/coreutils new file mode 100644 index 0000000..1250a6f --- /dev/null +++ b/LICENSES/coreutils/coreutils @@ -0,0 +1,17 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: coreutils 6.9.89 + +Copyright notice: Copyright(C) 2007 Free Software Foundation, Inc. <https://fsf.org/> +License: The GPL-3.0 License +License-Text: please refer to file GPL-3.0 diff --git a/LICENSES/coreutils/files b/LICENSES/coreutils/files new file mode 100644 index 0000000..8a2b690 --- /dev/null +++ b/LICENSES/coreutils/files @@ -0,0 +1 @@ +/compass-ci/container/os-nfs/entrypoint.sh diff --git a/LICENSES/docker-sshd/MIT b/LICENSES/docker-sshd/MIT new file mode 120000 index 0000000..91e6d98 --- /dev/null +++ b/LICENSES/docker-sshd/MIT @@ -0,0 +1 @@ +../MIT \ No newline at end of file diff --git a/LICENSES/docker-sshd/docker-sshd b/LICENSES/docker-sshd/docker-sshd new file mode 100644 index 0000000..603f668 --- /dev/null +++ b/LICENSES/docker-sshd/docker-sshd @@ -0,0 +1,17 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: docker-sshd 729faee00bcf0eff5f80059cd92aa425505820f0 + +Copyright notice: Copyright(c) 2015-2020 Volt Grid Pty Ltd +License: The MIT License +License-Text: please refer to file MIT diff --git a/LICENSES/docker-sshd/files b/LICENSES/docker-sshd/files new file mode 100644 index 0000000..018684c --- /dev/null +++ b/LICENSES/docker-sshd/files @@ -0,0 +1 @@ +/compass-ci/container/ssh-r/entry.sh diff --git a/LICENSES/dracut/GPL-2.0 b/LICENSES/dracut/GPL-2.0 new file mode 120000 index 0000000..9824569 --- /dev/null +++ b/LICENSES/dracut/GPL-2.0 @@ -0,0 +1 @@ +../GPL-2.0 \ No newline at end of file diff --git a/LICENSES/dracut/dracut b/LICENSES/dracut/dracut new file mode 100644 index 0000000..f09df60 --- /dev/null +++ b/LICENSES/dracut/dracut @@ -0,0 +1,18 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: dracut RHEL-7.1 + +Copyright notice: Copyright(C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, +Fifth Floor, Boston, MA 02110-1301 USA. +License: The GPL-2.0 License +License-Text: please refer to file GPL-2.0 diff --git a/LICENSES/dracut/files b/LICENSES/dracut/files new file mode 100644 index 0000000..78235de --- /dev/null +++ b/LICENSES/dracut/files @@ -0,0 +1 @@ +/compass-ci/container/dracut-initrd/bin/cifs-lib.sh diff --git a/LICENSES/kemal/MIT b/LICENSES/kemal/MIT new file mode 120000 index 0000000..91e6d98 --- /dev/null +++ b/LICENSES/kemal/MIT @@ -0,0 +1 @@ +../MIT \ No newline at end of file diff --git a/LICENSES/kemal/files b/LICENSES/kemal/files new file mode 100644 index 0000000..71a8cab --- /dev/null +++ b/LICENSES/kemal/files @@ -0,0 +1 @@ +/compass-ci/src/spec/scheduler/boot_spec.cr diff --git a/LICENSES/kemal/kemal b/LICENSES/kemal/kemal new file mode 100644 index 0000000..e08255f --- /dev/null +++ b/LICENSES/kemal/kemal @@ -0,0 +1,17 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: kemal 0.17.0 + +Copyright notice: Copyright(c) 2016 Serdar Doğruyol +License: The MIT License +License-Text: please refer to file MIT diff --git a/LICENSES/lkp-tests/GPL-2.0 b/LICENSES/lkp-tests/GPL-2.0 new file mode 120000 index 0000000..9824569 --- /dev/null +++ b/LICENSES/lkp-tests/GPL-2.0 @@ -0,0 +1 @@ +../GPL-2.0 \ No newline at end of file diff --git a/LICENSES/lkp-tests/files b/LICENSES/lkp-tests/files new file mode 100644 index 0000000..fde5c05 --- /dev/null +++ b/LICENSES/lkp-tests/files @@ -0,0 +1,8 @@ +/compass-ci/sbin/compare +/compass-ci/providers/multi-docker +/compass-ci/container/open-scheduler/build-depends +/compass-ci/user-client/jobs/iperf-pxe.yaml +/compass-ci/user-client/jobs/iperf-vm.yaml +/compass-ci/src/features/jobs/right_iperf.yaml +/compass-ci/lib/matrix2.rb +/compass-ci/sbin/es-find diff --git a/LICENSES/lkp-tests/lkp-tests b/LICENSES/lkp-tests/lkp-tests new file mode 100644 index 0000000..3f40d5f --- /dev/null +++ b/LICENSES/lkp-tests/lkp-tests @@ -0,0 +1,18 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: lkp-tests fcb47ccfc50c1f4da388aed9596c0acaac04a917 + +Copyright notice: Copyright(C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, +Fifth Floor, Boston, MA 02110-1301 USA +License: The GPL-2.0 License +License-Text: please refer to file GPL-2.0 diff --git a/LICENSES/mritd.github.io/MIT b/LICENSES/mritd.github.io/MIT new file mode 120000 index 0000000..91e6d98 --- /dev/null +++ b/LICENSES/mritd.github.io/MIT @@ -0,0 +1 @@ +../MIT \ No newline at end of file diff --git a/LICENSES/mritd.github.io/files b/LICENSES/mritd.github.io/files new file mode 100644 index 0000000..38b67c1 --- /dev/null +++ b/LICENSES/mritd.github.io/files @@ -0,0 +1 @@ +/compass-ci/container/registry/config.yml diff --git a/LICENSES/mritd.github.io/mritd.github.io b/LICENSES/mritd.github.io/mritd.github.io new file mode 100644 index 0000000..ee22568 --- /dev/null +++ b/LICENSES/mritd.github.io/mritd.github.io @@ -0,0 +1,17 @@ +THIRD PARTY OPEN SOURCE SOFTWARE NOTICE + +Please note we provide an open source software notice for the third party open source software along with +this software and/or this software component contributed by Huawei (in the following just "this SOFTWARE"). +The open source software licenses are granted by respective right holders. + +Warranty Disclaimer +THE OPEN SOURCE SOFTWARE IN THIS SOFTWARE IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, +BUT WITHOUT ANY WARRANTY, WITHOUT EVEN THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS +FOR A PARTICULAR PURPOSE. SEE THE APPLICABLE LICENSES FOR MORE DETAILS. + +Copyright Notice and License Texts +Software: mritd.github.io ed4350e91e9b76078a850ede8302854d3393f493 + +Copyright notice: Copyright(c) 2013-2016 Blackrock Digital LLC. +License: The MIT License +License-Text: please refer to file MIT -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/*: fix cci job dir layout
by Wang Yong 02 Nov '20

02 Nov '20

[Why] for multi version support, cgz file for cci-makepkg need change dir layout, and change cci-depends dir layout together to easily update scheduler code path: /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/ => /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/${pkgname}/ filename: cci-makepkg ${pkgver}-${pkgrel}.cgz latest.cgz -> ${pkgver}-${pkgrel}.cgz filename: cci-depends stay original ${benchmark}.cgz ${benchmark}.cgz -> ${benchmark}_${date}.cgz E.g. output: cci-depends stay original $ tree iperf ├── iperf_20201102.cgz └── iperf.cgz -> iperf_20201102.cgz output: cci-makepkg $ tree sar ├── git-1.cgz └── latest.cgz -> git-1.cgz Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/cci-depends | 2 +- tests/cci-makepkg | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/cci-depends b/tests/cci-depends index 36923ea4d..efa9a025b 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -18,7 +18,7 @@ DEPS_MNT=/initrd/deps && mkdir -p "$DEPS_MNT" DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} pack_arch=$os_arch [[ "$benchmark" = "${benchmark##*.}" ]] || diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 9d5082629..9551ebfe2 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -26,7 +26,7 @@ mkdir -p "$PKG_MNT" export DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} . $LKP_SRC/distro/${DISTRO} . $LKP_SRC/lib/install.sh @@ -65,8 +65,8 @@ update_shared_pkg() [ "$bm_name" = "$benchmark" ] && return # benchmark is a symlink - ln -sf "$bm_link" "$sync_dest/${benchmark}.cgz" || return - echo "update shared pkg link ${benchmark}.cgz -> $bm_link" + ln -sf latest.cgz "$sync_dest/${benchmark}.cgz" || return + echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } distro_install_depends lkp-dev @@ -86,7 +86,7 @@ date=$(date +"%Y%m%d") pkgver=$(get_pkg_info pkgver) pkgrel=$(get_pkg_info pkgrel) bm_name=$(check_shared_pkg) -cgz_name="${bm_name}-${pkgver:-0}-${pkgrel:-0}_${date}.cgz" +cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy @@ -98,8 +98,8 @@ update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return - ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/${bm_name}.cgz" || return - echo "create package: $sync_dest/${bm_name}.cgz -> $(realpath $sync_dest/$cgz_name)" + ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/latest.cgz" || return + echo "create package: $sync_dest/latest.cgz -> $(realpath $sync_dest/$cgz_name)" update_shared_pkg "${bm_name}.cgz" -- 2.23.0

1 0

[PATCH compass-ci] container: fix failed to build archlinux images
by Liu Yinsi 02 Nov '20

02 Nov '20

[why] when build archlinux images in x86_64 machine error: error: failed retrieving file 'core.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update core (failed to retrieve some files) error: failed retrieving file 'extra.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirror.archlinuxarm.org : Resolving timed out after 10000 milliseconds error: failed to update extra (download library error) error: failed retrieving file 'community.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update community (failed to retrieve some files) error: failed to synchronize all databases The command '/bin/sh -c pacman --needed --noprogressbar --noconfirm -Syu && pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk' returned a non-zero code: 1 becaue archlinux mirror not support x86_64 machine. [how] when deploy in x86_64 machine, change mirror http://mirrors.tuna.tsinghua.edu.cn/archlinuxarm/$arch/$repo => http://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/archlinux/build | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/archlinux/build b/container/archlinux/build index 81feda2..9194f4e 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ "$(arch)" == "x86_64" ] && + sed -i 's/archlinuxarm\/$arch\/$repo/archlinux\/$repo\/os\/$arch/g' root/etc/pacman.d/mirrorlist + docker build -t archlinux:testbed . -- 2.23.0

2 2

[PATCH compass-ci 2/2] container/assign-account: answerback-email.rb
by Luan Shengde 02 Nov '20

02 Nov '20

1. disable use golbal variable apply_info [why] Style/GlobalVars: Do not introduce global variables. [how] use local variables instead of global variables 2. enable transfer user info when execute apply account command [why] when applying account, the assign-account service will write the user info to user's default config file ~/.config/compass-ci/default/${USER}.yaml user info: - my_email - my_name - my_uuid - lab [how] transfer the user info along with the pub_key to assign-account service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/answerback-email.rb | 60 ++++++++++++-------- 1 file changed, 35 insertions(+), 25 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index bb8e809..4227081 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -18,29 +18,31 @@ names = Set.new %w[ JUMPER_PORT SEND_MAIL_HOST_INTERNET SEND_MAIL_PORT_INTERNET + LAB ] defaults = relevant_defaults(names) JUMPER_HOST = defaults['JUMPER_HOST'] || 'api.compass-ci.openeuler.org' JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 -SEND_MAIL_HOST = defaults['SEND_MAIL_HOST_INTERNET'] || 'localhost' -SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] || 11312 +SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 +LAB = defaults['LAB'] || z9 -$apply_info = { +apply_info = { 'my_email' => nil, + 'my_name' => nil, 'my_ssh_pubkey' => nil } -def init_info(email_file) - mail_content = Mail.read(email_file) +def init_info(mail_content, apply_info) + apply_info['my_email'] = mail_content.from[0] + apply_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') + apply_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' + mail_content.part[1].body.decoded.gsub(/\r|\n/, '') + end - $apply_info['my_email'] = mail_content.from[0] - $apply_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded.gsub(/\r|\n/, '') - end - - $apply_info + apply_info end options = OptionParser.new do |opts| @@ -52,15 +54,18 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e|--email email_address', 'appoint email address') do |email_address| - $apply_info['my_email'] = email_address + apply_info['my_email'] = email_address + # when apply account with email address, will not get the user name + apply_info['my_name'] = '' end opts.on('-s|--ssh-pubkey pub_key_file', 'ssh pub_key file, enable password-less login') do |pub_key_file| - $apply_info['my_ssh_pubkey'] = File.read(pub_key_file) + apply_info['my_ssh_pubkey'] = File.read(pub_key_file) end opts.on('-f|--raw-email email_file', 'email file') do |email_file| - init_info(email_file) + mail_content = Mail.read(email_file) + init_info(mail_content, apply_info) end opts.on_tail('-h|--help', 'show this message') do @@ -94,26 +99,31 @@ def build_message(email, acct_infos) return message end -def account_info(pub_key) - account_info_str = if pub_key.nil? - %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account') - else - %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d "pub_key: #{pub_key}") - end +def account_info(user_info) + account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{user_info.to_json}') JSON.parse account_info_str end -def send_account +def send_account(apply_info) message = "No email address specified\n" message += "use -e email_address add a email address\n" message += 'or use -f to add a email file' - raise message if $apply_info['my_email'].nil? + raise message if apply_info['my_email'].nil? + + user_info = { + 'my_email': apply_info['my_email'], + 'my_name': apply_info['my_name'], + # for manually assign account, there will be no my_uuid specified + 'my_uuid': '', + 'lab': LAB, + 'pub_key': apply_info['my_ssh_pubkey'] + } - acct_info = account_info($apply_info['my_ssh_pubkey']) + acct_info = account_info(user_info) - message = build_message($apply_info['my_email'], acct_info) + message = build_message(apply_info['my_email'], acct_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end -send_account +send_account(apply_info) -- 2.23.0

4 6

[PATCH v6 compass-ci 3/3] kernel_version.md: explain key "kernel_version"
by Xu Xijian 02 Nov '20

02 Nov '20

[why] Explain the meaning of new key "kernel_version" for scheduler, including what's the mapping between its typical values and actual files in the disk and how to set it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 doc/job/kernel_version.md diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md new file mode 100644 index 0000000..56a6b1a --- /dev/null +++ b/doc/job/kernel_version.md @@ -0,0 +1,32 @@ +# kernel_version + +Meaning: +- Every os has its kernel, however an os can start with different kernels according to different need. +- kernel_version is a key for users to specify a kernel version. +- If kernel_version is not given by users, it will use the default one. + +Related files: +- In initramfs boot process, every kernel version is related with a vmlinuz, module and headers. +- File paths like below, An example $boot_dir can be "/srv/os/openeuler/aarch64/20.03/boot" + +- $boot_dir/vmlinuz => $boot_dir/vmlinuz-$kernel_version + - default kernel file(vmlinuz) is a soft link + +- $boot_dir/modules => $boot_dir/modules-$kernel_version.cgz + - default modules is a soft link + +- $boot_dir/headers => $boot_dir/headers-$kernel_version.cgz + - default headers is a soft link + +- $boot_dir/vmlinuz-$kernel_version + - real kernel path + +- $boot_dir/modules-$kernel_version.cgz + - real modules path + +- $boot_dir/headers-$kernel_version.cgz + - real headers path + + +Usage example: +- submit iperf.yaml testbox=vm-hi1620-2p8g--xxj os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 -- 2.23.0

2 2

[PATCH compass-ci 1/2] container/assign-account: get_account_info add new function
by Luan Shengde 02 Nov '20

02 Nov '20

[why]: esaier for user to config the default yaml file [how]: when get request data, parse the data and write the user infos to default yaml file: ~/.config/compass-ci/default/${USER}.yaml include: - my_email - my_name - my_uuid - lab Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/get_account_info.rb | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 2f93d5b..4e61bdc 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -44,6 +44,8 @@ the returned data for setup_jumper_account_info like: =end +require 'fileutils' + # get jumper and account info class AccountStorage ACCOUNT_DIR = '/opt/account_data/' @@ -111,9 +113,23 @@ class AccountStorage } setup_authorized_key(account, pub_key) + setup_default_yaml(account) if @data.key? 'my_email' + return jumper_account_info end + def setup_default_yaml(account) + default_yaml_dir = File.join('/home', account, '.config/compass-ci/defaults') + FileUtils.mkdir_p default_yaml_dir + File.open("#{default_yaml_dir}/#{account}.yaml", 'a') do |file| + file.puts "my_email: #{@data['my_email']}" + file.puts "my_name: #{@data['my_name']}" if @data.key? 'my_name' + file.puts "my_uuid: #{@data['my_uuid']}" if @data.key? 'my_uuid' + file.puts "lab: #{@data['lab']}" if @data.key? 'lab' + file.close + end + end + def setup_authorized_key(account, pub_key) ssh_dir = File.join('/home/', account, '.ssh') Dir.mkdir ssh_dir, 0o700 -- 2.23.0

6 10

[PATCH lkp-tests] tests/*: fix cci job dir layout
by Wang Yong 02 Nov '20

02 Nov '20

[Why] for multi version support, cgz file for cci-makepkg need change dir layout, and change cci-depends dir layout together to easily update scheduler code path: /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/ => /srv/initrd/pkg/${os_mount}/${os}/${os_arch}/${os_version}/${pkgname}/ filename: cci-makepkg ${pkgver}-${pkgrel}.cgz latest.cgz -> ${pkgver}-${pkgrel}.cgz filename: cci-depends ${benchmark}.cgz latest.cgz -> ${benchmark}.cgz E.g. output: cci-depends $ tree iperf ├── iperf.cgz └── latest.cgz -> iperf.cgz output: cci-makepkg $ tree sar ├── git-1.cgz └── latest.cgz -> git-1.cgz Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/aliyun | 7 +++---- distro/clear | 7 +++---- distro/debian | 8 +++----- tests/cci-depends | 4 ++-- tests/cci-makepkg | 13 ++++++------- 5 files changed, 17 insertions(+), 22 deletions(-) diff --git a/distro/aliyun b/distro/aliyun index 79656f0e4..a6ca49968 100755 --- a/distro/aliyun +++ b/distro/aliyun @@ -80,7 +80,6 @@ yum_download() yum_pack() { local target_dir=/opt/rpms - local date=$(date +"%Y%m%d") local downloaded_rpms=$(find /var/ -type f -name "*.rpm") @@ -90,10 +89,10 @@ yum_pack() mv $downloaded_rpms $target_dir - find $target_dir | cpio --quiet -o -H newc --owner=root.root | gzip -n -9 >$pack_to/${BM_NAME}_$date.cgz + find $target_dir | cpio --quiet -o -H newc --owner=root.root | gzip -n -9 >$pack_to/${BM_NAME}.cgz - ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + ln -sf ${BM_NAME}.cgz $pack_to/latest.cgz || return + chown .lkp $pack_to/${BM_NAME}.cgz $pack_to/latest.cgz || return echo package installed to $pack_to/${BM_NAME}.cgz ls $target_dir/*.rpm > $pack_to/.${BM_NAME}.packages diff --git a/distro/clear b/distro/clear index f21de89c3..3b562aac6 100755 --- a/distro/clear +++ b/distro/clear @@ -121,7 +121,6 @@ list_bundles_files() pack_bundles_files() { - local date=$(date +"%Y%m%d") # allow empty ls $tmpdir | grep -q "\.list$" >/dev/null || { echo "empty PACKAGE_LIST for $BM_NAME" @@ -130,10 +129,10 @@ pack_bundles_files() cat $tmpdir/*.list | sort | uniq | cpio --quiet -o -H newc --owner=root.root | - gzip -n -9 >$pack_to/${BM_NAME}_$date.cgz || return + gzip -n -9 >$pack_to/${BM_NAME}.cgz || return - ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + ln -sf ${BM_NAME}.cgz $pack_to/latest.cgz || return + chown .lkp $pack_to/${BM_NAME}.cgz $pack_to/latest.cgz || return echo package installed to $pack_to/${BM_NAME}.cgz } diff --git a/distro/debian b/distro/debian index 13c4e983a..270f77e01 100755 --- a/distro/debian +++ b/distro/debian @@ -99,8 +99,6 @@ fixup_perf() pack() { - local date=$(date +"%Y%m%d") - [[ $(type -t fixup_$BM_NAME) = 'function' ]] && fixup_$BM_NAME { @@ -112,12 +110,12 @@ pack() cat $LKP_SRC/distro/$DISTRO/$BM_NAME-allowlist } | cpio --quiet -o -H newc --owner=root.root | - gzip -n -9 > $pack_to/${BM_NAME}_$date.cgz || return + gzip -n -9 > $pack_to/${BM_NAME}.cgz || return - ln -sf ${BM_NAME}_$date.cgz \ + ln -sf ${BM_NAME}.cgz \ $pack_to/${BM_NAME}.cgz || return - chown .lkp $pack_to/${BM_NAME}_$date.cgz \ + chown .lkp $pack_to/${BM_NAME}.cgz \ $pack_to/${BM_NAME}.cgz || return echo package installed to $pack_to/${BM_NAME}.cgz diff --git a/tests/cci-depends b/tests/cci-depends index 36923ea4d..1c840a4ee 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -18,7 +18,7 @@ DEPS_MNT=/initrd/deps && mkdir -p "$DEPS_MNT" DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${DEPS_MNT}/${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} pack_arch=$os_arch [[ "$benchmark" = "${benchmark##*.}" ]] || @@ -38,7 +38,7 @@ umask 002 } check_shared_package "$benchmark" && { - ln -sf $BM_NAME.cgz $pack_to/$benchmark.cgz + ln -sf latest.cgz $pack_to/$benchmark.cgz echo "create symlink for shared package: $benchmark -> $BM_NAME" exit 0 diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 9d5082629..115136f61 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -26,7 +26,7 @@ mkdir -p "$PKG_MNT" export DISTRO=${os} [[ "$os_mount" = "cifs" ]] && os_mount="nfs" -pack_to=${os_mount}/${os}/${os_arch}/${os_version} +pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} . $LKP_SRC/distro/${DISTRO} . $LKP_SRC/lib/install.sh @@ -65,8 +65,8 @@ update_shared_pkg() [ "$bm_name" = "$benchmark" ] && return # benchmark is a symlink - ln -sf "$bm_link" "$sync_dest/${benchmark}.cgz" || return - echo "update shared pkg link ${benchmark}.cgz -> $bm_link" + ln -sf latest.cgz "$sync_dest/${benchmark}.cgz" || return + echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } distro_install_depends lkp-dev @@ -82,11 +82,10 @@ distro_install_depends ${benchmark}-dev build_depends_pkg -i $benchmark $TMP sync_dest="$PKG_MNT/$pack_to" -date=$(date +"%Y%m%d") pkgver=$(get_pkg_info pkgver) pkgrel=$(get_pkg_info pkgrel) bm_name=$(check_shared_pkg) -cgz_name="${bm_name}-${pkgver:-0}-${pkgrel:-0}_${date}.cgz" +cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy @@ -98,8 +97,8 @@ update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return - ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/${bm_name}.cgz" || return - echo "create package: $sync_dest/${bm_name}.cgz -> $(realpath $sync_dest/$cgz_name)" + ln -sf "$(basename $(realpath $sync_dest/$cgz_name))" "$sync_dest/latest.cgz" || return + echo "create package: $sync_dest/latest.cgz -> $(realpath $sync_dest/$cgz_name)" update_shared_pkg "${bm_name}.cgz" -- 2.23.0

4 5

[PATCH compass-ci] area: fix src/lib/web_backend.rb "/compare_candidates" tbox_group regex error
by Zhang Yuhang 02 Nov '20

02 Nov '20

[error info] 1. /\d+$/ will be matched, such as "xxx123". 2. `index = "xxx123".index('--') || "xxx123".rindex('-')` - index equal nil. 3. `r = r[0, nil]` - raise a TypeError. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index c3f6bef..55c5e4a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -47,7 +47,7 @@ end def filter_tbox_group(es_result) result = Set.new es_result.each do |r| - if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$)/ + if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$)/ index = r.index('--') || r.rindex('-') r = r[0, index] end -- 2.23.0

2 2

[PATCH v4 compass-ci] container/logging-es: optimize bulk write performance
by Wu Zhende 02 Nov '20

02 Nov '20

[Why] fluentd uses "bulk" to continuously write data to es. The es write capability needs to be optimized to adapt to high-concurrency scenarios. Before: fluent.info: POST http://172.17.0.50:9200/_bulk [status:200, request:0.116s, query:0.114s] After: fluent.info: POST http://172.17.0.50:9200/_bulk [status:200, request:0.060s, query:0.057s] [How] 1.Increase index.refresh_interval 2.Set index.translog.durability to Async 3.Increase the JVM memory 4.Increase write.queue_size 5.Increase index_buffer_size [Reference Link] https://www.elastic.co/guide/en/elasticsearch/reference/master/tune-for-ind… https://ravishajava.com/2018/07/13/elastic-search-performance-tuning-parmet… https://thoughts.t37.net/designing-the-perfect-elasticsearch-cluster-the-al… https://thehoard.blog/techniques-to-achieve-high-write-throughput-with-elas… Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/logging-es/Dockerfile | 12 ++++++++---- container/logging-es/start | 12 ++++++++++++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/container/logging-es/Dockerfile b/container/logging-es/Dockerfile index 7f91f65..707299f 100644 --- a/container/logging-es/Dockerfile +++ b/container/logging-es/Dockerfile @@ -7,10 +7,14 @@ FROM gagara/elasticsearch-oss-arm64:7.6.2 MAINTAINER Wu Zhende <wuzhende666(a)163.com> -RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a path.data: /srv/es/logging-es' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a node.name: node-1' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml; +RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a path.data: /srv/es/logging-es' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a node.name: node-1' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a indices.memory.index_buffer_size: 20%' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a thread_pool.write.queue_size: 2000' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i "s/-Xms1g/-Xms30g/g" /usr/share/elasticsearch/config/jvm.options && \ + sed -i "s/-Xmx1g/-Xmx30g/g" /usr/share/elasticsearch/config/jvm.options RUN mkdir /usr/share/elasticsearch/tmp && \ chown -R 1090:1090 /usr/share/elasticsearch diff --git a/container/logging-es/start b/container/logging-es/start index 9be7a5b..25ed36b 100755 --- a/container/logging-es/start +++ b/container/logging-es/start @@ -19,3 +19,15 @@ cmd=( ) "${cmd[@]}" + +echo "Please wait 60s for setting index" +sleep 60 + +curl -H "Content-Type: application/json" -XPUT "localhost:9202/_settings" -d ' +{ + "index": { + "refresh_interval": "120s", + "translog.durability": "async", + "translog.sync_interval": "120s", + "translog.flush_threshold_size": "1024mb"} +}' -- 2.23.0

1 0

[PATCH v7 compass-ci 1/5] container/ntp-server: Dockerfile
by Wang Chenglong 01 Nov '20

01 Nov '20

[What] The docker file for build a container to provide network time synchronization service for virtual and physical machines. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/ntp-server/Dockerfile diff --git a/container/ntp-server/Dockerfile b/container/ntp-server/Dockerfile new file mode 100644 index 0000000..76f2a0e --- /dev/null +++ b/container/ntp-server/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:7 +MAINTAINER Wang Chenglong <18509160991(a)163.com> + +RUN yum repolist +RUN yum -y install ntp && \ + yum clean all +RUN ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + +COPY ntp.conf /etc/ntp.conf +COPY start-server.sh / + +EXPOSE 123 + +ENTRYPOINT ["/start-server.sh"] +CMD ["sh"] -- 2.23.0

3 3

[PATCH v5 lkp-tests 2/2] tests/iozone: add specified parameter for iozone test
by Lu Kaiyi 01 Nov '20

01 Nov '20

[why] need to test write, rewrite, read, reread, random write, random read for different block_size. [how] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..88a92a18 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,13 +1,49 @@ #!/bin/sh - +# - block_size +# - file_size +# - write_rewrite +# - read_reread +# - random_read_write +# - read_backwards +# - rewrite_record +# - stride_read +# - fwrite_refwrite +# - fread_refread +# - random_mix +# - pwrite_repwrite +# - pread_repread +# - pwritev_repwritev +# - preadv_repreadv ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh + +args="iozone" +if [ -n "$block_size" ]; then + args+=" -r $block_size" + [ -n "$file_size" ] && args+=" -s $file_size" + [ -n "$write_rewrite" ] && args+=" -i 0" + [ -n "$read_reread" ] && args+=" -i 1" + [ -n "$random_read_write" ] && args+=" -i 2" + [ -n "$read_backwards" ] && args+=" -i 3" + [ -n "$rewrite_record" ] && args+=" -i 4" + [ -n "$stride_read" ] && args+=" -i 5" + [ -n "$fwrite_refwrite" ] && args+=" -i 6" + [ -n "$fread_refread" ] && args+=" -i 7" + [ -n "$random_mix" ] && args+=" -i 8" + [ -n "$pwrite_repwrite" ] && args+=" -i 9" + [ -n "$pread_repread" ] && args+=" -i 10" + [ -n "$pwritev_repwritev" ] && args+=" -i 11" + [ -n "$preadv_repreadv" ] && args+=" -i 12" +else + args+=" -a" +fi + for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + cmd="${args} -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

2 1

[PATCH v3 compass-ci] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 31 Oct '20

31 Oct '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- doc/job/kernel_version.md | 19 +++++++++++++++++++ src/lib/job.cr | 14 +++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) create mode 100644 doc/job/kernel_version.md diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md new file mode 100644 index 0000000..bd02cd1 --- /dev/null +++ b/doc/job/kernel_version.md @@ -0,0 +1,19 @@ +# kernel_version + +Meaning: +- Every os has its kernel, however an os can start with different kernels according to different need. +- kernel_version is a key for users to specify a kernel version. +- If kernel_version is not given by users, it will use the default one. + +Related files: +- $os_dir/vmlinuz => boot/vmlinuz-$kernel_version + - default kernel file(vmlinuz) is a soft link +- $os_dir/boot/vmlinuz-$kernel_version + - real kernel path +- $os_dir/boot/modules-$kernel_version.cgz +- $os_dir/boot/headers-$kernel_version.cgz + +A example $os_dir can be "/srv/os/openeuler/aarch64/20.03" + +usage example: +- submit iperf.yaml testbox=vm-hi1620-2p8g--xxj os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 diff --git a/src/lib/job.cr b/src/lib/job.cr index 7fbc414..88f64ab 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -86,6 +86,7 @@ class Job kernel_append_root kernel_params docker_image + kernel_version ) macro method_missing(call) @@ -134,6 +135,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,8 +295,9 @@ class Job end private def set_kernel_uri + vmlinuz_sub_path = (("#{kernel_version}" == "default") ? "vmlinuz" : "boot/vmlinuz-#{kernel_version}") self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/#{vmlinuz_sub_path}")}" end private def kernel_common_params @@ -315,10 +318,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-#{kernel_version}.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-#{kernel_version}.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? @@ -359,6 +367,10 @@ class Job return basenames end + private def set_kernel_version + self["kernel_version"] ||= "default" + end + private def set_initrds_uri uris = "" -- 2.23.0

2 3

[PATCH v2 lkp-tests 3/4] tests/test-tools: upload the results and logs from mugen
by Li Ping 31 Oct '20

31 Oct '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- tests/test-tools | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/test-tools b/tests/test-tools index cee882aa..dba3d9f8 100755 --- a/tests/test-tools +++ b/tests/test-tools @@ -6,6 +6,9 @@ cd $BENCHMARK_ROOT/test-tools/mugen || exit : "${testsuite:=smoke-testing}" : "${case:=oe_test_dd_001}" +f_log=$BENCHMARK_ROOT/test-tools/mugen/logs +f_result=$BENCHMARK_ROOT/test-tools/mugen/results + set_env() { bash runoet.sh -c @@ -35,3 +38,7 @@ run() } run + +upload_files -t results $f_log +upload_files -t results $f_result + -- 2.23.0

2 1

[PATCH v2 lkp-tests 4/4] bin/lkp-setup-rootfs: fix auto submit makepkg task
by Li Ping 31 Oct '20

31 Oct '20

[why] the define_files will generate the PKGBUILD and when run define_files_auto_pack will automatic run makepkg even though the job is not for makepkg --- bin/lkp-setup-rootfs | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 2890601e..2e50b6a2 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -187,14 +187,14 @@ define_files_pkg_pack() ( local benchmark=${1#pkg/} export benchmark=${benchmark%/PKGBUILD} - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) } # There may be $program-dev or $program-dev-dev in the # "$LKP_SRC/distro/depends" directory, # when the value passed by $define_files is "distro/depends/$program-dev", -# you don't know which script in "makepkg" or "pack-deps" to execute, +# you don't know which script in "cci-makepkg" or "pack-deps" to execute, # so it needs to be handled sparately. # Save the result to # "/srv/initrd/deps/$os/$os_arch/$os_version/$deps_benchmark.cgz", @@ -208,7 +208,7 @@ define_files_deps_pack() [ $suffix == "dev" ] && [ -f "$LKP_SRC/pkg/$pkg_benchmark/PKGBUILD" ] && { ( export benchmark=$pkg_benchmark - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) return 0 } @@ -229,7 +229,7 @@ define_files_auto_pack() [ -f "$LKP_SRC/$path" ] || continue [ ${path##*/} = PKGBUILD ] && { - define_files_pkg_pack $path + define_files_pkg_pack $path } [ ${path#distro/depends/} != $path ] && { define_files_deps_pack $path @@ -251,8 +251,9 @@ chmod_root_ssh while true; do set_tbox_wtmp 'running' . $job_script define_files - define_files_auto_pack $define_files - if [ "$os_mount" != "initramfs" ]; then + if [ "$suite" == 'cci-makepkg'] || [ "$suite" == 'cci-depends' ];then + define_files_auto_pack $define_files + elif [ "$os_mount" != "initramfs" ]; then wget_install_cgz $initrd_deps wget_install_cgz $initrd_pkg fi -- 2.23.0

2 1

[PATCH compass-ci] tutorial: add document License
by Bai Jing 31 Oct '20

31 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- doc/tutorial.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/tutorial.md b/doc/tutorial.md index 03f1342..c335900 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -1,3 +1,11 @@ +#Copyright (c) 2020 Huawei Technologies Co., Ltd. +#Your copying, use, modification, and distribution of this document is subject to the +#Creative Commons Attribution-ShareAlike 4.0 International Public License ("CC BY-SA 4.0"). +#For ease of understanding, you can visit https://creativecommons.org/licenses/by-sa/4.0/ +#to learn about the outline of CC BY-SA 4.0(but not a substitute). +#The complete agreement of CC BY-SA 4.0 can be obtained from the following website: +#https://creativecommons.org/licenses/by-sa/4.0/legalcode. + # Compass-CI 用户指南 ## 简介 -- 2.23.0

4 3

[PATCH v4 lkp-tests] fix: btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 31 Oct '20

31 Oct '20

[why] mkfs.btrfs "command not found" when os_mount=initramfs [how] it will user mkfs.ext4 because mkfs.ext4 is common, other mkfs.*** is not considered Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- lib/bootstrap.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..44a7bdef 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,8 +456,16 @@ mount_rootfs() ROOTFS_DIR=/opt/rootfs mkdir -p $ROOTFS_DIR mount $rootfs_partition $ROOTFS_DIR || { - mkfs.btrfs -f $rootfs_partition - mount $rootfs_partition $ROOTFS_DIR + + if has_cmd mkfs.btrfs; then + mkfs.btrfs -f $rootfs_partition + mount $rootfs_partition $ROOTFS_DIR + else + has_cmd mkfs.ext4 && { + mkfs.ext4 -F $rootfs_partition + mount $rootfs_partition $ROOTFS_DIR + } + fi } mkdir -p $ROOTFS_DIR/tmp CACHE_DIR=$ROOTFS_DIR/tmp -- 2.23.0

2 1