November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] spec/submit: add job yaml for multi-args test
by Bai Jing 24 Nov '20

24 Nov '20

[test cmd] respec $LKP_SRC/spec/submit_spec.rb [output] lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml Signed-off-by: Bai Jing <799286817(a)qq.com> --- spec/submit/matrices-args/unixbench-spec.yaml | 10 +++ ...unixbench-spec-1-300-whetstone-double.yaml | 78 +++++++++++++++++++ .../unixbench-spec-3-300-shell1.yaml | 78 +++++++++++++++++++ spec/submit_spec.rb | 4 + 4 files changed, 170 insertions(+) create mode 100644 spec/submit/matrices-args/unixbench-spec.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml diff --git a/spec/submit/matrices-args/unixbench-spec.yaml b/spec/submit/matrices-args/unixbench-spec.yaml new file mode 100644 index 00000000..39447528 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec.yaml @@ -0,0 +1,10 @@ +suite: unixbench +category: benchmark + +runtime: 300s + +nr_task test: + - 1 whetstone-double + - 3 shell1 + +unixbench: diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml new file mode 100644 index 00000000..abc7fd53 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml @@ -0,0 +1,78 @@ +--- +suite: unixbench +category: benchmark +runtime: 300 +unixbenc: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '1' +test: whetstone-double +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kms: +boot-tim: +uptim: +iosta: +heartbea: +vmsta: +numa-numasta: +numa-vmsta: +numa-meminf: +proc-vmsta: +proc-sta: +meminf: +slabinf: +interrupt: +lock_sta: +latency_stat: +softirq: +bdi_dev_mappin: +diskstat: +nfssta: +cpuidl: +cpufreq-stat: +sched_debu: +perf-sta: +mpsta: +perf-profil: +cpufreq_governo: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: whetstone-double + nr_task: '1' + cpufreq_governor: {} +monitors: + kms: + boot-tim: + uptim: + iosta: + heartbea: + vmsta: + numa-numasta: + numa-vmsta: + numa-meminf: + proc-vmsta: + proc-sta: + meminf: + slabinf: + interrupt: + lock_sta: + latency_stat: + softirq: + bdi_dev_mappin: + diskstat: + nfssta: + cpuidl: + cpufreq-stat: + sched_debu: + perf-sta: + mpsta: + perf-profil: diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml new file mode 100644 index 00000000..559a04d2 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml @@ -0,0 +1,78 @@ +--- +suite: unixbench +category: benchmark +runtime: 300 +unixbenc: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '3' +test: shell1 +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kms: +boot-tim: +uptim: +iosta: +heartbea: +vmsta: +numa-numasta: +numa-vmsta: +numa-meminf: +proc-vmsta: +proc-sta: +meminf: +slabinf: +interrupt: +lock_sta: +latency_stat: +softirq: +bdi_dev_mappin: +diskstat: +nfssta: +cpuidl: +cpufreq-stat: +sched_debu: +perf-sta: +mpsta: +perf-profil: +cpufreq_governo: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: shell1 + nr_task: '3' + cpufreq_governor: {} +monitors: + kms: + boot-tim: + uptim: + iosta: + heartbea: + vmsta: + numa-numasta: + numa-vmsta: + numa-meminf: + proc-vmsta: + proc-sta: + meminf: + slabinf: + interrupt: + lock_sta: + latency_stat: + softirq: + bdi_dev_mappin: + diskstat: + nfssta: + cpuidl: + cpufreq-stat: + sched_debu: + perf-sta: + mpsta: + perf-profil: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ebebad5d 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'matrices-args spec' do + submit_job('matrices-args') + end end -- 2.23.0

1 0

[PATCH v2 compass-ci] optimize document for apply account
by Luan Shengde 24 Nov '20

24 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index d718d0d..1844c86 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -36,20 +36,12 @@ for external user email subject: apply account email body: - my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + example: + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… attachment: ssh pubkey file - it will return email with my_email, my_name, my_uuid if successfully applied account -2. prepare to submit job - execute on user's local host - - config default yaml file: - ~/.config/compass-ci/default/account.yaml - - prepare job.yaml - - submit job - -3. apply account email format - example: + - email example: To: compass-ci(a)qq.com Subject: apply account @@ -58,5 +50,19 @@ for external user # We'll validate whether the URL contains your email. # for example, my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - # attach your ssh pub key to the email as attachment + +2. successfully applied account email + it will return an email with following infomation: + - my_email + - my_name + - my_uuid + - SCHED_HOST + - SCHED_PORT + +3. environment configuration + follow steps in the email to finish the following configuration + - setup default yaml + ~/.config/compass-ci/default/account.yaml + - download lkp-tests and dependencies + - prepare to submit jobs -- 2.23.0

2 2

[PATCH v4 compass-ci 2/2] lib/job.cr: verify account information
by Wu Zhende 24 Nov '20

24 Nov '20

Verify account information when submitting a job. Must key:my_email, my_name, my_uuid. Check whether the information is the same as that stored in the ES database. Also need update pub_key. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 51 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 4bef329..785daea 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "../scheduler/elasticsearch_client" struct JSON::Any def []=(key : String, value : String) @@ -58,8 +59,11 @@ class Job return if @hash["id"] == "#{id}" end + @es = Elasticsearch::Client.new @hash["id"] = JSON::Any.new("#{id}") + @account_info = Hash(String, JSON::Any).new check_required_keys() + check_account_info() set_defaults() end @@ -152,7 +156,7 @@ class Job set_kernel_append_root() set_kernel_params() set_lkp_server() - set_sshr_port() + set_sshr_info() set_queue() end @@ -183,12 +187,34 @@ class Job end end - private def set_sshr_port + private def set_sshr_info return unless self["sshd"]? self["sshr_port"] = ENV["SSHR_PORT"] self["sshr_port_base"] = ENV["SSHR_PORT_BASE"] self["sshr_port_len"] = ENV["SSHR_PORT_LEN"] + + return if @account_info["found"]? == false + + update_sshd + end + + private def update_sshd + sshd = @hash["sshd"].as_h? || Hash(String, JSON::Any).new() + pub_key = sshd["pub_key"]?.to_s + update_account_my_pub_key(pub_key) + + sshd["pub_key"] = @account_info["my_ssh_pubkey"] + @hash["sshd"] = JSON.parse(sshd.to_json) + end + + private def update_account_my_pub_key(pub_key) + my_ssh_pubkey = @account_info["my_ssh_pubkey"].as_a + return if pub_key.empty? || my_ssh_pubkey.includes?(pub_key) + + my_ssh_pubkey << JSON::Any.new(pub_key) + @account_info["my_ssh_pubkey"] = JSON.parse(my_ssh_pubkey.to_json) + @es.update_account(JSON.parse(@account_info.to_json), self["my_email"].to_s) end private def set_os_dir @@ -271,6 +297,9 @@ class Job id suite testbox + my_email + my_name + my_uuid ] private def check_required_keys @@ -281,6 +310,20 @@ class Job end end + private def check_account_info + error_msg = "Failed to verify the account. Please check your configuration" + account_info = @es.get_account(self["my_email"]) + raise account_info unless account_info.is_a?(JSON::Any) + + @account_info = account_info.as_h + + raise error_msg if @account_info["found"]? == false + raise error_msg unless self["my_name"] == @account_info["my_name"].to_s + raise error_msg unless self["my_uuid"] == @account_info["my_uuid"] + + @hash.delete("my_uuid") + end + private def initialized? initialized_keys = [] of String @@ -302,6 +345,8 @@ class Job "SCHED_HOST", "SCHED_PORT"] + initialized_keys -= ["my_uuid"] + initialized_keys.each do |key| return false unless @hash.has_key?(key) end @@ -441,7 +486,7 @@ class Job else program_version = "latest" end - + deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}/#{program}.cgz" pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}/#{program_version}.cgz" -- 2.23.0

1 0

[PATCH v4 compass-ci 1/2] scheduler/elasticsearch_client.cr: add two account functions
by Wu Zhende 24 Nov '20

24 Nov '20

add two functions about account 1. get_account 2. update_account Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/scheduler/elasticsearch_client.cr | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/scheduler/elasticsearch_client.cr b/src/scheduler/elasticsearch_client.cr index ec6de0d..7c92204 100644 --- a/src/scheduler/elasticsearch_client.cr +++ b/src/scheduler/elasticsearch_client.cr @@ -69,6 +69,24 @@ class Elasticsearch::Client return job end + def get_account(my_email : String) + query = {:index => "accounts", :type => "_doc", :id => my_email} + response = JSON.parse({"_id" => my_email, "found" => false}.to_json) + return response unless @client.exists(query) + + @client.get_source(query) + end + + def update_account(account_content : JSON::Any, my_email : String) + return @client.update( + { + :index => "accounts", :type => "_doc", + :id => my_email, + :body => {:doc => account_content} + } + ) + end + private def create(job_content : JSON::Any, job_id : String) return @client.create( { -- 2.23.0

1 0

[PATCH v6 lkp-tests] pkg: add mysql
by Lin Jiaxin 24 Nov '20

24 Nov '20

sysbench-mysql need to test with mysql8.0.17 Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- ...00-mysql-add-fstack-protector-strong.patch | 20 ++++++ pkg/mysql/PKGBUILD | 60 +++++++++++++++++ pkg/mysql/my.cnf | 67 +++++++++++++++++++ 3 files changed, 147 insertions(+) create mode 100644 pkg/mysql/0000-mysql-add-fstack-protector-strong.patch create mode 100644 pkg/mysql/PKGBUILD create mode 100644 pkg/mysql/my.cnf diff --git a/pkg/mysql/0000-mysql-add-fstack-protector-strong.patch b/pkg/mysql/0000-mysql-add-fstack-protector-strong.patch new file mode 100644 index 00000000..23c07c87 --- /dev/null +++ b/pkg/mysql/0000-mysql-add-fstack-protector-strong.patch @@ -0,0 +1,20 @@ +diff --git a/cmake/build_configurations/compiler_options.cmake b/cmake/build_configurations/compiler_options.cmake +index fecf2a2..f3e012f 100644 +--- a/cmake/build_configurations/compiler_options.cmake ++++ b/cmake/build_configurations/compiler_options.cmake +@@ -43,6 +43,7 @@ IF(UNIX) + # Default GCC flags + IF(CMAKE_COMPILER_IS_GNUCC) + SET(COMMON_C_FLAGS "-fno-omit-frame-pointer") ++ SET(COMMON_C_FLAGS "-fstack-protector-strong") + # Disable inline optimizations for valgrind testing to avoid false positives + IF(WITH_VALGRIND) + STRING_PREPEND(COMMON_C_FLAGS "-fno-inline ") +@@ -57,6 +58,7 @@ IF(UNIX) + ENDIF() + IF(CMAKE_COMPILER_IS_GNUCXX) + SET(COMMON_CXX_FLAGS "-std=c++14 -fno-omit-frame-pointer") ++ SET(COMMON_CXX_FLAGS "-fstack-protector-strong") + # Disable inline optimizations for valgrind testing to avoid false positives + IF(WITH_VALGRIND) + STRING_PREPEND(COMMON_CXX_FLAGS "-fno-inline ") diff --git a/pkg/mysql/PKGBUILD b/pkg/mysql/PKGBUILD new file mode 100644 index 00000000..8869e62b --- /dev/null +++ b/pkg/mysql/PKGBUILD @@ -0,0 +1,60 @@ +pkgname=mysql +pkgver=8.0.17 +pkgrel=3 +arch=('aarch64' 'x86_64') +url="http://www.mysql.com" +license=('GPLv2') +source=("https://cdn.mysql.com/archives/mysql-8.0/mysql-boost-$pkgver.tar.gz" + "0000-mysql-add-fstack-protector-strong.patch" + "https://mirrors.huaweicloud.com/kunpeng/archive/openEuler/patch/database/my…" + "my.cnf") +md5sums=('7472a25d91973cbba5d8a8f176a3080b' '6e853850efbd6f887ac3e13b1bf99cc4' 'f13eb11de53616017f7428cc06d4e242' '7bb0372ec4e9281813a9afc67c6cfc20') + +prepare() +{ + cd $srcdir/$pkgname-$pkgver + patch -p1 < ../0000-mysql-add-fstack-protector-strong.patch +} + +build() +{ + build_getcpu + build_mysql +} + +build_mysql() +{ + cd $srcdir/$pkgname-$pkgver + + CMAKE=cmake + grep -sqF "CentOS Linux release 7" /etc/centos-release && CMAKE=cmake3 + + $CMAKE . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql/data -DSYSCONFDIR=/etc \ + -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATED_STORAGE_ENGINE=1 \ + -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_MYISAM_STORAGE_ENGINE=1 \ + -DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 -DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_general_ci \ + -DWITH_EMBEDDED_SERVER=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=./boost -DFORCE_INSOURCE_BUILD=1 + make -j8 +} + +build_getcpu() +{ + cd $srcdir + tar -zxvf getcpu.tgz + + cd getcpu || exit + make +} + + +package() +{ + mkdir -p $pkgdir/getcpu + cp getcpu/getcpu-hi.ko $pkgdir/getcpu/ + + mkdir -p $pkgdir/etc + cp my.cnf $pkgdir/etc/ + + cd $srcdir/$pkgname-$pkgver + make DESTDIR=$pkgdir install +} diff --git a/pkg/mysql/my.cnf b/pkg/mysql/my.cnf new file mode 100644 index 00000000..9732af40 --- /dev/null +++ b/pkg/mysql/my.cnf @@ -0,0 +1,67 @@ +[mysqld_safe] +log-error=/data/mysql/log/mysql.log +pid-file=/data/mysql/run/mysqld.pid +[mysqldump] +quick +[mysql] +[client] +default-character-set=utf8mb4 +socket=/data/mysql/run/mysql.sock + +[mysqld] +socket=/data/mysql/run/mysql.sock +tmpdir=/data/mysql/tmp +datadir=/data/mysql/data +default_authentication_plugin=mysql_native_password +port=3309 +user=mysql +memlock=on +bind-address=0.0.0.0 +skip-grant-tables + +lc-messages=en_US +lc-messages-dir=/usr/local/mysql/share + +innodb_page_size=64k +max_connections=4000 +back_log=2048 +performance_schema=OFF +max_prepared_stmt_count=128000 + +#file +innodb_file_per_table +innodb_log_file_size=2048M +innodb_log_files_in_group=32 +innodb_open_files=10000 +table_open_cache_instances=64 + +#buffers +innodb_buffer_pool_size=128G +innodb_buffer_pool_instances=64 +innodb_log_buffer_size=2048M + +#tune +sync_binlog=0 +innodb_flush_log_at_trx_commit=0 +innodb_use_native_aio=1 +innodb_spin_wait_delay=160 +innodb_sync_spin_loops=20 +innodb_flush_method=O_DIRECT +innodb_io_capacity=30000 +innodb_io_capacity_max=40000 +innodb_lru_scan_depth=9000 +innodb_page_cleaners=1 +innodb_spin_wait_pause_multiplier=25 + +#perf special +innodb_flush_neighbors=0 +innodb_write_io_threads=1 +innodb_read_io_threads=1 +innodb_purge_threads=1 + +sql_mode=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_AUTO_VALUE_ON_ZERO,STRICT_ALL_TABLES + +#skop_log_bin +ssl=0 + +table_open_cache=15000 -- 2.23.0

1 0

[PATCH compass-ci] container/os-cifs: add the container to share /srv/os
by Xiao Shenwei 24 Nov '20

24 Nov '20

[why] our os-cifs container will mount three dir, /srv/os, /srv/initrd, /srv/result in order to reduce the coupling, need split them [how] one folder mount by one container [usage] mount -t cifs o guest //ip/os /tmp/os mount -t cifs o guest //ip/os-rw /tmp/os-rw Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/os-cifs/Dockerfile | 8 ++++---- container/os-cifs/smb.conf | 26 ++------------------------ container/os-cifs/start | 7 +++---- 3 files changed, 9 insertions(+), 32 deletions(-) diff --git a/container/os-cifs/Dockerfile b/container/os-cifs/Dockerfile index 66f978d..c22ce35 100644 --- a/container/os-cifs/Dockerfile +++ b/container/os-cifs/Dockerfile @@ -4,16 +4,16 @@ FROM alpine:edge -RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories +MAINTAINER Xiao Shenwei <xiaoshenwei96(a)163.com> +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN adduser -u 1090 -D lkp - RUN apk add --update \ samba-common-tools \ samba-client \ samba-server \ - bash \ - && rm -rf /var/cache/apk/* + bash +RUN rm -rf /var/cache/apk/* COPY ./smb.conf /etc/samba/ diff --git a/container/os-cifs/smb.conf b/container/os-cifs/smb.conf index 5ff6841..5e454db 100644 --- a/container/os-cifs/smb.conf +++ b/container/os-cifs/smb.conf @@ -18,6 +18,7 @@ server min protocol = NT1 unix extensions = yes mangled names = no + [os] path = /srv/os/ comment = os @@ -28,6 +29,7 @@ # that can only be accessed by root. force user = root force group = root + [os-rw] path = /srv/os/ comment = os-rw @@ -36,27 +38,3 @@ public = yes force user = root force group = root -[initrd] - path = /srv/initrd/ - comment = initrd - browseable = yes - writable = yes - public = yes - force user = lkp - force group = lkp -[osimage] - path = /srv/initrd/ - comment = initrd - browseable = yes - writable = yes - public = yes - force user = lkp - force group = lkp -[result] - path = /srv/result/ - comment = result - browseable = yes - writable = yes - public = yes - force user = lkp - force group = lkp diff --git a/container/os-cifs/start b/container/os-cifs/start index f42c71c..5e08238 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -11,12 +11,11 @@ lsmod | grep -q "^cifs\s" || { docker_rm os-cifs cmd=( - docker run -dt + docker run + -d -p 445:445 - -v /srv/os:/srv/os -v /etc/localtime:/etc/localtime:ro - -v /srv/initrd:/srv/initrd - -v /srv/result:/srv/result + -v /srv/os:/srv/os --name os-cifs --restart=always os-cifs -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] scheduler/elasticsearch_client.cr: add two account functions
by Wu Zhende 24 Nov '20

24 Nov '20

add two functions about account 1. get_account 2. update_account Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/scheduler/elasticsearch_client.cr | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/scheduler/elasticsearch_client.cr b/src/scheduler/elasticsearch_client.cr index ec6de0d..7c92204 100644 --- a/src/scheduler/elasticsearch_client.cr +++ b/src/scheduler/elasticsearch_client.cr @@ -69,6 +69,24 @@ class Elasticsearch::Client return job end + def get_account(my_email : String) + query = {:index => "accounts", :type => "_doc", :id => my_email} + response = JSON.parse({"_id" => my_email, "found" => false}.to_json) + return response unless @client.exists(query) + + @client.get_source(query) + end + + def update_account(account_content : JSON::Any, my_email : String) + return @client.update( + { + :index => "accounts", :type => "_doc", + :id => my_email, + :body => {:doc => account_content} + } + ) + end + private def create(job_content : JSON::Any, job_id : String) return @client.create( { -- 2.23.0

2 1

[PATCH compass-ci] srv-http/start: add os images data download directory
by Cui Yili 24 Nov '20

24 Nov '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/srv-http/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start b/container/srv-http/start index b56fe4d..8c8ca0b 100755 --- a/container/srv-http/start +++ b/container/srv-http/start @@ -14,6 +14,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result:/srv/result:ro -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro + -v /srv/os:/srv/os:ro -d srv-http ) -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] lib/job.cr: check account info when submit job
by Wu Zhende 24 Nov '20

24 Nov '20

When a job submit, check it's account info. Must key:my_email, my_name, my_uuid. Check whether the information is the same as that stored in the ES database. Also need update pub_key. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 51 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 4bef329..785daea 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "../scheduler/elasticsearch_client" struct JSON::Any def []=(key : String, value : String) @@ -58,8 +59,11 @@ class Job return if @hash["id"] == "#{id}" end + @es = Elasticsearch::Client.new @hash["id"] = JSON::Any.new("#{id}") + @account_info = Hash(String, JSON::Any).new check_required_keys() + check_account_info() set_defaults() end @@ -152,7 +156,7 @@ class Job set_kernel_append_root() set_kernel_params() set_lkp_server() - set_sshr_port() + set_sshr_info() set_queue() end @@ -183,12 +187,34 @@ class Job end end - private def set_sshr_port + private def set_sshr_info return unless self["sshd"]? self["sshr_port"] = ENV["SSHR_PORT"] self["sshr_port_base"] = ENV["SSHR_PORT_BASE"] self["sshr_port_len"] = ENV["SSHR_PORT_LEN"] + + return if @account_info["found"]? == false + + update_sshd + end + + private def update_sshd + sshd = @hash["sshd"].as_h? || Hash(String, JSON::Any).new() + pub_key = sshd["pub_key"]?.to_s + update_account_my_pub_key(pub_key) + + sshd["pub_key"] = @account_info["my_ssh_pubkey"] + @hash["sshd"] = JSON.parse(sshd.to_json) + end + + private def update_account_my_pub_key(pub_key) + my_ssh_pubkey = @account_info["my_ssh_pubkey"].as_a + return if pub_key.empty? || my_ssh_pubkey.includes?(pub_key) + + my_ssh_pubkey << JSON::Any.new(pub_key) + @account_info["my_ssh_pubkey"] = JSON.parse(my_ssh_pubkey.to_json) + @es.update_account(JSON.parse(@account_info.to_json), self["my_email"].to_s) end private def set_os_dir @@ -271,6 +297,9 @@ class Job id suite testbox + my_email + my_name + my_uuid ] private def check_required_keys @@ -281,6 +310,20 @@ class Job end end + private def check_account_info + error_msg = "Failed to verify the account. Please check your configuration" + account_info = @es.get_account(self["my_email"]) + raise account_info unless account_info.is_a?(JSON::Any) + + @account_info = account_info.as_h + + raise error_msg if @account_info["found"]? == false + raise error_msg unless self["my_name"] == @account_info["my_name"].to_s + raise error_msg unless self["my_uuid"] == @account_info["my_uuid"] + + @hash.delete("my_uuid") + end + private def initialized? initialized_keys = [] of String @@ -302,6 +345,8 @@ class Job "SCHED_HOST", "SCHED_PORT"] + initialized_keys -= ["my_uuid"] + initialized_keys.each do |key| return false unless @hash.has_key?(key) end @@ -441,7 +486,7 @@ class Job else program_version = "latest" end - + deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}/#{program}.cgz" pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}/#{program_version}.cgz" -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] stats/mysql-server: remove redundant whitespace
by Lu Kaiyi 24 Nov '20

24 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/mysql-server | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/stats/mysql-server b/stats/mysql-server index d3f3fd79..776f3f2e 100755 --- a/stats/mysql-server +++ b/stats/mysql-server @@ -16,9 +16,9 @@ while (line = STDIN.gets) # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 when /\[\s*\d+\%\]/ - results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') - results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') - results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') + results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') + results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') + results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') end end -- 2.23.0

2 1

2024

2023

2022

2021

2020

Compass-ci November 2020