- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/build-config dir write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..d5e8ad5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,33 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? || @hash["cci-depends"]? || @hash["build-pkg"]? + mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + common_dir = "#{mount_type}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + if @hash["pkgbuild_repo"].to_s =~ /(packages|community)\/\// + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-2] + else + package_name = @hash["pkgbuild_repo"].to_s.split("/")[-1] + end + package_dir = ",/initrd/build-pkg/#{common_dir}/#{package_name}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 1

[PATCH compass-ci] modify options for auto submit api-avx2neon.yaml
by Li Ping 19 Dec '20

19 Dec '20

Signed-off-by: Li Ping <1477412247(a)qq.com> --- sbin/auto_submit.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index d0f4fbe..e1eb39b 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -1,5 +1,5 @@ a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 api-avx2neon.yaml +- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- 2.23.0

1 0

[PATCH v1 compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/config dir read and write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 19 ++++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..85159f5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,27 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? or @hash["cci-depends"]? or @hash["build-pkg"]? + common_dir = "#{os_mount}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + package_dir = ",/initrd/build-pkg/#{common_dir}/#{@hash["pkgbuild_repo"].split("/")[-1]}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end - self["upload_dirs"] = "#{result_root}#{package_dir}" + return package_dir + end + + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

2 3

[PATCH lkp-tests] lib/monitor: get host from job
by Wu Zhende 19 Dec '20

19 Dec '20

If @job is exist and monitor_url is not configured, set monitor_url based on SCHED_HOST Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index f65cfccf1..283cd788d 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -24,7 +24,6 @@ class Monitor # 1 means timeout exit @exit_status_code = 0 @defaults = {} - load_default @result = [] @stop_query = {} @reason = nil @@ -33,6 +32,10 @@ class Monitor def load_default return unless @monitor_url == '' + if host = @job["SCHED_HOST"] + return @monitor_url = "ws://#{host}:11310/filter" + end + Dir.glob(['/etc/compass-ci/monitor/*.yaml', "#{ENV['HOME']}/.config/compass-ci/monitor/*.yaml"]).each do |file| next unless File.exist? file @@ -98,6 +101,7 @@ class Monitor def run(timeout = nil) merge_overrides + load_default field_check @query.each do |k, v| -- 2.23.0

1 0

[PATCH compass-ci] modify assign-account-email
by Li Ping 19 Dec '20

19 Dec '20

[why]: source /root/.bashrc && /root/.bash_profile follow this step can not success. Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 77ed4e8..dccbd40 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -37,7 +37,7 @@ def build_apply_account_email(my_info) git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ${HOME}/.bashrc && source ${HOME}/.bash_profile + source ~/.${SHELL##*/}rc 3) submit job reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md -- 2.23.0

2 1

[PATCH compass-ci] sched: refactor set_upload_dirs and add config dir
by Cao Xueliang 19 Dec '20

19 Dec '20

When build-pkg job has config in job.yaml need support cci/config dir read and write. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/start | 1 + src/lib/job.cr | 17 ++++++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/start b/container/result-webdav/start index bbbbb24..a719806 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -21,6 +21,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ + -v /srv/cci/config/:/srv/cci/config/ result-webdav ) diff --git a/src/lib/job.cr b/src/lib/job.cr index a23475d..5644ddf 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -233,18 +233,25 @@ class Job set_upload_dirs() end - def set_upload_dirs + def get_package_dir + if @hash["cci-makepkg"]? or @hash["cci-depends"]? or @hash["build-pkg"]? + common_dir = "#{os_mount}/#{os}/#{os_arch}/#{os_version}" + end + if @hash["cci-makepkg"]? - package_dir = ",/initrd/pkg" + package_dir = ",/initrd/pkg/#{common_dir}/#{@hash["cci-makepkg"]["benchmark"]}" elsif @hash["cci-depends"]? - package_dir = ",/initrd/deps" + package_dir = ",/initrd/deps/#{common_dir}/#{@hash["cci-depends"]["benchmark"]}" elsif @hash["build-pkg"]? - package_dir = ",/initrd/build-pkg" + package_dir = ",/initrd/build-pkg/#{common_dir}/#{@hash["pkgbuild_repo"].split("/")[-1]}" + package_dir += ",/cci/config" if @hash["config"]? else package_dir = "" end + end - self["upload_dirs"] = "#{result_root}#{package_dir}" + def set_upload_dirs + self["upload_dirs"] = "#{result_root}#{get_package_dir}" end private def set_result_service -- 2.23.0

1 1

[PATCH compass-ci 3/3] rename my_uuid to my_token
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/mail-robot.md | 6 +++--- doc/manual/apply-account.md | 4 ++-- sbin/build-my-info.rb | 6 +++--- sbin/es-accounts-mapping.sh | 2 +- src/delimiter/utils.rb | 4 ++-- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index 06e4e87..d3778e7 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -58,14 +58,14 @@ attahcment first attachment - apply_my_account - my_uuid + my_token generate uuid apply_account invoke ApplyJumperAccount to apply new account with my_info and pub_key my_info: - - my_email - my_name - - my_uuid + - my_email + - my_token apply_jumper_account required data: pub_key complete my_info diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index d9a03c8..7e1a6a6 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -23,9 +23,9 @@ Email template: Which contains account information for you: - my_email my_name - my_uuid + my_email + my_token ## step 3: one-time setup diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 73f7390..d75f767 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -12,12 +12,12 @@ print 'email: ' my_email = $stdin.echo = gets.chomp print 'name: ' my_name = $stdin.echo = gets.chomp -my_uuid = %x(uuidgen).chomp +my_token = %x(uuidgen).chomp my_info = { - 'my_email' => my_email, 'my_name' => my_name, - 'my_uuid' => my_uuid + 'my_email' => my_email, + 'my_token' => my_token } def store_account_info(my_info) diff --git a/sbin/es-accounts-mapping.sh b/sbin/es-accounts-mapping.sh index d5566dc..d04b0f3 100755 --- a/sbin/es-accounts-mapping.sh +++ b/sbin/es-accounts-mapping.sh @@ -18,7 +18,7 @@ else "my_commit_url": { "type": "keyword" }, - "my_uuid": { + "my_token": { "type": "keyword" }, "my_email": { diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index e87bbbe..41803f3 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -101,9 +101,9 @@ module Utils raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info job['suite'] = 'bisect' - job['my_uuid'] = account_info['my_uuid'] job['my_name'] = account_info['my_name'] job['my_email'] = account_info['my_email'] + job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id job.delete('error_ids') @@ -127,7 +127,7 @@ module Utils def create_bisect_log(git_dir) FileUtils.mkdir_p TMP_RESULT_ROOT unless File.exist? TMP_RESULT_ROOT - log_file = File.join(TMP_RESULT_ROOT, "bisect.log") + log_file = File.join(TMP_RESULT_ROOT, 'bisect.log') log_content = parse_bisect_log(git_dir) File.open(log_file, 'w') do |f| log_content.each { |line| f.puts(line) } -- 2.23.0

1 0

[PATCH compass-ci 2/3] assign-account-email: remove .bash_profile usage
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 77ed4e8..ee3b574 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -37,7 +37,7 @@ def build_apply_account_email(my_info) git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ${HOME}/.bashrc && source ${HOME}/.bash_profile + source ${HOME}/.${SHELL##*/}rc 3) submit job reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md -- 2.23.0

1 0

[PATCH compass-ci 1/3] doc/manual: improve install client document
by Wu Fengguang 19 Dec '20

19 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- ...ci\345\256\242\346\210\267\347\253\257.md" | 27 +++++++------------ 1 file changed, 10 insertions(+), 17 deletions(-) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 5a02431..dd115f7 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -2,32 +2,25 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客户端，通过本地安装 lkp-tests 可以手动提交测试任务 +## 申请帐号 + :exclamation: 前提条件：按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… 完成如下动作： - send apply account email - receive email from compass-ci(a)qq.com +- get reply email and follow its instructions to + - setup default config -## Getting started +## 下载安装 lkp-tests -1. setup default config +Run the following command to install/setup lkp-test: - run the following command to add the below setup to default config file -```SHELL - mkdir -p ~/.config/compass-ci/defaults/ - cat >> ~/.config/compass-ci/defaults/${USER}.yaml <<-EOF - my_uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx - my_email: xxx - my_name: xx - EOF -``` -2. download lkp-tests and dependencies - - run the following command to install the lkp-test and effect the configuration ```SHELL git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source ~/.bashrc && source ~/.bash_profile + source ~/.${SHELL##*/}rc ``` -3. submit job - Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +## submit job + +Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

1 0

[PATCH compass-ci] fetch-mail: update Dockerfile
by Luan Shengde 19 Dec '20

19 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile index 0434e09..99abf86 100644 --- a/container/fetch-mail/Dockerfile +++ b/container/fetch-mail/Dockerfile @@ -5,6 +5,7 @@ FROM centos:latest MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive +RUN groupadd -g 1147 mailer RUN useradd -u 1144 -g 1147 mailer RUN yum update -y RUN yum install -y procmail fetchmail -- 2.23.0

1 0