- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib/git: use work_dir replace repo for GitCommit class
by Cao Xueliang 26 Jan '21

26 Jan '21

If we use "git -C /srv/git/#{repo}.git", we need mount "/srv/git", but sometimes we have clone the git repo, so we don't need mount "/srv/git". Now, we just need to pass the work_dir to the GitCommit class. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git.rb | 4 ++-- lib/git_bisect.rb | 2 +- lib/mail_bisect_result.rb | 8 +++++++- 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/lib/git.rb b/lib/git.rb index db97c47..9751dcf 100644 --- a/lib/git.rb +++ b/lib/git.rb @@ -2,8 +2,8 @@ # wrap common git commands class GitCommit - def initialize(repo, commit) - @git_prefix = "git -C /srv/git/#{repo}.git" + def initialize(work_dir, commit) + @git_prefix = "git -C #{work_dir}" @commit = commit end diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index e577afd..5d13074 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -100,10 +100,10 @@ class GitBisect bisect_error = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_error_id(@error_id) all_errors = Utils.obt_errors(@work_dir, first_bad_commit) first_bad_commit_result_root = Utils.obt_result_root_by_commit(first_bad_commit) - FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) return Hash['repo' => @upstream_repo, 'commit' => first_bad_commit, + 'work_dir' => @work_dir, 'bisect_error' => bisect_error, 'all_errors' => all_errors, 'pkgbuild_repo' => @pkgbuild_repo, diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 209a28d..637c008 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -12,12 +12,13 @@ require_relative 'assign_account_client' class MailBisectResult def initialize(bisect_info) @repo = bisect_info['repo'] + @work_dir = bisect_info['work_dir'] @commit_id = bisect_info['commit'] @all_errors = bisect_info['all_errors'] @bisect_error = bisect_info['bisect_error'] @pkgbuild_repo = bisect_info['pkgbuild_repo'] @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] - @git_commit = GitCommit.new(@repo, @commit_id) + @git_commit = GitCommit.new(@work_dir, @commit_id) @to = @git_commit.author_email # now send mail to review @bcc = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' @@ -26,6 +27,7 @@ class MailBisectResult def create_send_email send_report_mail(compose_mail) send_account_mail + rm_work_dir end def compose_mail @@ -80,4 +82,8 @@ class MailBisectResult apply_account = AutoAssignAccount.new(user_info) apply_account.send_account end + + def rm_work_dir + FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) + end end -- 2.23.0

1 0

[PATCH compass-ci] lib/es_jobs: fix error when search nothing
by Wu Zhende 26 Jan '21

26 Jan '21

[Before] es-jobs suite=test Traceback (most recent call last): 3: from /home/wuzhende/compass-ci/sbin/es-jobs:55:in `<main>' 2: from /home/wuzhende/compass-ci/lib/es_jobs.rb:167:in `output' 1: from /home/wuzhende/compass-ci/lib/es_jobs.rb:137:in `query_jobs_state' /home/wuzhende/compass-ci/lib/es_jobs.rb:99:in `initialize_result_hash': undefined method `each' for nil:NilClass (NoMethodError) [After] es-jobs suite=test No query result is found: {"suite"=>"test"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/es_jobs.rb | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/es_jobs.rb b/lib/es_jobs.rb index c35b182..eb3f70d 100644 --- a/lib/es_jobs.rb +++ b/lib/es_jobs.rb @@ -13,7 +13,8 @@ require_relative './es_query' # deal jobs search from es class ESJobs def initialize(es_query, my_refine = [], fields = [], stats_filter = []) - @jobs = query_jobs_from_es(es_query) + @es_query = es_query + @jobs = query_jobs_from_es @refine = my_refine @fields = fields @stats_filter = stats_filter @@ -21,9 +22,9 @@ class ESJobs set_jobs_summary end - def query_jobs_from_es(items) + def query_jobs_from_es es = ESQuery.new(ES_HOST, ES_PORT) - result = es.multi_field_query items + result = es.multi_field_query @es_query jobs = result['hits']['hits'] jobs.map! { |job| job['_source'] } return jobs @@ -164,6 +165,10 @@ class ESJobs end def output + if @jobs.empty? + puts "No query result is found: #{@es_query}" + return + end @result = query_jobs_state(@jobs) @result['kvcount'] = @result['kvcount'].sort.to_h @result['raw.id'] = @result['raw.id'].sort.to_h -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: the repo should be in our test coverage
by Wu Fengguang 26 Jan '21

26 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index b5430e7..3e253d0 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -27,8 +27,9 @@ Email template: - please replace the 3 parts in {{ }} with your information. - the email name should be in English or Chinese Pinyin, as: "David Rientjes" in the template. -- my oss commit: a git commit URL that has your contribution. It should be in a - well known git repo that's registered in [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos) +- my oss commit: a git commit URL that has your name and email. + The git repo should be in our test coverage and registered in + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). ## step 2: receive an email -- 2.23.0

1 0

[PATCH v6 compass-ci] lib/sched_client.rb: add some common methods for provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

[why] we have some providers: ipxe, container, libvirt all of them need register testbox info and consume job from scheduler [how] each method corresponds to a interface, so ensuring the independence of them. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/sched_client.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/lib/sched_client.rb b/lib/sched_client.rb index fec4b14..93f9ccd 100644 --- a/lib/sched_client.rb +++ b/lib/sched_client.rb @@ -16,4 +16,37 @@ class SchedClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/submit_job") resource.post(job_json) end + + # scheduler API /boot.:boot_type/:parameter/:value + # boot_type parameter + # 1 ipxe mac + # 2 container hostname + # 3 libvirt mac + def consume_job(boot_type, parameter, value) + RestClient.get "http://#{@host}:#{@port}/boot.#{boot_type}/#{parameter}/#{value}" + end + + def register_mac2host(hostname, mac) + RestClient.put( + "http://#{@host}:#{@port}/set_host_mac?hostname=#{hostname}&mac=#{mac}", {} + ) + end + + def register_host2queues(hostname, queues) + RestClient.put( + "http://#{@host}:#{@port}/set_host2queues?host=#{hostname}&queues=#{queues}", {} + ) + end + + def delete_mac2host(mac) + RestClient.put( + "http://#{@host}:#{@port}/del_host_mac?mac=#{mac}", {} + ) + end + + def delete_host2queues(hostname) + RestClient.put( + "http://#{@host}:#{@port}/del_host2queues?host=#{hostname}", {} + ) + end end -- 2.23.0

1 0

[PATCH lab-z9] add hosts-file of taishan200-2280-2s64p-256g--a5 which include NMVe SSD
by Zhang Dewan 26 Jan '21

26 Jan '21

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- hosts/taishan200-2280-2s64p-256g--a5 | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a5 diff --git a/hosts/taishan200-2280-2s64p-256g--a5 b/hosts/taishan200-2280-2s64p-256g--a5 new file mode 100644 index 0000000..fc9a594 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a5 @@ -0,0 +1,26 @@ +memory: 256G +nr_hdd_partitions: 1 +nr_ssd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399981306c5 +ssd_partitions: + - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43957 + - /dev/disk/by-id/nvme-HWE36P43012M000N_032BEJ10GC000328 +mac_addr: + - 84:46:fe:26:d3:47 + - 84:46:fe:26:d3:48 + - 84:46:fe:26:d3:49 + - 84:46:fe:26:d3:4a + - ac:b3:b5:1e:b4:ab + - ac:b3:b5:1e:b4:ac + - ac:b3:b5:1e:b4:ad + - ac:b3:b5:1e:b4:ae + - ac:b3:b5:1e:ba:9b + - ac:b3:b5:1e:ba:9c + - ac:b3:b5:1e:ba:9d + - ac:b3:b5:1e:ba:9e +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.1.5 -- 2.23.0

2 2

[PATCH v5 compass-ci] lib/sched_client.rb: add some common methods for provider
by Xiao Shenwei 26 Jan '21

26 Jan '21

[why] we have some providers: ipxe, container, libvirt all of them need register testbox info and consume job from scheduler Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/sched_client.rb | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/lib/sched_client.rb b/lib/sched_client.rb index fec4b14..a81ffbb 100644 --- a/lib/sched_client.rb +++ b/lib/sched_client.rb @@ -16,4 +16,41 @@ class SchedClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/submit_job") resource.post(job_json) end + + # scheduler API /boot.:boot_type/:parameter/:value + # boot_type parameter + # 1 ipxe mac + # 2 container hostname + # 3 libvirt mac + def consume_job(boot_type, parameter, value) + RestClient.get "http://#{@host}:#{@port}/boot.#{boot_type}/#{parameter}/#{value}" + end + + def register_mac2host(hostname, mac) + @hostname = hostname + @mac = mac + RestClient.put( + "http://#{@host}:#{@port}/set_host_mac?hostname=#{@hostname}&mac=#{@mac}", {} + ) + end + + def register_host2queues(hostname, queues) + @hostname = hostname + @queues = queues + RestClient.put( + "http://#{@host}:#{@port}/set_host2queues?host=#{@hostname}&queues=#{@queues}", {} + ) + end + + def delete_mac2host + RestClient.put( + "http://#{@host}:#{@port}/del_host_mac?mac=#{@mac}", {} + ) + end + + def delete_host2queues + RestClient.put( + "http://#{@host}:#{@port}/del_host2queues?host=#{@hostname}", {} + ) + end end -- 2.23.0

2 2

[PATCH v2 compass-ci] container: expose/report error when check modules exist
by Liu Yinsi 26 Jan '21

26 Jan '21

[why] sparrow/2-network/nfs: modprobe nfs modprobe nfsd sparrow/2-network/cifs: modprobe cifs when locally deploy compass-ci, the above codes have modprobe nfs, nfsd, cifs, no need to do again when execute sparrow/4-docker/buildall(deploy containers), exit and report error message to function callers is better. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 3 ++- container/os-cifs/start | 3 ++- container/os-nfs/start | 6 ++++-- container/result-cifs/start | 3 ++- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 77558a1..c70b318 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm initrd-cifs diff --git a/container/os-cifs/start b/container/os-cifs/start index 5e08238..6df167f 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm os-cifs diff --git a/container/os-nfs/start b/container/os-nfs/start index 8b8ff5f..3b0ebb5 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -5,10 +5,12 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^nfs\s" || { - sudo modprobe nfs + echo "[ERROR] nfs module is not loaded in the Docker host's kernel (try: sudo modprobe nfs)" + exit } lsmod | grep -q "^nfsd\s" || { - sudo modprobe nfsd + echo "[ERROR] nfsd module is not loaded in the Docker host's kernel (try: sudo modprobe nfsd)" + exit } # Several nfs servers running will cause conflict, so rm an old before running a new nfs server docker. diff --git a/container/result-cifs/start b/container/result-cifs/start index 2d6acba..3992cdd 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -5,7 +5,8 @@ . $CCI_SRC/container/defconfig.sh lsmod | grep -q "^cifs\s" || { - sudo modprobe cifs + echo "[ERROR] cifs module is not loaded in the Docker host's kernel (try: sudo modprobe cifs)" + exit } docker_rm result-cifs -- 2.23.0

2 4

[PATCH compass-ci] sparrow/3-code: set params
by Liu Yinsi 26 Jan '21

26 Jan '21

before: deploy compass-ci client and server in one machine. after: not only deploy compass-ci client and server in one machine, but also on different servers. [why] for user locally deploy compass-ci, sometimes need to deploy client in other machine, so add $1. it's interrelated with the follow patch: [PATCH lkp-tests] setup/simplify-ci: delete default value Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 2ba9662..00a2e0f 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -4,6 +4,9 @@ server_name=$(hostname | cut -f1 -d.) server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + +[ -n "$1" ] && server_ip=$1 + : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH v2 lkp-tests] lkp-bootstrap.service: fix killmode error
by Wang Chenglong 26 Jan '21

26 Jan '21

[Error] Welcome to openEuler 20.09! /usr/lib/systemd/system/lkp-bootstrap.service:11: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the ser vice. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. [How] config the lkp-bootstrap.service: KillMode=control-group Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service index 842c4aef8..a9eb5152a 100644 --- a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service +++ b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service @@ -8,7 +8,7 @@ Type=forking Environment=HOME=/root ExecStart=/etc/init.d/lkp-bootstrap IgnoreSIGPIPE=no -KillMode=none +KillMode=control-group TasksMax=infinity [Install] -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: add introduction
by Wu Fengguang 26 Jan '21

26 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index a0b2c82..b5430e7 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -1,5 +1,15 @@ # apply account +You'll need an account to submit jobs. The account is mainly for tracking your resource consumptions. +We have limited pool of testboxes, so there have to be some usage control. + +We offer free accounts to OSS (Open Source Software) contributors. +You can prove yourself as OSS contributor by providing "my oss commit" in the below email. + +We offer free accounts to our collaborators too. If you are one of our +collaborators, you should know the collaboration channel to apply account. +In this case you just need provide name, email, ssh pubkey and purpose. + ## step 1: send email to apply account Email template: @@ -16,8 +26,9 @@ Email template: --- - please replace the 3 parts in {{ }} with your information. -- the email name should be in english, as: "David Rientjes" in the template. -- my oss commit: a git commit URL that has your contribution. +- the email name should be in English or Chinese Pinyin, as: "David Rientjes" in the template. +- my oss commit: a git commit URL that has your contribution. It should be in a + well known git repo that's registered in [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos) ## step 2: receive an email @@ -29,7 +40,7 @@ Which contains account information for you: ## step 3: one-time setup -**Follow the email** to setup your local environment: +**Follow instructions in the email** to setup your local environment: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml @@ -38,10 +49,3 @@ Which contains account information for you: Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -## notice: - -In case that you received a failed email with message as below: - - The repo url for your commit is not in our upstream-repo list. - -Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… -- 2.23.0

1 0