- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] modify README
by Li Ping 30 Nov '20

30 Nov '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- README.md | 162 ++++++++---------------------------------------------- 1 file changed, 24 insertions(+), 138 deletions(-) diff --git a/README.md b/README.md index 644e9ba..80e0766 100644 --- a/README.md +++ b/README.md @@ -1,74 +1,32 @@ -# Compass-CI 用户指南 +## Compass-CI -## 简介 +### 关于Compass-CI -### 概念 +Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上游开源软件（来自 Github, Gitee, Gitlab 等托管平台）的测试服务、登录服务、故障辅助定界服务和基于历史数据的分析服务--。Compass-CI 基于开源软件PR进行自动化测试(包括构建测试，软件包自带用例测试等)，共同构建一个开放、完整的开源软件生态测试系统。 -Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上游开源软件（来自 Github, Gitee, Gitlab 等托管平台）的测试服务、登录服务、故障辅助定界服务和基于历史数据的分析服务。通过 Compass-CI，社区开发者可以将开源软件快速引入 openEuler 社区，并补充更多的测试用例，共同构建一个开放、完整的开源软件生态系统。 - -### 功能描述 +### 功能介绍 - **测试服务** - 支持开发者基于本地设备开发，往 github 提交代码，Compass-CI 自动获取代码开展测试，并向开发者反馈测试结果。 + 使用Compass-CI 基于开源软件 PR 触发[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… - **调测环境登录** - Compass-CI 提供 SSH 登录能力，测试过程中如果测出问题，开发者可根据需要登录环境进行调测。 + 使用 SSH [登录测试环境进行调测](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… - **测试结果分析** - Compass-CI 记录历史测试结果，对外提供 web 及命令行接口，支持开发者针对已有的测试结果进行分析，挖掘影响测试结果的因素。 - -- **辅助定位** - - Compass-CI 在测试过程中可以自动识别错误信息，触发基于 git tree 的测试，找出引入问题模块的变化点。 - - -### 架构简介 -![Compass-CI架构](doc/pictures/compass-ci-architecture.png) -- **接入层** - - 成为衔接 Compass-CI 服务于开发者的纽带，同时提供内部设备管理的平台，提供 web 门户查阅测试结果、项目清单、ssh 环境登录。 - -- **服务层** - - 提供测试服务、环境登录、结果分析、辅助定界、用户项目注册能力，为本项目的主要功能。 - -- **支撑层** - - 作为项目主体功能实现，详细描述资源编排、部署、构建、测试等详细实现。 - -- **数据层** - - 为项目提供数据处理服务，包括测试结果格式化及归档。 - -- **资源层** - - 提供硬件设备，作为Compass-CI 服务部署的设备以及测试的物理设备、虚拟机、容器资源；提供测试过程中依赖软件仓库及依赖仓库、部署服务器。 - + 通过 [web](https://compass-ci.openeuler.org) 接口，对历史测试结果进行分析和比较。 -### 应用场景 - -- **聚合开发者测试用例** - - 开发者向代码托管平台提交代码、测试用例、测试工具时，Compass-CI 自动获取提交的代码开展构建测试，同时获取开发者编写到开源软件包的测试用例进行自动化测试，并反馈测试结果。 +- **测试结果复现** -- **登录环境随时调测** - - 测试过程中，发现有 Bug 时，可随时提供调测资源服务，登录到环境进行复现、调试。 - -- **快照数据分析对比** - - 测试过程中，全面监控系统运行信息（CPU/MEM/IO/网络等），对测试过程中的数据做快照归档，提供多次测试之间快照数据分析对比能力，协助开发者对测试结果开展分析，找出影响测试结果的因素。 - -- **辅助定界** + Compass-CI 把测试过程中的各项环境参数保存在任务结果输出的job.yaml文件中，开发者可以通过重新提交job.yaml复现测试结果。 - 测试过程中，发现有 Bug 时，自动触发 Regression 机制，找出首次引入问题的 Commit 信息。 - +- **辅助定位** + Compass-CI 可以识别自动化构建测试过程中的错误，触发基于 git tree 的测试，找出引入问题的commit。 ## 使用 Compass-CI > 您只需要先注册自己的仓库，当您的仓库有 commit 提交时，构建测试会自动执行，并且可在我们的网站中查看结果。 @@ -89,97 +47,25 @@ Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上 web: https://compass-ci.openeuler.org/jobs +## Source +我们最新的，最完整的源码都存放在 [Gitee](https://gitee.com/wu_fengguang/compass-ci.git) 上，Fork 我们吧！ -## 高级功能 - -### 添加 OS 支持 - -制作一个 initramfs 启动的 cgz 镜像，当系统内核启动时，直接从打包的 cgz 镜像中导出 rootfs，在内存中展开文件系统，利用磁盘的高速缓存机制让系统直接在内存中读写文件，以提升系统的 I/O 性能。 - -#### 操作步骤 - - 1. 获取对应 os 版本的 rootfs（以 openEuler 为例） - - 通过 docker 获取 rootfs - - 1) 下载 openEuler 官方提供的 docker 镜像压缩包 - - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) 加载 docker 镜像 - ```bash - docker load -i openEuler-docker.aarch64 - ``` - - 3) 启动 openEuler 容器 - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) 拷贝 docker 的 rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - - 通过 qemu.img(qcow2格式)获取 rootfs (以 openEuler 为例) - - - 1) 下载 openEuler 官方网站提供的 qcow2 格式镜像 - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… - ``` - 2) 使用 {compass-ci}/container/qcow2rootfs 制作 rootfs - ```bash - cd {compass-ci}/container/qcow2rootfs - ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs - ``` - 2. 定制 rootfs - 1. 使用 chroot 命令切换到 rootfs 中(此步骤需要 root 权限) - ```bash - chroot openEuler-rootfs - ``` - 2. 根据个人需要安装并配置服务 - - a. 修改 root 密码 - b. 配置 ssh 服务 - c. 检查系统时间 - d. 如果使用 docker 制作 osimage 还需要以下操作： - > 1. 安装所需版本内核 - > 2. 从 centos 官方网站下载内核 rpm 包 - > 3. 使用 yum 进行安装 - > 4. 删除 docker 环境变量文件 - - 3. 退出 rootfs，并打包 - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` -#### FAQ -1. 日志报错 “Unable to mount root fs on unknown-block” - - 问题现象 - ```bash - [ 0.390437] List of all partitions: - [ 0.390806] No filesystem could mount root, tried: - [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) - ... - [ 0.399404] Memory Limit: none - [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- - ``` - - 解决方法 - - 1）启动内存不足，增加内存可解决。 - 2）内核文件权限不足，给予 644 权限。 - -2. 系统运行缓慢 - - - 问题现象 +## Contributing to Compass-CI + +我们非常欢迎有新的贡献者，我们也很乐意为我们的贡献者提供一些指导 + +## Website - 打包镜像体积过大，会消耗很大内存。 - - 解决方法 +所有的测试结果，已加入的开源软件清单，历史测试查询比较都可以在我们的官网 [Website](https://compass-ci.openeuler.org) 上找到。 - 建议用户根据具体需要对 rootfs 进行裁剪。 +## 加入我们 +您可以通过以下的方式加入我们： + - 您可以加入我们的 [mailing list](https://mailweb.openeuler.org/postorius/lists/compass-ci.openeuler.or… + - 您可以加入我们每周四下午四点 16：00 的项目例会（会议链接将会通过邮件发送给您） -## 呼吁合作 +欢迎您跟我们一起： - 增强 git bisect 能力 - 增强数据分析能力 - 增强数据结果可视化能力 -- 2.23.0

1 0

[PATCH compass-ci] doc: how to apply account
by Wu Fengguang 30 Nov '20

30 Nov '20

- fix title - fix format - improve logic - fix broken link Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 40 +++++++++++++++++++++++++++++++++ doc/manual/apply-ssh-account.md | 38 ------------------------------- 2 files changed, 40 insertions(+), 38 deletions(-) create mode 100644 doc/manual/apply-account.md delete mode 100644 doc/manual/apply-ssh-account.md diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md new file mode 100644 index 0000000..804b6ca --- /dev/null +++ b/doc/manual/apply-account.md @@ -0,0 +1,40 @@ +# apply account + +## step 1: send email to apply account + +Email template: + +--- + From: {{ David Rientjes <rientjes(a)google.com> }} + To: compass-ci(a)qq.com + Subject: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ attach your ssh pubkey file(s), e.g. ~/.ssh/id_rsa.pub }} + +--- + +- please replace the 3 parts in {{ }} with your information. +- my oss commit: a git commit URL that has your contribution. + +## step 2: receive an email + +Which contains account information for you: + + - my_email + - my_name + - my_uuid + - SCHED_HOST + - SCHED_PORT + +## step 3: one-time setup + +Follow the email to setup your local environment: + + - clone git repos + - write ~/.config/compass-ci/default/account.yaml + +## step 4: try it out + +Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… diff --git a/doc/manual/apply-ssh-account.md b/doc/manual/apply-ssh-account.md deleted file mode 100644 index 160657e..0000000 --- a/doc/manual/apply-ssh-account.md +++ /dev/null @@ -1,38 +0,0 @@ -# apply account - -## steps -1. send apply account email - - send apply account email to: compass-ci(a)qq.com - attention: - email subject: - apply account - email body: - example: - my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - attachment: - ssh pubkey file - - - email example: - - To: compass-ci(a)qq.com - Subject: apply account - - my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - - attach your ssh pub key to the email as attachment - -2. receive email from compass-ci(a)qq.com - the email contains following information: - - my_email - - my_name - - my_uuid - - SCHED_HOST - - SCHED_PORT - -3. local environment configuration - follow steps in the email to finish the following configuration - - setup default yaml - ~/.config/compass-ci/default/account.yaml - -for more information: how to use submit - https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/compass-ci… -- 2.23.0

2 3

[PATCH v3 compass-ci] scheduler: fix the paths to find cluster config file
by Ren Wen 30 Nov '20

30 Nov '20

To find cluster config file at *lab-z9/cluster/* first, *lkp-tests/cluster/* otherwise. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/constants.cr | 3 +++ src/scheduler/submit_job.cr | 14 +++++++++----- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index 1b37333..1afdda0 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -24,6 +24,9 @@ OS_HTTP_PORT = (ENV.has_key?("OS_HTTP_PORT") ? ENV["OS_HTTP_PORT"] : 8000).to_i3 SRV_OS = "/srv/os" SRV_INITRD = "/srv/initrd" +CCI_REPOS = "/c" +LAB_REPO = "lab-z9" + INITRD_HTTP_PREFIX = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" OS_HTTP_PREFIX = "http://#{OS_HTTP_HOST}:#{OS_HTTP_PORT}" SCHED_HTTP_PREFIX = "http://#{SCHED_HOST}:#{SCHED_PORT}" diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index c8d0875..7e1ac27 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -15,7 +15,7 @@ class Sched else cluster_config = get_cluster_config(cluster_file, job.lkp_initrd_user, - job.os_arch) + job.os_arch).not_nil! return submit_cluster_job(job, cluster_config) end rescue ex @@ -147,12 +147,16 @@ class Sched @es.set_job_content(job) end - # get cluster config using own lkp_src cluster file, - # a hash type will be returned def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) - cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)) + + cluster_file_paths = [ + Path.new(CCI_REPOS, LAB_REPO, "cluster", cluster_file), + Path.new(lkp_src, "cluster", cluster_file) + ] + cluster_file_paths.each do |f| + return YAML.parse(File.read(f)) if File.file?(f) + end end def get_commit_date(job) -- 2.23.0

1 0

[PATCH lkp-tests] job.rb: disable submit warn
by Wei Jihui 30 Nov '20

30 Nov '20

[why] it has a warn when submit job: WARN -- skip non-executable */lkp-tests/monitors/plain/wait [how] lkp-tests/monitors/plain/wait is link to bin/event/wakeup but bin/event/wakeup does not exist. so we will do not check link file. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index 8d1f7469..78604d64 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -115,6 +115,8 @@ def __create_programs_hash(glob, lkp_src) next if path =~ /\.[0-9]+$/ unless File.executable?(path) + next if File.symlink?(path) + log_warn "skip non-executable #{path}" unless path =~ /\.cr$/ next end -- 2.23.0

2 1

[PATCH v7 compass-ci] answerback-email: fix bugs and optimize
by Luan Shengde 30 Nov '20

30 Nov '20

1. set default option '-h' [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined 'nil' for my_email when execute answerback-email with no '-e' or '-f', it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' 3. option parse error [why] if add no required value for option, the option will use the field that follows as its value. example: ./answerback-email.rb -e -n name Not a standard format email: -n. [how] add value check for options that need a value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 49 ++++++++++++++++++-- 1 file changed, 44 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..825d815 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -94,19 +94,45 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + unless email_address =~ /[^@]+@[\d\w]+\.[\w\d]+/ + message = "Not a standard format email: #{email_address}.\n\n" + puts message + + return false + end + my_info['my_email'] = email_address end opts.on('-n name', '--name name', 'appoint name') do |name| + unless name =~ /^\w[\w\d ]+/ + message = "Name should only contains letters, digits and spaces\n\n" + puts message + + return false + end + stdin_info['my_name'] = name end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| + unless File.exist? pub_key_file + message = "File not found: #{pub_key_file}.\n\n" + puts message + + return false + end stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + unless File.exist? email_file + message = "Email file not found: #{email_file}.\n\n" + puts message + + return false + end mail_content = Mail.read(email_file) init_info(mail_content, email_info, my_info) end @@ -121,12 +147,16 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + message = "-l: bad value #{value}, please use y|n\n\n" + puts message + + return false end end @@ -136,6 +166,15 @@ options = OptionParser.new do |opts| end end +# if no option specified, set default option '-h' +# if both '-e' and '-f' are not specified, will clear the ARGV, +# and use '-h' instead +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +191,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v2 lkp-tests] spec/submit_spec.rb: update submit_job
by Hu Xuejiao 30 Nov '20

30 Nov '20

[why] Originally submit_job is too complicated, it need to be simplified. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit_spec.rb | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 4096cb73..2ea689ec 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -20,10 +20,12 @@ def traverse_file(output_dir) end end -def submit_job(submit_item) - submit_yaml_path = File.join("#{LKP_SRC}/spec/submit", submit_item) - Dir.glob("#{submit_yaml_path}/*.yaml").each do |yaml_file| - output_dir = File.join(submit_yaml_path, File.basename(yaml_file, '.yaml')) +def submit_job() + Dir.glob("#{LKP_SRC}/spec/submit/*/*.yaml").each do |yaml_file| + next unless File.exist?(yaml_file) + + path = File.dirname(yaml_file) + output_dir = File.join(path, File.basename(yaml_file, '.yaml')) submit_cmd = [ "#{LKP_SRC}/sbin/submit", '-o', output_dir, @@ -36,24 +38,4 @@ def submit_job(submit_item) end end -describe 'submit job spec' do - it 'link jobs spec' do - submit_job('link_jobs') - end - - it 'link matrix' do - submit_job('matrix') - end - - it 'separate yaml spec' do - submit_job('separate_yaml') - end - - it 'job on fail' do - submit_job('job_on_fail') - end - - it 'merge yaml' do - submit_job('merge_yaml') - end -end +submit_job -- 2.23.0

2 1

[PATCH v6 compass-ci] answerback-email: fix bugs and optimizi
by Luan Shengde 30 Nov '20

30 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined nil for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' 3. option parse error [why] if add no required value for option, it will use the field that follow as the value, example: ./answerback-email.rb -e -n name Not a standard format email: -n. [how] add value check for options that need a value Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 49 ++++++++++++++++++-- 1 file changed, 44 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..825d815 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -94,19 +94,45 @@ options = OptionParser.new do |opts| opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| + unless email_address =~ /[^@]+@[\d\w]+\.[\w\d]+/ + message = "Not a standard format email: #{email_address}.\n\n" + puts message + + return false + end + my_info['my_email'] = email_address end opts.on('-n name', '--name name', 'appoint name') do |name| + unless name =~ /^\w[\w\d ]+/ + message = "Name should only contains letters, digits and spaces\n\n" + puts message + + return false + end + stdin_info['my_name'] = name end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| + unless File.exist? pub_key_file + message = "File not found: #{pub_key_file}.\n\n" + puts message + + return false + end stdin_info['new_ssh_pubkey'] = File.read(pub_key_file).strip end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| + unless File.exist? email_file + message = "Email file not found: #{email_file}.\n\n" + puts message + + return false + end mail_content = Mail.read(email_file) init_info(mail_content, email_info, my_info) end @@ -121,12 +147,16 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + message = "-l: bad value #{value}, please use y|n\n\n" + puts message + + return false end end @@ -136,6 +166,15 @@ options = OptionParser.new do |opts| end end +# if no option specified, set default option '-h' +# if both '-e' and '-f' are not specified, will clear the ARGV, +# and use '-h' instead +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +191,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v3 compass-ci 4/4] user-client/maintain: send report mail
by Liu Yinsi 30 Nov '20

30 Nov '20

call $CCI_SRC/lib/mail_client.rb to mail. 'LOG' is a file => walk-test.report, job result will write in it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 42a8b97..2ec95ce 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,17 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def mail_report + message = File.read(LOG).chomp + data = " + subject: os rootfs test report + to: #{EMAIL_ADDRESS} + body: #{message}" + + email = MailClient.new + email.send_mail(data) +end + def write_report(report_all) File.open(LOG, 'a') do |f| log = Logger.new(f, 'weekly') -- 2.23.0

1 0

[PATCH v3 compass-ci 3/4] user-client/maintain: write job result into report
by Liu Yinsi 30 Nov '20

30 Nov '20

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- user-client/maintain/walk-os-test/walk-os-iperf-test | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 4f3bc88..42a8b97 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,16 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def write_report(report_all) + File.open(LOG, 'a') do |f| + log = Logger.new(f, 'weekly') + log.formatter = proc { |_severity, datetime, _progname, msg| + "[#{datetime}]: #{msg}\n" + } + log.info(report_all) + end +end + def monitor(query, actions, timeout) monitor = Monitor.new monitor.overrides = query -- 2.23.0

1 0

[PATCH v3 compass-ci 2/4] user-client/maintain: get job result by monitor and es query
by Liu Yinsi 30 Nov '20

30 Nov '20

1. monitor job, start time => job_state is 'boot', end time => job_state is 'extract_finished'; 2. query es by field 'job_state' and 'result_root'. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/walk-os-iperf-test | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 091b01b..4f3bc88 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -36,6 +36,46 @@ OS_MOUNT = %w[cifs initramfs].freeze EMAIL_ADDRESS = (ENV['USER']).to_s.freeze +def monitor(query, actions, timeout) + monitor = Monitor.new + monitor.overrides = query + monitor.action = actions + monitor.run(timeout) + monitor +end + +def monitor_state(job_id) + time_list = [] + + %w[boot extract_finished].each do |state| + result_array = monitor({ 'job_id' => job_id, 'job_state' => state }, { 'stop' => true }, '1800').result + time_list.push(Time.parse(result_dict[0]['time'])) unless result_array.empty? + end + + time_list +end + +def es_search(job_id) + es_list = [] + + es = ESQuery.new + result_dict = es.query_by_id(job_id) + %w[job_state result_root].each do |field| + es_list.push(result_dict[field]) unless result_dict.nil? + end + + es_list +end + +def find_by_id(job_id) + start_time, end_time = monitor_state(job_id) + cost_time = end_time - start_time unless end_time.nil? + + job_state, result_root = es_search(job_id) + + return start_time, end_time, cost_time.to_i.floor, job_state, result_root +end + def run_qemu Process.fork do %x(#{ENV['CCI_SRC']}/providers/my-qemu.sh >/dev/null 2>&1) -- 2.23.0

1 0