- Compass-ci - mailweb.openeuler.org

review star
by Wu Fengguang 25 Mar '21

25 Mar '21

Luan Shengde is our review star, thanks Luan Shengde! wfg@crystal ~/todo/bin% ./review-star Review email records in the last 7 days total review paragraph point people 9 4 19 9 Wu Fengguang 19 9 18 7 Luan Shengde 49 6 6 6 Wei Jihui 21 5 5 6 Liu Shaofei 39 2 2 5 Cao Xueliang 14 4 4 3 Lin Jiaxin 22 3 5 2 Li Yuanchao 3 0 0 2 Xu Xijian 8 3 5 2 Bai Jing 20 2 2 1 Wang Chenglong 17 3 5 1 Wang Yong 9 1 0 1 Lu Weitao 7 2 3 1 Cui Yili 3 0 0 0 Zhang Dewan 4 1 1 0 Yu Chuan 14 0 0 0 Wu Zhende 11 1 1 0 Li Ping 9 0 0 0 Liu Yinsi

1 0

[PATCH v5 compass-ci] container/web-backend: add API /get_table_error
by Lu Weitao 25 Mar '21

25 Mar '21

[Example] curl -X GET 'localhost:10002/get_job_error?group_id=wcl_ansible-openeuler-03-10' return: { "filter":{"group_id":"wcl_ansible-openeuler-03-10"}, "attributes":["job_id","error_id","error_message","link_to_result"], "objects":[ { "job_id": "crystal.1354921", "error_id": "ansible_test.error.Unable-to-start-service-httpd-Job-for-httpdservice-xxx.fail", "error_message": "{\"changed\": false, \"msg\": \"Unable to start service httpd: Job for httpd.service failed because a timeout was exceeded.}", "link_to_result": "http://172.17.0.1:11300/result/ansible_test/2021-03-10/vm-2p16g--wcl1/opene…", ate/crystal.1354921" }, { "job_id": "crystal.1354929", ... }, ... ] } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/web-backend/web-backend | 18 +++++++++ src/lib/web_backend.rb | 65 +++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/container/web-backend/web-backend b/container/web-backend/web-backend index adb91e0..c7d1ec6 100755 --- a/container/web-backend/web-backend +++ b/container/web-backend/web-backend @@ -170,4 +170,22 @@ end # } get '/get_jobs_summary' do group_jobs_stats(params) + +# GET /get_job_error?suite=virttest&tbox_group=vm-2p8g +# Response +# { +# "filter": {"group_id":"wcl_ansible-openeuler-03-10"}, +# "attributes": ["job_id", "error_id", "error_message", "link_to_result"], +# "objects": [ +# { +# "job_id": "crystal.1354921", +# "error_id": "ansible_test.error.Unable-to-start-service-httpd-Job-for-httpdservice-failed.fail", +# "error_message": "{\"changed\": false, \"msg\": \"Unable to start service httpd: Job for httpd.service failed because a timeout was exceeded.}", +# "link_to_result": "http://172.17.0.1:11300/result/ansible_test/2021-03-10/vm-2p16g--wcl1/opene…" +# }, +# ... +# ] +# } +get '/get_job_error' do + get_job_error(params) end diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 40471e0..7f26b9c 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -12,6 +12,7 @@ require "#{CCI_SRC}/lib/compare.rb" require "#{CCI_SRC}/lib/constants.rb" require "#{CCI_SRC}/lib/es_query.rb" require "#{CCI_SRC}/lib/matrix2.rb" +require "#{CCI_SRC}/lib/params_group.rb" require "#{CCI_SRC}/lib/compare_data_format.rb" UPSTREAM_REPOS_PATH = ENV['UPSTREAM_REPOS_PATH'] || '/c/upstream-repos' @@ -613,4 +614,68 @@ def group_jobs_stats(params) return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'group jobs table error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] + +# ------------------------------------------------------------------------------------------- +# job error table like: +# job_id error_id error_message link to result +# ------------------------------------------------------------------------------------- +# crystal.630608 "stderr.xxx" "messag:xxxx" https://$host:$port/$result_root +# ... +# ------------------------------------------------------------------------------------------- + +def get_job_error(params) + begin + body = job_error_body(params) + rescue StandardError => e + warn e.message + return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get error table error'] + end + + [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] +end + +def job_error_body(params) + error_objects = get_error_objects(params) + { + filter: params, + attributes: ['job_id', 'error_id', 'error_message', 'link_to_result'], + objects: error_objects, + }.to_json +end + +def get_error_objects(filter_items) + error_objs = [] + + job_list = get_job_list(filter_items) + job_list.each do |job| + error_obj = get_error_from_job(job) + error_objs << error_obj unless error_obj.empty? + end + + error_objs +end + +def get_job_list(items) + es = ESQuery.new(ES_HOST, ES_PORT) + query_results = es.multi_field_query(items) + + extract_jobs_list(query_results['hits']['hits']) +end + +# get all error_id from one job +def get_error_from_job(job) + job_error_obj = {} + job['stats'].each do |metric, value| + next unless metric.end_with?('.message') + + result_host = ENV['SRV_HTTP_RESULT_HOST'] || SRV_HTTP_RESULT_HOST + result_port = ENV['SRV_HTTP_RESULT_PORT'] || SRV_HTTP_RESULT_PORT + error_id = metric.sub('.message', '.fail') + job_error_obj['id'] = job['id'] + job_error_obj['error_id'] = error_id + job_error_obj['error_message'] = value + job_error_obj['link_to_result'] = value + end + + job_error_obj end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code/dev-env: set vars defaults values
by Liu Yinsi 25 Mar '21

25 Mar '21

[why] add vars and defaults values is easy to management. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f16cf80..b2c1814 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -34,12 +34,33 @@ server_name=$(hostname | cut -f1 -d.) : ${DOCKER_REGISTRY_PORT:=5001} : ${LOCAL_SEND_MAIL_PORT:=11311} : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} +: ${GIT_SERVER:=$server_ip} +: ${SCHED_HTTPS_HOST:=$server_ip} +: ${SCHED_HTTPS_PORT:=11306} +: ${DOCKER_PROXY_HOST:=$server_ip} +: ${DOCKER_PROXY_PORT:=3128} +: ${LKP_CGI_PORT:=3000} +: ${JUMPER_HOST:=$server_ip} +: ${JUMPER_PORT:=10000} +: ${MAIL_HOST:=$server_ip} +: ${MAIL_PORT:=10001} +: ${MONITOR_HOST:=$server_ip} +: ${FETCH_MAIL_DIR:=/srv/cci/Maildir} +: ${FETCH_MAIL_BOX:=.compass-ci} +: ${GIT_DAEMON_HOST:=$server_ip} +: ${GIT_MIRROR_HOST:=$server_ip} +: ${REDIS_PORT:=6379} +: ${MAILDIR:=/srv/cci/Maildir/.compass-ci} +: ${SENT_MAILDIR:=/srv/cci/Maildir/.sent} + mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF SCHED_HOST: $sched_host SCHED_PORT: $sched_port lab: $lab +SCHED_HTTPS_HOST: $SCHED_HTTPS_HOST +SCHED_HTTPS_PORT: $SCHED_HTTPS_PORT EOF [[ -f /etc/compass-ci/defaults/upstream-config ]] || { @@ -88,6 +109,21 @@ DOCKER_REGISTRY_PORT: $DOCKER_REGISTRY_PORT LOCAL_SEND_MAIL_PORT: $LOCAL_SEND_MAIL_PORT MAILDIR: /srv/cci/Maildir/.compass-ci LOCAL_ROBOT_EMAIL_ADDRESS: $LOCAL_ROBOT_EMAIL_ADDRESS +DOCKER_PROXY_HOST: $DOCKER_PROXY_HOST +DOCKER_PROXY_PORT: $DOCKER_PROXY_PORT +LKP_CGI_PORT: $LKP_CGI_PORT +JUMPER_HOST: $JUMPER_HOST +JUMPER_PORT: $JUMPER_PORT +MAIL_HOST: $MAIL_HOST +MAIL_PORT: $MAIL_PORT +MONITOR_HOST: $MONITOR_HOST +FETCH_MAIL_DIR: $FETCH_MAIL_DIR +FETCH_MAIL_BOX: $FETCH_MAIL_BOX +GIT_DAEMON_HOST: $GIT_DAEMON_HOST +GIT_MIRROR_HOST: $GIT_MIRROR_HOST +REDIS_PORT: $REDIS_PORT +MAILDIR: $MAILDIR +SENT_MAILDIR: $SENT_MAILDIR EOF mkdir -p /etc/compass-ci/scheduler -- 2.23.0

2 2

[PATCH v2 compass-ci] sparrow/3-code/dev-env: set vars defaults values
by Liu Yinsi 25 Mar '21

25 Mar '21

[why] add vars and defaults values is easy to management. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f16cf80..005e29e 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -34,12 +34,32 @@ server_name=$(hostname | cut -f1 -d.) : ${DOCKER_REGISTRY_PORT:=5001} : ${LOCAL_SEND_MAIL_PORT:=11311} : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} +: ${GIT_SERVER:=$server_ip} +: ${SCHED_HTTPS_HOST:=$server_ip} +: ${SCHED_HTTPS_PORT:=11306} +: ${DOCKER_PROXY_HOST:=$server_ip} +: ${DOCKER_PROXY_PORT:=3128} +: ${LKP_CGI_PORT:=3000} +: ${JUMPER_HOST:=$server_ip} +: ${JUMPER_PORT:=10000} +: ${MAIL_HOST:=$server_ip} +: ${MAIL_PORT:=10001} +: ${MONITOR_HOST:=$server_ip} +: ${FETCH_MAIL_DIR:=/srv/cci/Maildir} +: ${FETCH_MAIL_BOX:=.compass-ci} +: ${GIT_DAEMON_HOST:=$server_ip} +: ${GIT_MIRROR_HOST:=$server_ip} +: ${REDIS_PORT:=6379} +: ${MAILDIR:=/srv/cci/Maildir/.compass-ci} +: ${SENT_MAILDIR:=/srv/cci/Maildir/.sent} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF SCHED_HOST: $sched_host SCHED_PORT: $sched_port lab: $lab +SCHED_HTTPS_HOST: $SCHED_HTTPS_HOST +SCHED_HTTPS_PORT: $SCHED_HTTPS_PORT EOF [[ -f /etc/compass-ci/defaults/upstream-config ]] || { @@ -88,6 +108,21 @@ DOCKER_REGISTRY_PORT: $DOCKER_REGISTRY_PORT LOCAL_SEND_MAIL_PORT: $LOCAL_SEND_MAIL_PORT MAILDIR: /srv/cci/Maildir/.compass-ci LOCAL_ROBOT_EMAIL_ADDRESS: $LOCAL_ROBOT_EMAIL_ADDRESS +DOCKER_PROXY_HOST: $DOCKER_PROXY_HOST +DOCKER_PROXY_PORT: $DOCKER_PROXY_PORT +LKP_CGI_PORT: $LKP_CGI_PORT +JUMPER_HOST: $JUMPER_HOST +JUMPER_PORT: $JUMPER_PORT +MAIL_HOST: $MAIL_HOST +MAIL_PORT: $MAIL_PORT +MONITOR_HOST: $MONITOR_HOST +FETCH_MAIL_DIR: $FETCH_MAIL_DIR +FETCH_MAIL_BOX: $FETCH_MAIL_BOX +GIT_DAEMON_HOST: $GIT_DAEMON_HOST +GIT_MIRROR_HOST: $GIT_MIRROR_HOST +REDIS_PORT: $REDIS_PORT +MAILDIR: $MAILDIR +SENT_MAILDIR: $SENT_MAILDIR EOF mkdir -p /etc/compass-ci/scheduler -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats/build-pkg: update filtering rule
by Lin Jiaxin 25 Mar '21

25 Mar '21

error_ids that are filtered out: /tmp/lkp/clementine-git/src/Clementine/3rdparty/qsqlite/qsql_sqlite.cpp: In function ‘QSqlError qMakeError(sqlite3*, const QString&, QSqlError::ErrorType, int)’: make: *** [makefile.linux:68: makefile.dependencies] Error 1 go: finding github.com/pingcap/errors v0.11.5-0.20201126102027-b0a155152ca3 Cyclomatic Complexity 8 ./include/trace/events/error_report.h:trace_error_report_end etc. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- stats/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stats/build-pkg b/stats/build-pkg index 90f27656..764594b7 100755 --- a/stats/build-pkg +++ b/stats/build-pkg @@ -20,7 +20,7 @@ while (line = STDIN.gets) seqno = '' end next unless in_stderr - next unless line.downcase =~ /error|warning/ + next unless line.downcase =~ /(error|warning):[^:]/ error_ids << build_pkg_error_id(line) end -- 2.23.0

3 2

[PATCH lkp-tests] stats/ansible_test: add module name in error id
by Wang Chenglong 25 Mar '21

25 Mar '21

before: error.Failed-to-install-some-of-the-specified-packages: 1 after: error.[msg].Failed-to-install-some-of-the-specified-packages: 1 Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index d6acad1ab..3d3b0b0cb 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -2,18 +2,18 @@ require 'json' -def output_error(error_msg) +def output_error(error_msg, error_key) return if error_msg.nil? if error_msg.is_a? Array error_msg.each do |i| error_id = common_error_id i - puts "error.#{error_id}: 1" + puts "error.[#{error_key}].#{error_id}: 1" puts "error.#{error_id}.message: #{@ansible_failed_info}" end elsif error_msg.is_a? String error_id = common_error_id error_msg - puts "error.#{error_id}: 1" + puts "error.[#{error_key}].#{error_id}: 1" puts "error.#{error_id}.message: #{@ansible_failed_info}" end end @@ -33,8 +33,8 @@ while (line = STDIN.gets) ansible_failed_json = JSON.parse @ansible_failed_info - output_error ansible_failed_json['msg'] - output_error ansible_failed_json['message'] - output_error ansible_failed_json['cmd'] - output_error ansible_failed_json['failures'] + output_error ansible_failed_json['msg'],'msg' + output_error ansible_failed_json['message'],'message' + output_error ansible_failed_json['cmd'],'cmd' + output_error ansible_failed_json['failures'],'failures' end -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/rpmbuild-pkg: add build way from srpm
by Wang Yong 25 Mar '21

25 Mar '21

original build RPMs from git repo, now support building from SRPM Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/rpmbuild-pkg | 67 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 51 insertions(+), 16 deletions(-) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 256e836bf..90905b220 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -1,6 +1,8 @@ #!/bin/bash # - upstream_repo # - compat_os +# - repo_name +# - repo_addr . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh @@ -8,9 +10,30 @@ : "${compat_os:=budding-openeuler}" [ -n "$upstream_repo" ] || die "upstream_repo is empty" -package_name=${upstream_repo##*/} -rpm_dest="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}/${os_arch}/Packages" -src_rpm_dest="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}/source/Packages" + +dest_dir="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}" + +from_git() +{ + package_name=${upstream_repo##*/} + rpm_dest="${dest_dir}/${os_arch}/Packages" + src_rpm_dest="${dest_dir}/source/Packages" + + init_workspace + download_upstream_repo +} + +from_srpm() +{ + [ -n "$repo_name" ] || die "repo_name is empty" + [ -n "$repo_addr" ] || die "repo_addr is empty" + + rpm_dest="${dest_dir}/${repo_name}/${os_arch}/Packages" + src_rpm_dest="${dest_dir}/${repo_name}/source/Packages" + + install_srpm +} + init_workspace() { @@ -20,30 +43,43 @@ init_workspace() download_upstream_repo() { - git clone "git://$GIT_SERVER/openeuler/${upstream_repo}" || die "clone git repo ${package_name} failed: git://$GIT_SERVER/openeuler/${upstream_repo}" - cd $package_name || exit + local url="git://${GIT_SERVER}/openeuler/${upstream_repo}" + + git clone "$url" || die "clone git repo ${package_name} failed: $url" + cd "$package_name" || exit filelist=$(git ls-files) + for pkgfile in ${filelist[@]} do local dir="SOURCES" - echo $pkgfile | egrep "\.spec$" && dir="SPECS" + + echo "$pkgfile" | grep -E "\\.spec$" && dir="SPECS" mv "$pkgfile" "${HOME}/rpmbuild/${dir}/" done } -build_rpm() +install_srpm() { - local spec_file=${HOME}/rpmbuild/SPECS/$package_name.spec - - # HTTP is proxy cache friendly - sed -i 's/^$Source[^ ]*:[ \t]*$https/\1http/g' `grep http -rl $spec_file` + rpm -i "${repo_addr}/${upstream_repo}" >/dev/null || die "failed to install source rpm: ${repo_addr}/${upstream_repo}" +} +build_rpm() +{ + local spec_dir="${HOME}/rpmbuild/SPECS" + [ -n "$package_name" ] && + { + # HTTP is proxy cache friendly + sed -i 's/^$Source[^ ]*:[ \t]*$https/\1http/g' "$(grep http -rl "$spec_dir/${package_name}.spec")" + } # Install build depends - yum-builddep -y $spec_file || exit + yum-builddep -y "$spec_dir"/*.spec || die "failed to solve dependencies" # Download tar.gz to default path ${HOME}/rpmbuild/SOURCE - spectool -g -R $spec_file || exit + [ -n "$package_name" ] && + { + spectool -g -R "$spec_dir/${package_name}.spec" || die "failed to download source file" + } # Building rpm or srpm packages - rpmbuild -ba $spec_file || exit + rpmbuild -ba "$spec_dir"/*.spec || die "failed to build rpms" } show_rpm_files() @@ -71,7 +107,6 @@ upload_rpm_pkg() done } -init_workspace -download_upstream_repo +[ -n "$repo_name" ] && from_srpm || from_git build_rpm upload_rpm_pkg -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/rpmbuild-pkg: add build way from srpm
by Wang Yong 25 Mar '21

25 Mar '21

original build RPMs from git repo, now support building from SRPM Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/rpmbuild-pkg | 73 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 57 insertions(+), 16 deletions(-) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 256e836bf..282d473fd 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -1,6 +1,8 @@ #!/bin/bash # - upstream_repo # - compat_os +# - repo_name +# - repo_addr . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh @@ -8,9 +10,38 @@ : "${compat_os:=budding-openeuler}" [ -n "$upstream_repo" ] || die "upstream_repo is empty" -package_name=${upstream_repo##*/} -rpm_dest="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}/${os_arch}/Packages" -src_rpm_dest="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}/source/Packages" + +dest_dir="/initrd/rpmbuild-pkg/${os}-${os_version}/${compat_os}" + +from_git() +{ + package_name=${upstream_repo##*/} + rpm_dest="${dest_dir}/${os_arch}/Packages" + src_rpm_dest="${dest_dir}/source/Packages" + + init_workspace + download_upstream_repo +} + +from_srpm() +{ + [ -n "$repo_name" ] || die "repo_name is empty" + [ -n "$repo_addr" ] || die "repo_addr is empty" + + rpm_dest="${dest_dir}/${repo_name}/${os_arch}/Packages" + src_rpm_dest="${dest_dir}/${repo_name}/source/Packages" + + install_srpm +} + +check_flow() +{ + if [ -n "$repo_name" ]; then + from_srpm + else + from_git + fi +} init_workspace() { @@ -20,30 +51,41 @@ init_workspace() download_upstream_repo() { - git clone "git://$GIT_SERVER/openeuler/${upstream_repo}" || die "clone git repo ${package_name} failed: git://$GIT_SERVER/openeuler/${upstream_repo}" - cd $package_name || exit + git clone "git://${GIT_SERVER}/openeuler/${upstream_repo}" || die "clone git repo ${package_name} failed: git://${GIT_SERVER}/openeuler/${upstream_repo}" + cd "$package_name" || exit filelist=$(git ls-files) + for pkgfile in ${filelist[@]} do local dir="SOURCES" - echo $pkgfile | egrep "\.spec$" && dir="SPECS" + + echo "$pkgfile" | grep -E "\\.spec$" && dir="SPECS" mv "$pkgfile" "${HOME}/rpmbuild/${dir}/" done } -build_rpm() +install_srpm() { - local spec_file=${HOME}/rpmbuild/SPECS/$package_name.spec - - # HTTP is proxy cache friendly - sed -i 's/^$Source[^ ]*:[ \t]*$https/\1http/g' `grep http -rl $spec_file` + rpm -i "${repo_addr}/${upstream_repo}" >/dev/null || die "failed to install source rpm: ${repo_addr}/${upstream_repo}" +} +build_rpm() +{ + local spec_dir="${HOME}/rpmbuild/SPECS" + [ -n "$package_name" ] && + { + # HTTP is proxy cache friendly + sed -i 's/^$Source[^ ]*:[ \t]*$https/\1http/g' "$(grep http -rl "$spec_dir/${package_name}.spec")" + } # Install build depends - yum-builddep -y $spec_file || exit + yum-builddep -y "$spec_dir"/*.spec || die "failed to solve dependencies" # Download tar.gz to default path ${HOME}/rpmbuild/SOURCE - spectool -g -R $spec_file || exit + [ -n "$package_name" ] && + { + spectool -g -R "$spec_dir/${package_name}.spec" || die "failed to download source file" + } # Building rpm or srpm packages - rpmbuild -ba $spec_file || exit + rpmbuild -ba "$spec_dir"/*.spec || die "failed to build rpms" } show_rpm_files() @@ -71,7 +113,6 @@ upload_rpm_pkg() done } -init_workspace -download_upstream_repo +check_flow build_rpm upload_rpm_pkg -- 2.23.0

3 6

[PATCH v5 compass-ci 5/5] doc/manual: update test-oss-project.en.md
by Li Yuanchao 24 Mar '21

24 Mar '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- doc/manual/test-oss-project.en.md | 152 ++++++++++++++++-------------- 1 file changed, 81 insertions(+), 71 deletions(-) diff --git a/doc/manual/test-oss-project.en.md b/doc/manual/test-oss-project.en.md index 1b551b5..a3c0781 100644 --- a/doc/manual/test-oss-project.en.md +++ b/doc/manual/test-oss-project.en.md @@ -1,71 +1,81 @@ -# How DO I Use the Compass-CI Platform to Test Open Source Projects? - -This document describes how to use the Compass-CI platform to test open source projects. - -### Adding the URL of the Repository to Be Tested to the upstream-repos Repository - -Perform the following steps to add the information of the code repository to be tested to the **upstream-repos** repository (https://gitee.com/wu_fengguang/upstream-repos) in YAML format: - -1. Fork the upstream-repos repository and clone it to the local host. This document uses the **backlight** repository (https://github.com/baskerville/backlight) as an example. - -![](./../pictures/fork_blacklight.png) - -2. Run the following command to create a file path named with the repository name and its initial letter: - - ``` - mkdir -p b/backlight - ``` - -3. Run the following command to create the **backlight** file in the directory: - - ``` - cd b/backlight - touch backlight - ``` - -4. Run the following command to write the URL of the **backlight** repository to the **backlight** file: - - ``` - vim backlight - ``` - - The format is as follows: - - ``` - --- - url: - - https://github.com/baskerville/backlight - ``` - - > ![](./../public_sys-resources/icon-notice.gif) **Note** - > - > You can refer to the existing file format in the **upstream-repos** repository. Ensure that the formats are consistent. - -5. Run the **Pull Request** command to submit the new **backlight** file to the **upstream-repos** repository. - -### Submitting the Test Task to the Compass-CI Platform - -1. Prepare a test case. - - You can compile and add a test case to the **lkp-tests** repository, or directly use the existing test cases in the **jobs** directory of the **lkp-tests** repository (https://gitee.com/wu_fengguang/lkp-tests) - - * Use the test cases that have been adapted in the repository. - - Use the test cases in the **lkp-tests** repository that meet the requirements. The **iperf.yaml** file is used as an example. The **iperf.yaml** file is a test case that has been adapted. It is stored in the **jobs** directory of the **lkp-tests** repository, and contains some basic test parameters. - - * Compile a test case and add it to the repository. - - For details, see [How To Add Test Cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcas…. - -2. Configure the **auto\_submit.yaml** file and submit the test task. - - You only need to add the following configuration information to the **sbin/auto\_submit.yaml** file in the **compass-ci** repository: - - ``` - b/backlight/backlight: - - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml - ``` - - Submit the modified **auto\_submit.yaml** file to the **compass-ci** repository using Pull Request. Then you can use the Compass-CI platform to test your project. - - For details about how to set parameters in the **auto\_submit.yaml** file, see https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job. +# How Do I Use the Compass-CI Platform to Test Open Source Projects? + +This document describes how to use the Compass-CI platform to test open source projects. + +### Adding the URL of the Repository to Be Tested to the upstream-repos Repository + +Perform the following steps to add the information of the code repository to be tested to the **upstream-repos** repository (https://gitee.com/wu_fengguang/upstream-repos) in YAML format: + +1. Fork the upstream-repos repository and clone it to the local host. This document uses the **backlight** repository (https://github.com/baskerville/backlight) as an example. + +![](./../pictures/fork_blacklight.png) + +2. Run the following command to create a file path named with the repository name and its initial letter: + + ``` + mkdir -p b/backlight + ``` + +3. Run the following command to create the **backlight** file in the directory: + + ``` + cd b/backlight + touch backlight + ``` + +4. Run the following command to write the URL of the **backlight** repository to the **backlight** file: + + ``` + vim backlight + ``` + + The format is as follows: + + ``` + --- + url: + - https://github.com/baskerville/backlight + ``` + + > ![](./../public_sys-resources/icon-notice.gif) **Note** + > + > You can refer to the existing file format in the **upstream-repos** repository. Ensure that the formats are consistent. + +5. Run the **Pull Request** command to submit the new **backlight** file to the **upstream-repos** repository. + +### Submitting the Test Task to the Compass-CI Platform + +1. Prepare a test case. + + You can compile and add a test case to the **lkp-tests** repository, or directly use the existing test cases in the **jobs** directory of the **lkp-tests** repository (https://gitee.com/wu_fengguang/lkp-tests) + + * Use the test cases that have been adapted in the repository. + + Use the test cases in the **lkp-tests** repository that meet the requirements. The **iperf.yaml** file is used as an example. The **iperf.yaml** file is a test case that has been adapted. It is stored in the **jobs** directory of the **lkp-tests** repository, and contains some basic test parameters. + + * Compile a test case and add it to the repository. + + For details, see [How To Add Test Cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcas…. + +2. Configure **DEFAULTS** files in **upstream-repos** repository and submit the test task. + + You only need to add **DEFAULTS** file in the directory of file **backlight** we referred to above, and configure it like: + + ``` + submit: + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml + branches: + - master + - next + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 other-avx2neon.yaml + branches: + - branch_name_a + - branch_name_b + + ``` + + Submit the modified **DEFAULTS** file to the **upstream-repos** repository using Pull Request. Then you can use the Compass-CI platform to test your project. + + For details about how to configure DEFAULTS files, see https://gitee.com/wu_fengguang/upstream-repos/blob/master/README.md. + + For meaning and effect of parameters in the command, see https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job. -- 2.23.0

1 0

[PATCH v5 compass-ci 4/5] doc/manual: update test-oss-project.zh.md
by Li Yuanchao 24 Mar '21

24 Mar '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- doc/manual/test-oss-project.zh.md | 146 ++++++++++++++++-------------- 1 file changed, 78 insertions(+), 68 deletions(-) diff --git a/doc/manual/test-oss-project.zh.md b/doc/manual/test-oss-project.zh.md index aa2cd15..b455c32 100644 --- a/doc/manual/test-oss-project.zh.md +++ b/doc/manual/test-oss-project.zh.md @@ -1,68 +1,78 @@ -# 使用 compass-ci 平台测试开源项目 - -本文介绍了使用 compass-ci 平台测试开源项目的操作。 - -### 添加待测试仓库 URL 到 upstream-repos 仓库 - -执行以下步骤，将想要测试的代码仓信息以 yaml 文件的方式添加到 upstream-repos 仓库（https://gitee.com/wu_fengguang/upstream-repos ）。 - -1. Fork 要测试的代码仓库并 git clone 到本地，本文以 blacklight 仓库（https://github.com/baskerville/backlight ）为例说明。 - -![](./../pictures/fork_blacklight.png) - -2. 执行以下命令，以首字母和仓库名创建文件路径。 - - ``` - mkdir -p b/backlight - ``` - -3. 执行以下命令，在该目录下新建同名文件 backlight。 - ``` - cd b/backlight - touch backlight - ``` - -4. 执行以下命令，将 backlight 仓库 url 信息写入 backlight 文件。 - - ``` - vim backlight - ``` - 内容格式为 - - ``` - --- - url: - - https://github.com/baskerville/backlight - ``` - - >![](./../public_sys-resources/icon-notice.gif) **注意：** - > - >可参考 upstream-repos 仓库中已有文件格式,请保持格式一致。 - -5. 通过 Pull Request 命令将新增的 backlight 文件提交到 blacklight 仓库。 - - -### 提交测试任务到 compass-ci 平台 - -1. 准备测试用例 - - 测试用例可以自己编写并添加到 lkp-tests 仓库，也可以直接使用 lkp-tests 仓库（https://gitee.com/wu_fengguang/lkp-tests ）的 jobs 目录下已有的测试用例。 - - * 使用仓库中已经适配好的测试用例 - 如果 lkp-tests 仓库中正好有你想要的测试用例，你可以直接使用。以 iperf.yaml 文件为例说明如下： - iperf.yaml 是一个已经适配好的测试用例，它位于 lkp-tests 仓库的 jobs 目录下，其中有一些基本的测试参数。 - - * 编写测试用例并添加到仓库 - - 请参考：[如何添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add… ) - -2. 配置 auto_submit.yaml 文件，提交测试任务 - - 你只需要在 compass-ci 仓库下面的 sbin/auto_submit.yaml 文件中添加配置信息，如： - ``` - b/backlight/backlight: - - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml - ``` - 通过 Pull Request 的方式将修改好的 auto_submit.yaml 文件提交到 compass-ci 仓库，就可以使用 compass-ci 测试你的项目了。 - - auto_submit.yaml 文件的参数配置请参考 https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job 。 +# 使用 compass-ci 平台测试开源项目 + +本文介绍了使用 compass-ci 平台测试开源项目的操作。 + +### 添加待测试仓库 URL 到 upstream-repos 仓库 + +执行以下步骤，将想要测试的代码仓信息以 yaml 文件的方式添加到 upstream-repos 仓库（https://gitee.com/wu_fengguang/upstream-repos ）。 + +1. Fork upstream-repos 仓库并 git clone 到本地，本文以 blacklight 仓库（https://github.com/baskerville/backlight ）为例说明。 + +![](./../pictures/fork_blacklight.png) + +2. 执行以下命令，以首字母和仓库名创建文件路径。 + + ``` + mkdir -p b/backlight + ``` + +3. 执行以下命令，在该目录下新建同名文件 backlight。 + ``` + cd b/backlight + touch backlight + ``` + +4. 执行以下命令，将 backlight 仓库 url 信息写入 backlight 文件。 + + ``` + vim backlight + ``` + 内容格式为 + + ``` + --- + url: + - https://github.com/baskerville/backlight + ``` + + >![](./../public_sys-resources/icon-notice.gif) **注意：** + > + >可参考 upstream-repos 仓库中已有文件格式,请保持格式一致。 + +5. 通过 Pull Request 命令将新增的 backlight 文件提交到 upstream-repos 仓库。 + + +### 提交测试任务到 compass-ci 平台 + +1. 准备测试用例 + + 测试用例可以自己编写并添加到 lkp-tests 仓库，也可以直接使用 lkp-tests 仓库（https://gitee.com/wu_fengguang/lkp-tests ）的 jobs 目录下已有的测试用例。 + + * 使用仓库中已经适配好的测试用例 + 如果 lkp-tests 仓库中正好有你想要的测试用例，你可以直接使用。以 iperf.yaml 文件为例说明如下： + iperf.yaml 是一个已经适配好的测试用例，它位于 lkp-tests 仓库的 jobs 目录下，其中有一些基本的测试参数。 + + * 编写测试用例并添加到仓库 + + 请参考：[如何添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add… ) + +2. 配置 upstream-repos仓库中的DEFAULTS文件，提交测试任务 + + 你只需要在上述backlight文件所在目录增加 DEFAULTS 文件并添加配置信息，如： + ``` + submit: + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml + branches: + - master + - next + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 other-avx2neon.yaml + branches: + - branch_name_a + - branch_name_b + + ``` + 通过 Pull Request 的方式将修改好的 DEFAULTS 文件提交到 upstream-repos 仓库，就可以使用 compass-ci 测试你的项目了。 + + 详细配置方式请参考 https://gitee.com/wu_fengguang/upstream-repos/blob/master/README.md 。 + + 命令参数意义及作用请参考 https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job 。 -- 2.23.0

1 0