- Compass-ci - mailweb.openeuler.org

[PATCH v3 lkp-tests 1/2] tests: replace *.tar.* address with local address
by Liu Shaofei 02 Dec '20

02 Dec '20

When the address of source field is the format of "*.tar.*", we need to replace it with local address. The following example: source=("https://github.com/azukiapp/${pkgname}/archive/v${pkgver}.tar.gz") => source=("http://${LKP_SERVER}:8800/initrd/build-tar/${pkgname}/v${pkgver}.tar.gz") Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/build-pkg b/tests/build-pkg index bdaef035..cde2f0ed 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -98,6 +98,26 @@ replace_source() sed -i "s|^source=[^)]*|$1|g" "$2" } +# replace *.tar.* address from source field in PKGBUILD +replace_tar_source() +{ + echo "source=(" > PKGBUILD.src + for url in ${source[@]} + do + echo "$url" | egrep '.+\.tar' && { + local_url="http://${LKP_SERVER}:8800/initrd/build-tar/${pkgname}/${url##*/}" + + # check whether local server exists *.tar.* file + wget -q --spider "${local_url}" + [ "$?" == 0 ] || continue + + url="\"${local_url}\"" + } + echo "$url" >> PKGBUILD.src + done + echo ")" >> PKGBUILD.src +} + build_source_pkg() { local repo_dir="" @@ -116,6 +136,8 @@ build_source_pkg() [ -n "$PKGBUILD_TAG" ] && replace_source "${upstream_source}" "PKGBUILD-$PKGBUILD_TAG" } + replace_tar_source + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 -- 2.23.0

2 2

[PATCH compass-ci] params_group.rb: delete $user if tbox_group has
by Li Yuanchao 02 Dec '20

02 Dec '20

Some tbox_groups have $user in es. Such as vm-2p8g--$user or vm-2p8g.$user. But their config is the same, they should be compared together. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/params_group.rb | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/params_group.rb b/lib/params_group.rb index cde40af..6d5c47b 100644 --- a/lib/params_group.rb +++ b/lib/params_group.rb @@ -67,10 +67,17 @@ def filter_groups(groups) end end +def get_tbox_group(tbox_group) + tbox_group = tbox_group.split('--')[0] if tbox_group.include?('--') + tbox_group = tbox_group.split('.')[0] if tbox_group.include?('.') + return tbox_group +end + def get_all_params(job) all_params = {} job.each_key do |param| all_params[param] = job[param] if COMMON_PARAMS.include?(param) + all_params[param] = get_tbox_group(job[param]) if param == 'tbox_group' next unless param == 'pp' pp_params = get_pp_params(job[param]) -- 2.23.0

3 4

[PATCH compass-ci] sched: no need to check kernel config in initialized? function
by Cao Xueliang 02 Dec '20

02 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 7758e78..2a811cb 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -146,8 +146,8 @@ class Job set_result_root() set_result_service() set_os_mount() - set_kernel() set_depends_initrd() + set_kernel() set_initrds_uri() set_lkp_server() set_sshr_info() @@ -345,7 +345,11 @@ class Job "SCHED_HOST", "SCHED_PORT"] - initialized_keys -= ["my_uuid"] + initialized_keys -= ["my_uuid", + "kernel_version", + "kernel_uri", + "kernel_params", + "linux_vmlinuz_path"] initialized_keys.each do |key| return false unless @hash.has_key?(key) -- 2.23.0

1 0

[PATCH compass-ci] auto_submit.yaml: change git_repo
by Li Yuanchao 02 Dec '20

02 Dec '20

To keep style the same Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index f741b79..e0f52ba 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -1,6 +1,6 @@ -AvxToNeon/AvxToNeon: +a/AvxToNeon/AvxToNeon: - testbox=vm-2p8g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 api-avx2neon.yaml -compass-ci/compass-ci: +c/compass-ci/compass-ci: - testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=vm-2p8g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=vm-2p8g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- 2.23.0

1 0

[PATCH compass-ci] result-webdav/start: add /etc/localtime
by Cui Yili 02 Dec '20

02 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/result-webdav/start b/container/result-webdav/start index 01141c3..d7b1097 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -13,6 +13,7 @@ cmd=( --restart=always -p 3080:3080 -v $PWD/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf + -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -v /srv/initrd/:/srv/initrd/ result-webdav -- 2.23.0

1 0

[PATCH compass-ci] answerback-email: add execute tool limit
by Luan Shengde 02 Dec '20

02 Dec '20

limit the scope of executing answerback-email to 'z9' [why] when call es_client to store the account data, the data will be stored in the es installed at the localhost. It will cause validation to fail when submit a job if the data is not at the server 'z9'. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 33 +++++++++++++++++--- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 825d815..77a2c1f 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -187,11 +187,14 @@ def apply_account(my_info, conf_info) end def check_my_email(my_info) + return true if my_info['my_email'] + message = "No email address specified\n" message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' + puts message - raise message if my_info['my_email'].nil? + return false end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) @@ -219,13 +222,34 @@ def build_my_info_from_account_info(my_info, account_info, conf_info) my_info['my_login_name'] = account_info['my_login_name'] unless conf_info['is_update_account'] end +def check_server + return true if ENV['HOSTNAME'] == 'crystal.ci' + + message = 'please run the tool on z9 server' + + puts message + return false +end + +def check_my_name_exist(my_info) + return true if my_info['my_name'] + + message = 'No my_name found, please use -n to add one' + puts message + + return false +end + def send_account(my_info, conf_info, email_info, my_info_es, stdin_info) - check_my_email(my_info) + puts 'check_erver' + return unless check_server + return unless check_my_email(my_info) + my_info['my_uuid'] = %x(uuidgen).chomp unless conf_info['is_update_account'] build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) - message = 'No my_name found, please use -n to add one' - raise message if my_info['my_name'].nil? + puts 'check_my_name' + return unless check_my_name_exist(my_info) account_info = apply_account(my_info, conf_info) build_my_info_from_account_info(my_info, account_info, conf_info) @@ -241,6 +265,7 @@ def send_mail(my_info, account_info, conf_info) else build_message(my_info['my_email'], account_info) end + %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") end -- 2.23.0

1 1

[PATCH v4 compass-ci] container/result-webdav: adjust the value of second_dir
by Cui Yili 02 Dec '20

02 Dec '20

[why] 1. when the uri starts with "//", etc., the previous regular expression did not get the second level directory correctly. 2. when the uploaded data package does not meet the format of "xxxx_yyyymmdd.cgz", uploading is also allowed, but no symbolic link. [how] 1. modify the variable second_dir regular expression. 2. make conditional judgments on link_name. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/nginx.conf | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/container/result-webdav/nginx.conf b/container/result-webdav/nginx.conf index c880522..98382bd 100644 --- a/container/result-webdav/nginx.conf +++ b/container/result-webdav/nginx.conf @@ -54,17 +54,21 @@ http { path = string.match(uri,"%g*/") upload_file = string.match(uri, ".+/([^/]*%.%w+)$") - second_dir = string.match(uri, "/.+/(.*)/.+") + second_dir = string.match(uri, "/*%w+/(.-)/.+") - if (second_dir == "pkg") + if (second_dir == "pkg" or second_dir == "build-pkg") then - link_name = "latest.cgz" + link_name = "latest" else - link_name = string.match(upload_file, "(.*)_%d+%.cgz")..".cgz" + link_name = string.match(upload_file, "(.*)_%d+%.cgz") + end + + if (link_name ~= '') + then + link_name = link_name..".cgz" + root_patch = "/srv/"..path + io.popen("cd "..root_patch.."&& ln -sf "..upload_file.." "..link_name) end - - root_patch = "/srv/"..path - io.popen("cd "..root_patch.."&& ln -sf "..upload_file.." "..link_name) } } -- 2.23.0

1 0

[PATCH v4 compass-ci] scheduler: fix the paths to find cluster config file
by Ren Wen 02 Dec '20

02 Dec '20

Find cluster config file at *lab-z9/cluster/* first, *lkp-tests/cluster/* otherwise. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/constants.cr | 3 +++ src/scheduler/submit_job.cr | 14 +++++++++----- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index 418e7ff..b2ea831 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -21,6 +21,9 @@ INITRD_HTTP_PORT = (ENV.has_key?("INITRD_HTTP_PORT") ? ENV["INITRD_HTTP_PORT"] : OS_HTTP_HOST = (ENV.has_key?("OS_HTTP_HOST") ? ENV["OS_HTTP_HOST"] : "172.17.0.1") OS_HTTP_PORT = (ENV.has_key?("OS_HTTP_PORT") ? ENV["OS_HTTP_PORT"] : 8000).to_i32 +CCI_REPOS = (ENV.has_key?("CCI_REPOS") ? ENV["CCI_REPOS"] : "/c") +LAB_REPO = "lab-z9" + SRV_OS = "/srv/os" SRV_INITRD = "/srv/initrd" diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 74ae4b2..a9eb3a7 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -15,7 +15,7 @@ class Sched else cluster_config = get_cluster_config(cluster_file, job.lkp_initrd_user, - job.os_arch) + job.os_arch).not_nil! return submit_cluster_job(job, cluster_config) end rescue ex @@ -147,12 +147,16 @@ class Sched @es.set_job_content(job) end - # get cluster config using own lkp_src cluster file, - # a hash type will be returned def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) - cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)) + + cluster_file_paths = [ + Path.new(CCI_REPOS, LAB_REPO, "cluster", cluster_file), + Path.new(lkp_src, "cluster", cluster_file) + ] + cluster_file_paths.each do |f| + return YAML.parse(File.read(f)) if File.file?(f) + end end def get_commit_date(job) -- 2.23.0

1 0

[PATCH lkp-tests] scheduler: fix: $server not $direct_server_ips
by Zhang Yu 02 Dec '20

02 Dec '20

[why] Now the server ip is defined by scheduler is $server not $direct_server_ips [how] Change the $direct_server_ips to $server Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/sysbench-mysql | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/sysbench-mysql b/tests/sysbench-mysql index 106ee0b9..1d424aaa 100755 --- a/tests/sysbench-mysql +++ b/tests/sysbench-mysql @@ -19,7 +19,7 @@ # db_driver : "${mysql_user:=root}" -: "${mysql_host:=$direct_server_ips}" +: "${mysql_host:=$server}" : "${mysql_port:=$mysql_port}" : "${mysql_db:=sysbench_1}" : "${mysql_password:=$mysql_password}" -- 2.23.0

1 0

[PATCH lab-z9] hosts: add taishan200-2280-2s48p-256g--a12 host-info
by Xiao Shenwei 02 Dec '20

02 Dec '20

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- hosts/taishan200-2280-2s48p-256g--a12 | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 hosts/taishan200-2280-2s48p-256g--a12 diff --git a/hosts/taishan200-2280-2s48p-256g--a12 b/hosts/taishan200-2280-2s48p-256g--a12 new file mode 100644 index 0000000..ce014c1 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a12 @@ -0,0 +1,22 @@ +arch: aarch64 +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-35000c500c276c347 + - /dev/disk/by-id/scsi-35000c500c276eb87 + - /dev/disk/by-id/scsi-35000c500c276b6ff +rootfs_partition: + - /dev/disk/by-id/scsi-35000c500c276f47b +mac_addr: + - 44:67:47:97:22:37 + - 44:67:47:97:22:38 + - 44:67:47:97:22:39 + - 44:67:47:97:22:3a + - 44:67:47:97:22:3b + - 44:67:47:97:22:3c + - 44:67:47:97:22:3d + - 44:67:47:97:22:3e +model_name: Kunpeng-920 +ipmi_ip: 9.3.5.2 -- 2.23.0

1 0