November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] libmicro: add PKGBUILD for libmicro
by Bai Jing 04 Nov '20

04 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- pkg/libmicro/PKGBUILD | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 pkg/libmicro/PKGBUILD diff --git a/pkg/libmicro/PKGBUILD b/pkg/libmicro/PKGBUILD new file mode 100644 index 00000000..e421c402 --- /dev/null +++ b/pkg/libmicro/PKGBUILD @@ -0,0 +1,20 @@ +pkgname=libMicro +pkgver=0.4 +pkgrel=0 +pkgdesc="A portable set of microbenchmarks focused on the syscall and system library interfaces." +arch=('i386' 'x86_64') +url="https://github.com/rzezeski/libMicro" +license=('CDDL') +source=("https://codeload.github.com/redhat-performance/libMicro/zip/0.4.0-rh") +md5sums=('SKIP') + + +build() { + cd "$srcdir/$pkgname-$pkgver.$pkgrel-rh" + make +} + +package() { + mkdir -p "$pkgdir/lkp/benchmarks/libMicro" + cp -r "$srcdir/$pkgname-$pkgver.$pkgrel-rh/"* "$pkgdir/lkp/benchmarks/libMicro" +} -- 2.23.0

2 2

[PATCH compass-ci] container/sub-fluentd: fix heartbeat error
by Wu Zhende 04 Nov '20

04 Nov '20

[Error] fluent.warn: Unknown heartbeat response received from 172.17.0.1:24224. It may service out Can't use udp to send heartbeat package. Change to tcp. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/sub-fluentd/docker-fluentd.conf | 1 - 1 file changed, 1 deletion(-) diff --git a/container/sub-fluentd/docker-fluentd.conf b/container/sub-fluentd/docker-fluentd.conf index c674fb1..09fa461 100644 --- a/container/sub-fluentd/docker-fluentd.conf +++ b/container/sub-fluentd/docker-fluentd.conf @@ -56,7 +56,6 @@ @type forward flush_interval 0 send_timeout 60 - heartbeat_type udp heartbeat_interval 1 recover_wait 10 hard_timeout 60 -- 2.23.0

3 2

[PATCH v5 lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 04 Nov '20

04 Nov '20

[why] the path of *.cgz file generated by build-pkg script cannot be obtained during test. So the path of cgz file need to be changed. [how] below is example for openeuler: latest.cgz -> f71820d6055373c9b9381615e28a28f307bb2df4.cgz f71820d6055373c9b9381615e28a28f307bb2df4.cgz v1.8.1.cgz -> 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz v1.5.1.cgz -> df0584e46db9e7b412626deab10b923504745b2c.cgz df0584e46db9e7b412626deab10b923504745b2c.cgz Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..7c494a15 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,10 +2,12 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url # - upstream_commit +# - upstream_tag ## makepkg is a script that automates the building of packages; it can download and validate source files, ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, @@ -21,15 +23,19 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount $LKP_SERVER:$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } @@ -75,12 +81,22 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 +} + +create_softlink() +{ + [ -n "$upstream_tag" ] || return 0 + [ -e "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" ] && return 0 + + ln -sf "$(basename $(realpath ${cgz_name}))" "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" || return 0 + echo "create soft link: $PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz -> $(basename $(realpath ${cgz_name}))" } check_vars mount_dest request_pkg build_source_pkg +create_softlink -- 2.23.0

4 5

[PATCH v3 compass-ci] sparrow/3-code: add ENV LOGGING_ES_HOST and LOGGING_ES_PORT
by Liu Yinsi 04 Nov '20

04 Nov '20

need LOGGING_ES_HOST and LOGGING_ES_PORT to start logging-es service. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 64f34f8..f4e4fa9 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -13,6 +13,8 @@ server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${MASTER_FLUENTD_PORT:=24224} : ${DOCKER_REGISTRY_HOST:=$server_ip} : ${DOCKER_REGISTRY_PORT:=5001} +: ${LOGGING_ES_HOST:=$server_ip} +: ${LOGGING_ES_PORT:=9202} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF @@ -28,9 +30,11 @@ MASTER_FLUENTD_HOST: $MASTER_FLUENTD_HOST MASTER_FLUENTD_PORT: $MASTER_FLUENTD_PORT DOCKER_REGISTRY_HOST: $DOCKER_REGISTRY_HOST DOCKER_REGISTRY_PORT: $DOCKER_REGISTRY_PORT +LOGGING_ES_HOST: $LOGGING_ES_HOST +LOGGING_ES_PORT: $LOGGING_ES_PORT EOF -cat > /etc/profile.d/compass.sh <<'EOF' +cat > /etc/profile.d/compass.sh <<EOF export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci export REPO_SRC=/c/upstream-repos -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/3-code: add ENV LOGGING_ES_HOST and LOGGING_ES_PORT
by Liu Yinsi 04 Nov '20

04 Nov '20

need LOGGING_ES_HOST and LOGGING_ES_PORT to start logging-es service. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 64f34f8..dc329fe 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -13,6 +13,8 @@ server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${MASTER_FLUENTD_PORT:=24224} : ${DOCKER_REGISTRY_HOST:=$server_ip} : ${DOCKER_REGISTRY_PORT:=5001} +: ${LOGGING_ES_HOST:=$server_ip} +: ${LOGGING_ES_PORT:=9202} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF @@ -28,6 +30,8 @@ MASTER_FLUENTD_HOST: $MASTER_FLUENTD_HOST MASTER_FLUENTD_PORT: $MASTER_FLUENTD_PORT DOCKER_REGISTRY_HOST: $DOCKER_REGISTRY_HOST DOCKER_REGISTRY_PORT: $DOCKER_REGISTRY_PORT +LOGGING_ES_HOST: $LOGGING_ES_HOST +LOGGING_ES_PORT: $LOGGING_ES_PORT EOF cat > /etc/profile.d/compass.sh <<'EOF' -- 2.23.0

2 2

[PATCH v3 compass-ci] monitoring/filter.cr: query value support regularity
by Wu Zhende 04 Nov '20

04 Nov '20

[Why] Enhanced monitoring function. Regular expressions can be used in query's value. query can be {"job_id": "z9.*"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/monitoring/filter.cr | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/src/monitoring/filter.cr b/src/monitoring/filter.cr index bc6f901..88702b6 100644 --- a/src/monitoring/filter.cr +++ b/src/monitoring/filter.cr @@ -56,13 +56,20 @@ class Filter def match_query(query : Hash(String, JSON::Any), msg : Hash(String, JSON::Any)) query.each do |key, value| - value = value.as_a - if value.includes?(nil) - return false unless msg.has_key?(key) - else - return false unless value.includes?(msg[key]?) - end + return false unless msg.has_key?(key) + + values = value.as_a + next if values.includes?(nil) || values.includes?(msg[key]?) + + return false unless regular_match(values, msg[key]?.to_s) end return true end + + private def regular_match(rules, string) + rules.each do |rule| + return true if string =~ /#{rule}/ + end + return false + end end -- 2.23.0

1 0

[PATCH lkp-tests] job.rb: fix add second program to pp
by Wei Jihui 04 Nov '20

04 Nov '20

[Why] Now we add new field in job yaml like this: on_fail: sleep: 300 and we want add sleep to pp, but at pp() function, it only traverse job's keys--->on_fail, this can not find sleep. it should find sleep at on_fail's keys. [How] when traverse job to find program, in this case, it is sleep, then find it at on_fail's keys, not job's keys. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 5c2c9517..3fdf8c79 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -576,10 +576,10 @@ class Job @job[comment_to_symbol('auto generated by add_pp()')] = nil @job["pp"] = Hash.new() init_program_options() - for_each_in(@job, @referenced_programs.keys) do |_pk, h, p_n, p_args| - k = hash_key_re_string(@job, p_n, @job['pp'].keys) - if k && !(a)job[k].is_a?(Hash) - @job['pp'][k] = @job[k] + for_each_in(@job, @referenced_programs.keys) do |_pk, h, p_n, _p_args| + k = hash_key_re_string(h, p_n, @job['pp'].keys) + if k && h[k] && !h[k].is_a?(Hash) + @job['pp'][k] = h[k] next end options_hash = @referenced_programs[p_n] -- 2.23.0

1 0

[PATCH lab-z9] hosts: fix taishan200-2280-2s48p-256g--a102 to taishan200-2280-2s48p-512g--a102
by Zhang Yu 04 Nov '20

04 Nov '20

[Why] The memory of taishan200-2280-2s48p-256g--a95 is 512g, so need to fix 256g to 512g. Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- ...200-2280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename hosts/{taishan200-2280-2s48p-256g--a102 => taishan200-2280-2s48p-512g--a102} (100%) diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-512g--a102 similarity index 100% rename from hosts/taishan200-2280-2s48p-256g--a102 rename to hosts/taishan200-2280-2s48p-512g--a102 -- 2.23.0

1 1

[PATCH v4 lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 04 Nov '20

04 Nov '20

[why] the path of *.cgz file generated by build-pkg script cannot be obtained during test. So the path of cgz file need to be changed. [how] below is example for openeuler: latest.cgz -> f71820d6055373c9b9381615e28a28f307bb2df4.cgz f71820d6055373c9b9381615e28a28f307bb2df4.cgz v1.8.1.cgz -> 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz v1.5.1.cgz -> df0584e46db9e7b412626deab10b923504745b2c.cgz df0584e46db9e7b412626deab10b923504745b2c.cgz Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..f9bfa377 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,10 +2,12 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url # - upstream_commit +# - upstream_tag ## makepkg is a script that automates the building of packages; it can download and validate source files, ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, @@ -21,15 +23,19 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount $LKP_SERVER:$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } @@ -75,12 +81,22 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 } +create_softlink() +{ + [ -n "$upstream_tag" ] || return 0 + [ -e "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" ] && return 0 + + ln -sf "$(basename $(realpath ${cgz_name}))" "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" || return 0 + echo "create soft link: $PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz -> $(basename $(realpath ${cgz_name}))" +} + check_vars mount_dest request_pkg build_source_pkg +create_softlink -- 2.23.0

3 4

[PATCH v2 compass-ci] monitoring/filter.cr: query value support regularity
by Wu Zhende 04 Nov '20

04 Nov '20

[Why] Enhanced monitoring function. Regular expressions can be used in query's value. query can be {"job_id": "z9.*"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/monitoring/filter.cr | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/src/monitoring/filter.cr b/src/monitoring/filter.cr index bc6f901..d520e6e 100644 --- a/src/monitoring/filter.cr +++ b/src/monitoring/filter.cr @@ -56,13 +56,21 @@ class Filter def match_query(query : Hash(String, JSON::Any), msg : Hash(String, JSON::Any)) query.each do |key, value| + return false unless msg.has_key?(key) + value = value.as_a - if value.includes?(nil) - return false unless msg.has_key?(key) - else - return false unless value.includes?(msg[key]?) - end + next if value.includes?(nil) + next if value.includes?(msg[key]?) + + return false unless regular_match(value, msg[key]?.to_s) end return true end + + private def regular_match(rules, string) + rules.each do |rule| + return true if string =~ /#{rule}/ + end + return false + end end -- 2.23.0

2 1