From: Wu Fengguang wfg@mail.ustc.edu.cn
Signed-off-by: Wu Fengguang wfg@mail.ustc.edu.cn --- doc/manual/apply-for-a-test-machine.en.md | 490 ++++++------- doc/manual/compare-test-results.en.md | 96 +-- doc/manual/submit-command-description.md | 416 +++++------ ...latform-to-test-open-source-projects.en.md | 142 ++-- doc/manual/view-task-results.en.md | 46 +- ...00\346\272\220\351\241\271\347\233\256.md" | 136 ++-- doc/tutorial.md | 674 +++++++++--------- 7 files changed, 1000 insertions(+), 1000 deletions(-)
diff --git a/doc/manual/apply-for-a-test-machine.en.md b/doc/manual/apply-for-a-test-machine.en.md index 33ef47f..14dd064 100644 --- a/doc/manual/apply-for-a-test-machine.en.md +++ b/doc/manual/apply-for-a-test-machine.en.md @@ -1,245 +1,245 @@ -# Prerequisites - -Ensure that you have performed the following operations according to the [apply-for-an-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-accou...): - -- Send an email to apply for an account. -- Receive an email from compass-ci@qq.com. -- Configure the local environment. - -# Applying for a Test Machine (VM) - -1. Generate a local RSA private-public key pair. - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. Select the YAML file as required. - - A **lkp-tests** folder is stored in each user directory `/home/${USER}`. - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. Submit the YAML file and connect to the test machine (VM). - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170593 - query=>{"job_id":["z9.170593"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170593","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593"} - {"job_id": "z9.170593", "result_root": "/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593", "job_state": "set result root"} - {"job_id": "z9.170593", "job_state": "boot"} - {"job_id": "z9.170593", "job_state": "download"} - {"time":"2020-11-26 14:45:06","mac":"0a-1f-0d-3c-91-5c","ip":"172.18.156.13","job_id":"z9.170593","state":"running","testbox":"vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - {"job_state":"running","job_id":"z9.170593"} - {"job_id": "z9.170593", "state": "set ssh port", "ssh_port": "51840", "tbox_name": "vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:51840' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Thu Nov 26 06:44:18 CST 2020 - - System load: 0.83 - Processes: 107 - Memory used: 6.1% - Swap used: 0.0% - Usage On: 89% - IP address: 172.18.156.13 - Users online: 1 - - - - root@vm-2p8g ~# - ``` - - For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. - -4. Return the test machine (VM) after use. - - ```shell - root@vm-2p8g ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` - -# Applying for a Test Machine (Physical Machine) - -1. Generate a local RSA private-public key pair. - - ```shell - hi684@account-vm ~% ssh-keygen -t rsa - Generating public/private rsa key pair. - Enter file in which to save the key (/home/hi684/.ssh/id_rsa): - Created directory '/home/hi684/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/hi684/.ssh/id_rsa. - Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm - The key's randomart image is: - +---[RSA 2048]----+ - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxx| - +----[SHA256]-----+ - hi684@account-vm ~% ls -hla .ssh - total 16K - drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . - drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. - -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa - -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub - ``` - -2. Select the YAML file as required. - - A **lkp-tests** folder is stored in each user directory `/home/${USER}`. - - ```shell - hi684@account-vm ~% cd lkp-tests/jobs - hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* - -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml - -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml - -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml - ``` - -3. Submit the YAML file and connect to the test machine (physical machine). - - ```shell - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=taishan200-2280-2s64p-256g borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.170594 - query=>{"job_id":["z9.170594"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.170594","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594"} - {"job_id": "z9.170594", "result_root": "/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594", "job_state": "set result root"} - {"job_id": "z9.170594", "job_state": "boot"} - {"job_id": "z9.170594", "job_state": "download"} - {"time":"2020-11-26 14:51:56","mac":"84-46-fe-26-d3-47","ip":"172.168.178.48","job_id":"z9.170594","state":"running","testbox":"taishan200-2280-2s64p-256g--a5"} - {"job_state":"running","job_id":"z9.170594"} - {"job_id": "z9.170594", "state": "set ssh port", "ssh_port": "50420", "tbox_name": "taishan200-2280-2s64p-256g--a5"} - Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:50420' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Thu Nov 26 14:51:59 CST 2020 - - System load: 1.31 - Processes: 1020 - Memory used: 5.1% - Swap used: 0.0% - Usage On: 3% - IP address: 172.168.178.48 - Users online: 1 - - - - root@taishan200-2280-2s64p-256g--a5 ~# - ``` - - For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. - -4. Return the test machine (physical machine) after use. - - ```shell - root@taishan200-2280-2s64p-256g--a5 ~# reboot - Connection to 172.168.131.2 closed by remote host. - Connection to 172.168.131.2 closed. - hi684@account-vm ~/lkp-tests/jobs% - ``` - -# FAQ - -* How Do I Change the Duration of Keeping the Test Machine when Applying for It? - - ```shell - hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml - suite: borrow - testcase: borrow - - pub_key: <%= - begin - File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp - rescue - nil - end - %> - sshd: - # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. - hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml - # After changing the value, submit the request again. - hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml - ``` - -* Guide to the **submit** Command - - Reference: [submit Command Description.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit%C3%A...) - -* What Are the testbox Options? - - For details about the testbox options, visit https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts. - - >  **Note** - > - > VM testbox: vm-xxx - > - > PM testbox: taishan200-2280-xxx - - >  **Notice** - > -> - If the testbox of a physical machine ends with `--axx`, a physical machine is specified. If a task is already in the task queue of the physical machine, the borrow task you submitted will not be processed until the previous task in the queue is completed. - > - If the testbox of a physical machine does not end with `-axx`, no physical machine is specified. In this case, the borrow task you submitted will be immediately allocated to idle physical machines in the cluster for execution. - -* How Do I Borrow the Specified Operating System? - - For details about the supported `os`, `os_arch`, and `os_version`, see [os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/os-os_verison-...). \ No newline at end of file +# Prerequisites + +Ensure that you have performed the following operations according to the [apply-for-an-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-accou...): + +- Send an email to apply for an account. +- Receive an email from compass-ci@qq.com. +- Configure the local environment. + +# Applying for a Test Machine (VM) + +1. Generate a local RSA private-public key pair. + + ```shell + hi684@account-vm ~% ssh-keygen -t rsa + Generating public/private rsa key pair. + Enter file in which to save the key (/home/hi684/.ssh/id_rsa): + Created directory '/home/hi684/.ssh'. + Enter passphrase (empty for no passphrase): + Enter same passphrase again: + Your identification has been saved in /home/hi684/.ssh/id_rsa. + Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. + The key fingerprint is: + SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm + The key's randomart image is: + +---[RSA 2048]----+ + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + +----[SHA256]-----+ + hi684@account-vm ~% ls -hla .ssh + total 16K + drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . + drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. + -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa + -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub + ``` + +2. Select the YAML file as required. + + A **lkp-tests** folder is stored in each user directory `/home/${USER}`. + + ```shell + hi684@account-vm ~% cd lkp-tests/jobs + hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* + -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml + -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml + -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml + ``` + +3. Submit the YAML file and connect to the test machine (VM). + + ```shell + hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml + submit borrow-1h.yaml, got job_id=z9.170593 + query=>{"job_id":["z9.170593"]} + connect to ws://172.168.131.2:11310/filter + {"job_id":"z9.170593","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593"} + {"job_id": "z9.170593", "result_root": "/srv/result/borrow/2020-11-26/vm-2p8g/openeuler-20.03-aarch64/3600/z9.170593", "job_state": "set result root"} + {"job_id": "z9.170593", "job_state": "boot"} + {"job_id": "z9.170593", "job_state": "download"} + {"time":"2020-11-26 14:45:06","mac":"0a-1f-0d-3c-91-5c","ip":"172.18.156.13","job_id":"z9.170593","state":"running","testbox":"vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} + {"job_state":"running","job_id":"z9.170593"} + {"job_id": "z9.170593", "state": "set ssh port", "ssh_port": "51840", "tbox_name": "vm-2p8g.taishan200-2280-2s64p-256g--a38-12"} + Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts + Warning: Permanently added '[172.168.131.2]:51840' (ECDSA) to the list of known hosts. + Last login: Wed Sep 23 11:10:58 2020 + + + Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 + + System information as of time: Thu Nov 26 06:44:18 CST 2020 + + System load: 0.83 + Processes: 107 + Memory used: 6.1% + Swap used: 0.0% + Usage On: 89% + IP address: 172.18.156.13 + Users online: 1 + + + + root@vm-2p8g ~# + ``` + + For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. + +4. Return the test machine (VM) after use. + + ```shell + root@vm-2p8g ~# reboot + Connection to 172.168.131.2 closed by remote host. + Connection to 172.168.131.2 closed. + hi684@account-vm ~/lkp-tests/jobs% + ``` + +# Applying for a Test Machine (Physical Machine) + +1. Generate a local RSA private-public key pair. + + ```shell + hi684@account-vm ~% ssh-keygen -t rsa + Generating public/private rsa key pair. + Enter file in which to save the key (/home/hi684/.ssh/id_rsa): + Created directory '/home/hi684/.ssh'. + Enter passphrase (empty for no passphrase): + Enter same passphrase again: + Your identification has been saved in /home/hi684/.ssh/id_rsa. + Your public key has been saved in /home/hi684/.ssh/id_rsa.pub. + The key fingerprint is: + SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx hi684@account-vm + The key's randomart image is: + +---[RSA 2048]----+ + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxx| + +----[SHA256]-----+ + hi684@account-vm ~% ls -hla .ssh + total 16K + drwx------. 2 hi684 hi684 4.0K Nov 26 16:37 . + drwx------. 7 hi684 hi684 4.0K Nov 26 16:37 .. + -rw-------. 1 hi684 hi684 1.8K Nov 26 16:37 id_rsa + -rw-r--r--. 1 hi684 hi684 398 Nov 26 16:37 id_rsa.pub + ``` + +2. Select the YAML file as required. + + A **lkp-tests** folder is stored in each user directory `/home/${USER}`. + + ```shell + hi684@account-vm ~% cd lkp-tests/jobs + hi684@account-vm ~/lkp-tests/jobs% ls -hl borrow-* + -rw-r--r--. 1 root root 53 Nov 2 14:54 borrow-10d.yaml + -rw-r--r--. 1 root root 64 Nov 2 14:54 borrow-1d.yaml + -rw-r--r--. 1 root root 235 Nov 19 15:27 borrow-1h.yaml + ``` + +3. Submit the YAML file and connect to the test machine (physical machine). + + ```shell + hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=taishan200-2280-2s64p-256g borrow-1h.yaml + submit borrow-1h.yaml, got job_id=z9.170594 + query=>{"job_id":["z9.170594"]} + connect to ws://172.168.131.2:11310/filter + {"job_id":"z9.170594","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594"} + {"job_id": "z9.170594", "result_root": "/srv/result/borrow/2020-11-26/taishan200-2280-2s64p-256g/openeuler-20.03-aarch64/3600/z9.170594", "job_state": "set result root"} + {"job_id": "z9.170594", "job_state": "boot"} + {"job_id": "z9.170594", "job_state": "download"} + {"time":"2020-11-26 14:51:56","mac":"84-46-fe-26-d3-47","ip":"172.168.178.48","job_id":"z9.170594","state":"running","testbox":"taishan200-2280-2s64p-256g--a5"} + {"job_state":"running","job_id":"z9.170594"} + {"job_id": "z9.170594", "state": "set ssh port", "ssh_port": "50420", "tbox_name": "taishan200-2280-2s64p-256g--a5"} + Host 172.168.131.2 not found in /home/hi684/.ssh/known_hosts + Warning: Permanently added '[172.168.131.2]:50420' (ECDSA) to the list of known hosts. + Last login: Wed Sep 23 11:10:58 2020 + + + Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 + + System information as of time: Thu Nov 26 14:51:59 CST 2020 + + System load: 1.31 + Processes: 1020 + Memory used: 5.1% + Swap used: 0.0% + Usage On: 3% + IP address: 172.168.178.48 + Users online: 1 + + + + root@taishan200-2280-2s64p-256g--a5 ~# + ``` + + For more information about how to use the **submit** command, testbox options, and how to borrow the specified operating system, see the FAQ at the end of this document. + +4. Return the test machine (physical machine) after use. + + ```shell + root@taishan200-2280-2s64p-256g--a5 ~# reboot + Connection to 172.168.131.2 closed by remote host. + Connection to 172.168.131.2 closed. + hi684@account-vm ~/lkp-tests/jobs% + ``` + +# FAQ + +* How Do I Change the Duration of Keeping the Test Machine when Applying for It? + + ```shell + hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml + suite: borrow + testcase: borrow + + pub_key: <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> + sshd: + # sleep at the bottom + sleep: 1h + hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml + sleep: 1h + # Use the VIM editor to change the value of the sleep field. + hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml + # After changing the value, submit the request again. + hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml + ``` + +* Guide to the **submit** Command + + Reference: [submit Command Description.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit%C3%A...) + +* What Are the testbox Options? + + For details about the testbox options, visit https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts. + + >  **Note** + > + > VM testbox: vm-xxx + > + > PM testbox: taishan200-2280-xxx + + >  **Notice** + > +> - If the testbox of a physical machine ends with `--axx`, a physical machine is specified. If a task is already in the task queue of the physical machine, the borrow task you submitted will not be processed until the previous task in the queue is completed. + > - If the testbox of a physical machine does not end with `-axx`, no physical machine is specified. In this case, the borrow task you submitted will be immediately allocated to idle physical machines in the cluster for execution. + +* How Do I Borrow the Specified Operating System? + + For details about the supported `os`, `os_arch`, and `os_version`, see [os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/os-os_verison-...). diff --git a/doc/manual/compare-test-results.en.md b/doc/manual/compare-test-results.en.md index 8a7e5f7..5a5dd22 100644 --- a/doc/manual/compare-test-results.en.md +++ b/doc/manual/compare-test-results.en.md @@ -1,48 +1,48 @@ -# How Do I Compare Test Results? -The Compare feature is used to analyze the results of different jobs, show the performance waves and changes under different influencing factors, for users to analyze performance influencing factor. - -## url: https://compass-ci.openeuler.org/compare - -## options directions -### filters: - - suite: iperf, netperf, mysql, ... - - OS: openeuler 20.03, centos 7.6, ... - - os_arch: aarch64, x86 - - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... - we can combine the above options arbitrarily to limit compare scope. - choose at least one option as filter. - -### dimension - Dimension can select: os, os_version, os_arch, suite, tbox_group. - Within filter, we will compare all different job result by dimension - and keep other test conditions are same. - -## example: - filter: suite = iperf - dimension: os_version - - result: - os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same - - - 20.09 20.03 metric - -------------------- ------------------------------ ------------------------------ - fails:runs change fails:runs - | | | - 3:3 -100.0% 0:3 last_state.exit_fail - 3:3 -100.0% 0:3 last_state.is_incomplete_run - 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 - 3:3 -100.0% 0:3 stderr.perf_command_failed - - - - 20.09 20.03 metric - -------------------- ------------------------------ ------------------------------ - %stddev change %stddev - \ | \ - 4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps - 4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps - 94.82 -44.0% 53.10 boot-time.boot - 123.11 -58.4% 51.19 boot-time.idle - 0.00 0 4.87 boot-time.kernel_boot - 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 \ No newline at end of file +# How Do I Compare Test Results? +The Compare feature is used to analyze the results of different jobs, show the performance waves and changes under different influencing factors, for users to analyze performance influencing factor. + +## url: https://compass-ci.openeuler.org/compare + +## options directions +### filters: + - suite: iperf, netperf, mysql, ... + - OS: openeuler 20.03, centos 7.6, ... + - os_arch: aarch64, x86 + - tbox_group: vm-2p8g, taishan200-2880-2s48p-256g, ... + we can combine the above options arbitrarily to limit compare scope. + choose at least one option as filter. + +### dimension + Dimension can select: os, os_version, os_arch, suite, tbox_group. + Within filter, we will compare all different job result by dimension + and keep other test conditions are same. + +## example: + filter: suite = iperf + dimension: os_version + + result: + os=openeuler/os_arch=aarch64/pp.iperf.protocol=tcp/pp.iperf.runtime=20/tbox_group=vm-2p8g # other test conditions keep same + + + 20.09 20.03 metric + -------------------- ------------------------------ ------------------------------ + fails:runs change fails:runs + | | | + 3:3 -100.0% 0:3 last_state.exit_fail + 3:3 -100.0% 0:3 last_state.is_incomplete_run + 3:3 -100.0% 0:3 last_state.test.iperf.exit_code.127 + 3:3 -100.0% 0:3 stderr.perf_command_failed + + + + 20.09 20.03 metric + -------------------- ------------------------------ ------------------------------ + %stddev change %stddev + \ | \ + 4.461021e+10 ± 6% -17.4% 3.686392e+10 ± 12% iperf.tcp.receiver.bps + 4.461112e+10 ± 6% -17.4% 3.686935e+10 ± 12% iperf.tcp.sender.bps + 94.82 -44.0% 53.10 boot-time.boot + 123.11 -58.4% 51.19 boot-time.idle + 0.00 0 4.87 boot-time.kernel_boot + 4165.50 ± 12% -99.9% 5.00 interrupts.38:GICv3.36.Level.virtio0 diff --git a/doc/manual/submit-command-description.md b/doc/manual/submit-command-description.md index b0faf99..4ea431c 100644 --- a/doc/manual/submit-command-description.md +++ b/doc/manual/submit-command-description.md @@ -1,208 +1,208 @@ -# submit Command Description - -### Prerequisites - -The Compass-CI client has been installed. For details, see [Installing the Local Compass-CI Client](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E6%9C%AC%E...). - -### Purpose - -You can run the **submit** command to submit a test task. This command provides multiple options to help you submit tasks more flexibly. You can enter the **submit** command in the command line to view the help information and use the command flexibly as required. - -### Basic Usage - -The test task is submitted in a YAML file. You need to prepare the YAML file of the test task. This document uses **iperf.yaml** as an example. Run the following command to submit a test task: - -``` -submit iperf.yaml -``` - -The following message is displayed: - -```shell -hi8109@account-vm ~% submit iperf.yaml -submit iperf.yaml, got job_id=z9.173924 -``` - -The **testbox** field has been added to the **iperf.yaml** file shown in this document. If the YAML file does not contain this field, an error is reported: - -```shell -hi8109@account-vm ~% submit iperf.yaml -submit iperf.yaml failed, got job_id=0, error: Missing required job key: 'testbox' -``` - -You can add the **testbox** field to the YAML file or run the following command: - -``` -submit iperf.yaml testbox=vm-2p8g -``` - -The value of the **testbox** field specifies the required test machine. You can run the `ls` command to view the available test machines in the `lkp-tests/hosts` path, as shown in the following figure: - -```shell -hi8109@account-vm ~/lkp-tests/hosts% ll -total 120K --rw-r--r--. 1 root root 76 2020-11-02 14:54 vm-snb --rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p8g --rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p4g --rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p1g --rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-1p1g --rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p8g --rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p4g --rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p1g --rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-1p1g --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p8g-pxe --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p8g --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p4g-pxe --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p4g --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p1g-pxe --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p1g --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-1p1g-pxe --rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-1p1g --rw-r--r--. 1 root root 14 2020-11-02 14:54 taishan200-2280-2s64p-256g --rw-r--r--. 1 root root 497 2020-11-02 14:54 lkp-skl-d01 --rw-r--r--. 1 root root 574 2020-11-02 14:54 lkp-ivb-d04 --rw-r--r--. 1 root root 606 2020-11-02 14:54 lkp-ivb-d02 --rw-r--r--. 1 root root 384 2020-11-02 14:54 lkp-ivb-d01 --rw-r--r--. 1 root root 388 2020-11-02 14:54 lkp-hsw-d01 --rw-r--r--. 1 root root 385 2020-11-02 14:54 lkp-bdw-de1 --rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-8g --rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-4g --rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-2g --rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-1g --rw-r--r--. 1 root root 13 2020-11-02 14:54 2288hv5-2s64p --rw-r--r--. 1 root root 74 2020-11-02 14:54 vm-snb-i386 -``` - ->  **Note** -> -> Use the equal sign (=) to update the fields in the YAML file. The priority of the ***=*** varies with its position in the command line. -> -> * In the **submit iperf.yaml testbox=vm-2p8g** command, the **=** is defined after the YAML file. As a result, the priority of the **=** is higher than that of the YAML file. **testbox=vm-2p8g** overrides the **testbox** field defined in the YAML file. -> * In the **submit testbox=vm-2p8g iperf.yaml** command, the **=** is defined before the YAML file. As a result, the priority of **=** is lower than that of the YAML file. **testbox=vm-2p8g** does not override the **testbox** field defined in the YAML file. A value is assigned only when the YAML file does not contain the **testbox** field. - -### Advanced Usage - -The following figure shows the options of the **submit** command: - -```shell -hi8109@account-vm ~% submit -Usage: submit [options] job1.yaml job2.yaml ... - submit test jobs to the scheduler - -options: - -s, --set 'KEY: VALUE' add YAML hash to job - -o, --output DIR save job yaml to DIR/ - -a, --auto-define-files auto add define_files - -c, --connect auto connect to the host - -m, --monitor monitor job status: use -m 'KEY: VALUE' to add rule - --my-queue add to my queue -``` - -* **Usage of -s** - - You can use the **-s'KEY:VALUE'** parameter to update the key-value pair to the submitted task. An example is shown in the following figure: - - ``` - submit -s 'testbox: vm-2p8g' iperf.yaml - ``` - - * If the **iperf.yaml** file does not contain **testbox: vm-2p8g**, the field will be added to the submitted task. - * If the **iperf.yaml** file contains the **testbox** field but the value is not **vm-2p8g**, the value of **testbox** in the submitted task will be updated as **vm-2p8g**. - -* **Usage of -o** - - You can run the **-o DIR** command to save the generated YAML file to the specified directory **DIR**. An example is shown in the following figure: - - ``` - submit -o ~/iperf.yaml - ``` - - After the command is executed, the YAML file that has been processed by the **submit** command is generated in the specified directory. - -* **Usage of -a** - - If the **lkp-tests** on the client is changed in the test case, you need to use the **-a** option for adaptation. Synchronize the modification made in the **lkp-tests** on the client to the server, and generate a customized test script on the test machine. An example is shown in the following figure: - - ``` - submit -a iperf.yaml - ``` - -* **Usage of -m** - - You can use the **-m** parameter to enable the task monitoring function and print the status information during the task execution on the console. In this way, you can monitor the execution process of the test task in real time. An example is shown in the following figure: - - ``` - submit -m iperf.yaml - ``` - - The following information is displayed on the console: - - ```shell - hi8109@account-vm ~% submit -m iperf.yaml - submit iperf.yaml, got job_id=z9.173923 - query=>{"job_id":["z9.173923"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.173923","message":"","job_state":"submit","result_root":"/srv/result/iperf/2020-11-30/vm-2p8g/openeuler-20.03-aarch6 - {"job_id": "z9.173923", "result_root": "/srv/result/iperf/2020-11-30/vm-2p8g/openeuler-20.03-aarch64/tcp-30/z9.173923", "job_state - {"job_id": "z9.173923", "job_state": "boot"} - {"job_id": "z9.173923", "job_state": "download"} - {"time":"2020-11-30 20:28:16","mac":"0a-f5-9f-83-62-ea","ip":"172.18.192.21","job_id":"z9.173923","state":"running","testbox":"vm- - {"job_state":"running","job_id":"z9.173923"} - {"job_state":"post_run","job_id":"z9.173923"} - {"start_time":"2020-11-30 12:25:15","end_time":"2020-11-30 12:25:45","loadavg":"1.12 0.38 0.14 1/105 1956","job_id":"z9.173923"} - {"job_state":"finished","job_id":"z9.173923"} - {"job_id": "z9.173923", "job_state": "complete"} - {"time":"2020-11-30 20:28:54","mac":"0a-f5-9f-83-62-ea","ip":"172.18.192.21","job_id":"z9.173923","state":"rebooting","testbox":"v - {"job_id": "z9.173923", "job_state": "extract_finished"} - connection closed: normal - ``` - -* **Usage of -c** - - The **-c** parameter must be used together with the **-m** parameter to implement the automatic login function in the task of applying for a device. - - An example is shown in the following figure: - - ``` - submit -m -c borrow-1h.yaml - ``` - - After submitting a task of applying for a device, you will receive the returned login information, such as `ssh ip -p port`. After adding the **-c** parameter, you can log in to the executor without manually entering the SSH login command. - - The following information is displayed on the console: - - ```shell - hi8109@account-vm ~% submit -m -c borrow-1h.yaml - submit borrow-1h.yaml, got job_id=z9.173925 - query=>{"job_id":["z9.173925"]} - connect to ws://172.168.131.2:11310/filter - {"job_id":"z9.173925","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-30/vm-2p8g/openeuler-20.03-aarch - {"job_id": "z9.173925", "result_root": "/srv/result/borrow/2020-11-30/vm-2p8g/openeuler-20.03-aarch64/3600/z9.173925", "job_state" - {"job_id": "z9.173925", "job_state": "boot"} - {"job_id": "z9.173925", "job_state": "download"} - {"time":"2020-11-30 20:35:04","mac":"0a-24-5d-c8-aa-d0","ip":"172.18.101.4","job_id":"z9.173925","state":"running","testbox":"vm-2 - {"job_state":"running","job_id":"z9.173925"} - {"job_id": "z9.173925", "state": "set ssh port", "ssh_port": "50200", "tbox_name": "vm-2p8g.taishan200-2280-2s48p-256g--a52-7"} - Host 172.168.131.2 not found in /home/hi8109/.ssh/known_hosts - Warning: Permanently added '[172.168.131.2]:50200' (ECDSA) to the list of known hosts. - Last login: Wed Sep 23 11:10:58 2020 - - - Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 - - System information as of time: Mon Nov 30 12:32:04 CST 2020 - - System load: 0.50 - Processes: 105 - Memory used: 6.1% - Swap used: 0.0% - Usage On: 89% - IP address: 172.17.0.1 - Users online: 1 - - - - root@vm-2p8g ~# - ``` - - You log in to the executor successfully. \ No newline at end of file +# submit Command Description + +### Prerequisites + +The Compass-CI client has been installed. For details, see [Installing the Local Compass-CI Client](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E6%9C%AC%E...). + +### Purpose + +You can run the **submit** command to submit a test task. This command provides multiple options to help you submit tasks more flexibly. You can enter the **submit** command in the command line to view the help information and use the command flexibly as required. + +### Basic Usage + +The test task is submitted in a YAML file. You need to prepare the YAML file of the test task. This document uses **iperf.yaml** as an example. Run the following command to submit a test task: + +``` +submit iperf.yaml +``` + +The following message is displayed: + +```shell +hi8109@account-vm ~% submit iperf.yaml +submit iperf.yaml, got job_id=z9.173924 +``` + +The **testbox** field has been added to the **iperf.yaml** file shown in this document. If the YAML file does not contain this field, an error is reported: + +```shell +hi8109@account-vm ~% submit iperf.yaml +submit iperf.yaml failed, got job_id=0, error: Missing required job key: 'testbox' +``` + +You can add the **testbox** field to the YAML file or run the following command: + +``` +submit iperf.yaml testbox=vm-2p8g +``` + +The value of the **testbox** field specifies the required test machine. You can run the `ls` command to view the available test machines in the `lkp-tests/hosts` path, as shown in the following figure: + +```shell +hi8109@account-vm ~/lkp-tests/hosts% ll +total 120K +-rw-r--r--. 1 root root 76 2020-11-02 14:54 vm-snb +-rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p8g +-rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p4g +-rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-2p1g +-rw-r--r--. 1 root root 64 2020-11-02 14:54 vm-pxe-hi1620-1p1g +-rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p8g +-rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p4g +-rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-2p1g +-rw-r--r--. 1 root root 75 2020-11-02 14:54 vm-hi1620-1p1g +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p8g-pxe +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p8g +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p4g-pxe +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p4g +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p1g-pxe +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-2p1g +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-1p1g-pxe +-rw-r--r--. 1 root root 61 2020-11-02 14:54 vm-1p1g +-rw-r--r--. 1 root root 14 2020-11-02 14:54 taishan200-2280-2s64p-256g +-rw-r--r--. 1 root root 497 2020-11-02 14:54 lkp-skl-d01 +-rw-r--r--. 1 root root 574 2020-11-02 14:54 lkp-ivb-d04 +-rw-r--r--. 1 root root 606 2020-11-02 14:54 lkp-ivb-d02 +-rw-r--r--. 1 root root 384 2020-11-02 14:54 lkp-ivb-d01 +-rw-r--r--. 1 root root 388 2020-11-02 14:54 lkp-hsw-d01 +-rw-r--r--. 1 root root 385 2020-11-02 14:54 lkp-bdw-de1 +-rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-8g +-rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-4g +-rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-2g +-rw-r--r--. 1 root root 11 2020-11-02 14:54 dc-1g +-rw-r--r--. 1 root root 13 2020-11-02 14:54 2288hv5-2s64p +-rw-r--r--. 1 root root 74 2020-11-02 14:54 vm-snb-i386 +``` + +>  **Note** +> +> Use the equal sign (=) to update the fields in the YAML file. The priority of the ***=*** varies with its position in the command line. +> +> * In the **submit iperf.yaml testbox=vm-2p8g** command, the **=** is defined after the YAML file. As a result, the priority of the **=** is higher than that of the YAML file. **testbox=vm-2p8g** overrides the **testbox** field defined in the YAML file. +> * In the **submit testbox=vm-2p8g iperf.yaml** command, the **=** is defined before the YAML file. As a result, the priority of **=** is lower than that of the YAML file. **testbox=vm-2p8g** does not override the **testbox** field defined in the YAML file. A value is assigned only when the YAML file does not contain the **testbox** field. + +### Advanced Usage + +The following figure shows the options of the **submit** command: + +```shell +hi8109@account-vm ~% submit +Usage: submit [options] job1.yaml job2.yaml ... + submit test jobs to the scheduler + +options: + -s, --set 'KEY: VALUE' add YAML hash to job + -o, --output DIR save job yaml to DIR/ + -a, --auto-define-files auto add define_files + -c, --connect auto connect to the host + -m, --monitor monitor job status: use -m 'KEY: VALUE' to add rule + --my-queue add to my queue +``` + +* **Usage of -s** + + You can use the **-s'KEY:VALUE'** parameter to update the key-value pair to the submitted task. An example is shown in the following figure: + + ``` + submit -s 'testbox: vm-2p8g' iperf.yaml + ``` + + * If the **iperf.yaml** file does not contain **testbox: vm-2p8g**, the field will be added to the submitted task. + * If the **iperf.yaml** file contains the **testbox** field but the value is not **vm-2p8g**, the value of **testbox** in the submitted task will be updated as **vm-2p8g**. + +* **Usage of -o** + + You can run the **-o DIR** command to save the generated YAML file to the specified directory **DIR**. An example is shown in the following figure: + + ``` + submit -o ~/iperf.yaml + ``` + + After the command is executed, the YAML file that has been processed by the **submit** command is generated in the specified directory. + +* **Usage of -a** + + If the **lkp-tests** on the client is changed in the test case, you need to use the **-a** option for adaptation. Synchronize the modification made in the **lkp-tests** on the client to the server, and generate a customized test script on the test machine. An example is shown in the following figure: + + ``` + submit -a iperf.yaml + ``` + +* **Usage of -m** + + You can use the **-m** parameter to enable the task monitoring function and print the status information during the task execution on the console. In this way, you can monitor the execution process of the test task in real time. An example is shown in the following figure: + + ``` + submit -m iperf.yaml + ``` + + The following information is displayed on the console: + + ```shell + hi8109@account-vm ~% submit -m iperf.yaml + submit iperf.yaml, got job_id=z9.173923 + query=>{"job_id":["z9.173923"]} + connect to ws://172.168.131.2:11310/filter + {"job_id":"z9.173923","message":"","job_state":"submit","result_root":"/srv/result/iperf/2020-11-30/vm-2p8g/openeuler-20.03-aarch6 + {"job_id": "z9.173923", "result_root": "/srv/result/iperf/2020-11-30/vm-2p8g/openeuler-20.03-aarch64/tcp-30/z9.173923", "job_state + {"job_id": "z9.173923", "job_state": "boot"} + {"job_id": "z9.173923", "job_state": "download"} + {"time":"2020-11-30 20:28:16","mac":"0a-f5-9f-83-62-ea","ip":"172.18.192.21","job_id":"z9.173923","state":"running","testbox":"vm- + {"job_state":"running","job_id":"z9.173923"} + {"job_state":"post_run","job_id":"z9.173923"} + {"start_time":"2020-11-30 12:25:15","end_time":"2020-11-30 12:25:45","loadavg":"1.12 0.38 0.14 1/105 1956","job_id":"z9.173923"} + {"job_state":"finished","job_id":"z9.173923"} + {"job_id": "z9.173923", "job_state": "complete"} + {"time":"2020-11-30 20:28:54","mac":"0a-f5-9f-83-62-ea","ip":"172.18.192.21","job_id":"z9.173923","state":"rebooting","testbox":"v + {"job_id": "z9.173923", "job_state": "extract_finished"} + connection closed: normal + ``` + +* **Usage of -c** + + The **-c** parameter must be used together with the **-m** parameter to implement the automatic login function in the task of applying for a device. + + An example is shown in the following figure: + + ``` + submit -m -c borrow-1h.yaml + ``` + + After submitting a task of applying for a device, you will receive the returned login information, such as `ssh ip -p port`. After adding the **-c** parameter, you can log in to the executor without manually entering the SSH login command. + + The following information is displayed on the console: + + ```shell + hi8109@account-vm ~% submit -m -c borrow-1h.yaml + submit borrow-1h.yaml, got job_id=z9.173925 + query=>{"job_id":["z9.173925"]} + connect to ws://172.168.131.2:11310/filter + {"job_id":"z9.173925","message":"","job_state":"submit","result_root":"/srv/result/borrow/2020-11-30/vm-2p8g/openeuler-20.03-aarch + {"job_id": "z9.173925", "result_root": "/srv/result/borrow/2020-11-30/vm-2p8g/openeuler-20.03-aarch64/3600/z9.173925", "job_state" + {"job_id": "z9.173925", "job_state": "boot"} + {"job_id": "z9.173925", "job_state": "download"} + {"time":"2020-11-30 20:35:04","mac":"0a-24-5d-c8-aa-d0","ip":"172.18.101.4","job_id":"z9.173925","state":"running","testbox":"vm-2 + {"job_state":"running","job_id":"z9.173925"} + {"job_id": "z9.173925", "state": "set ssh port", "ssh_port": "50200", "tbox_name": "vm-2p8g.taishan200-2280-2s48p-256g--a52-7"} + Host 172.168.131.2 not found in /home/hi8109/.ssh/known_hosts + Warning: Permanently added '[172.168.131.2]:50200' (ECDSA) to the list of known hosts. + Last login: Wed Sep 23 11:10:58 2020 + + + Welcome to 4.19.90-2003.4.0.0036.oe1.aarch64 + + System information as of time: Mon Nov 30 12:32:04 CST 2020 + + System load: 0.50 + Processes: 105 + Memory used: 6.1% + Swap used: 0.0% + Usage On: 89% + IP address: 172.17.0.1 + Users online: 1 + + + + root@vm-2p8g ~# + ``` + + You log in to the executor successfully. diff --git a/doc/manual/use-the-compass-ci-platform-to-test-open-source-projects.en.md b/doc/manual/use-the-compass-ci-platform-to-test-open-source-projects.en.md index bb0af45..7a38e5e 100644 --- a/doc/manual/use-the-compass-ci-platform-to-test-open-source-projects.en.md +++ b/doc/manual/use-the-compass-ci-platform-to-test-open-source-projects.en.md @@ -1,71 +1,71 @@ -# How DO I Use the Compass-CI Platform to Test Open Source Projects? - -This document describes how to use the Compass-CI platform to test open source projects. - -### Adding the URL of the Repository to Be Tested to the upstream-repos Repository - -Perform the following steps to add the information of the code repository to be tested to the **upstream-repos** repository (https://gitee.com/wu_fengguang/upstream-repos) in YAML format: - -1. Fork the code repository to be tested and clone it to the local host. This document uses the **backlight** repository (https://github.com/baskerville/backlight) as an example. - - - -2. Run the following command to create a file path named with the repository name and its initial letter: - - ``` - mkdir -p b/backlight - ``` - -3. Run the following command to create the **backlight** file in the directory: - - ``` - cd b/backlight - touch backlight - ``` - -4. Run the following command to write the URL of the **backlight** repository to the **backlight** file: - - ``` - vim backlight - ``` - - The format is as follows: - - ``` - --- - url: - - https://github.com/baskerville/backlight - ``` - - >  **Note** - > - > You can refer to the existing file format in the **upstream-repos** repository. Ensure that the formats are consistent. - -5. Run the **Pull Request** command to submit the new **backlight** file to the **backlight** repository. - -### Submitting the Test Task to the Compass-CI Platform - -1. Prepare a test case. - - You can compile and add a test case to the **lkp-tests** repository, or directly use the existing test cases in the **jobs** directory of the **lkp-tests** repository (https://gitee.com/wu_fengguang/lkp-tests). - - * Use the test cases that have been adapted in the repository. - - Use the test cases in the **lkp-tests** repository that meet the requirements. The **iperf.yaml** file is used as an example. The **iperf.yaml** file is a test case that has been adapted. It is stored in the **jobs** directory of the **lkp-tests** repository, and contains some basic test parameters. - - * Compile a test case and add it to the repository. - - For details, see [How To Add Test Cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md). - -2. Configure the **auto_submit.yaml** file and submit the test task. - - You only need to add the following configuration information to the **sbin/auto_submit.yaml** file in the **compass-ci** repository: - - ``` - b/backlight/backlight: - - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml - ``` - - Submit the modified **auto_submit.yaml** file to the **compass-ci** repository using Pull Request. Then you can use the Compass-CI platform to test your project. - - For details about how to set parameters in the **auto_submit.yaml** file, see https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job. \ No newline at end of file +# How DO I Use the Compass-CI Platform to Test Open Source Projects? + +This document describes how to use the Compass-CI platform to test open source projects. + +### Adding the URL of the Repository to Be Tested to the upstream-repos Repository + +Perform the following steps to add the information of the code repository to be tested to the **upstream-repos** repository (https://gitee.com/wu_fengguang/upstream-repos) in YAML format: + +1. Fork the code repository to be tested and clone it to the local host. This document uses the **backlight** repository (https://github.com/baskerville/backlight) as an example. + + + +2. Run the following command to create a file path named with the repository name and its initial letter: + + ``` + mkdir -p b/backlight + ``` + +3. Run the following command to create the **backlight** file in the directory: + + ``` + cd b/backlight + touch backlight + ``` + +4. Run the following command to write the URL of the **backlight** repository to the **backlight** file: + + ``` + vim backlight + ``` + + The format is as follows: + + ``` + --- + url: + - https://github.com/baskerville/backlight + ``` + + >  **Note** + > + > You can refer to the existing file format in the **upstream-repos** repository. Ensure that the formats are consistent. + +5. Run the **Pull Request** command to submit the new **backlight** file to the **backlight** repository. + +### Submitting the Test Task to the Compass-CI Platform + +1. Prepare a test case. + + You can compile and add a test case to the **lkp-tests** repository, or directly use the existing test cases in the **jobs** directory of the **lkp-tests** repository (https://gitee.com/wu_fengguang/lkp-tests). + + * Use the test cases that have been adapted in the repository. + + Use the test cases in the **lkp-tests** repository that meet the requirements. The **iperf.yaml** file is used as an example. The **iperf.yaml** file is a test case that has been adapted. It is stored in the **jobs** directory of the **lkp-tests** repository, and contains some basic test parameters. + + * Compile a test case and add it to the repository. + + For details, see [How To Add Test Cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md). + +2. Configure the **auto_submit.yaml** file and submit the test task. + + You only need to add the following configuration information to the **sbin/auto_submit.yaml** file in the **compass-ci** repository: + + ``` + b/backlight/backlight: + - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml + ``` + + Submit the modified **auto_submit.yaml** file to the **compass-ci** repository using Pull Request. Then you can use the Compass-CI platform to test your project. + + For details about how to set parameters in the **auto_submit.yaml** file, see https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job. diff --git a/doc/manual/view-task-results.en.md b/doc/manual/view-task-results.en.md index bd266df..427f74c 100644 --- a/doc/manual/view-task-results.en.md +++ b/doc/manual/view-task-results.en.md @@ -1,23 +1,23 @@ -# How Do I View Task Results? - -After a test case is executed, you can log in to the https://compass-ci.openeuler.org/jobs to view the execution result. - -You can find the task on the website using the suite defined during task submission or the ID returned after the task is submitted, and click the **job_state** column corresponding to the task to view the task result. - - - -## Result Files - -**job.yaml File** - -Some fields in the **job.yaml** file are submitted by users, and other fields in the file are automatically added by the platform based on the submitted jobs. This file contains all the parameters required for the test task. - -**output File** - -The **output** file records the execution process of a test case. The **check_exit_code** status code is usually displayed at the end of the file. If the status code is not 0, the test case is incorrect. - -**stats.json** - -After a test case is executed, a file with the same name as that of the test case is generated. The file records the test commands and standardized output results. Compass-CI parses these files and generates a file with the file name extension **.json**. - -The **stats.json** file incorporates all JSON files. The key results of all test commands are included in this file for subsequent comparison and analysis. \ No newline at end of file +# How Do I View Task Results? + +After a test case is executed, you can log in to the https://compass-ci.openeuler.org/jobs to view the execution result. + +You can find the task on the website using the suite defined during task submission or the ID returned after the task is submitted, and click the **job_state** column corresponding to the task to view the task result. + + + +## Result Files + +**job.yaml File** + +Some fields in the **job.yaml** file are submitted by users, and other fields in the file are automatically added by the platform based on the submitted jobs. This file contains all the parameters required for the test task. + +**output File** + +The **output** file records the execution process of a test case. The **check_exit_code** status code is usually displayed at the end of the file. If the status code is not 0, the test case is incorrect. + +**stats.json** + +After a test case is executed, a file with the same name as that of the test case is generated. The file records the test commands and standardized output results. Compass-CI parses these files and generates a file with the file name extension **.json**. + +The **stats.json** file incorporates all JSON files. The key results of all test commands are included in this file for subsequent comparison and analysis. diff --git "a/doc/manual/\345\246\202\344\275\225\344\275\277\347\224\250compass-ci\346\265\213\350\257\225\345\274\200\346\272\220\351\241\271\347\233\256.md" "b/doc/manual/\345\246\202\344\275\225\344\275\277\347\224\250compass-ci\346\265\213\350\257\225\345\274\200\346\272\220\351\241\271\347\233\256.md" index e73b9f9..cbbcaef 100644 --- "a/doc/manual/\345\246\202\344\275\225\344\275\277\347\224\250compass-ci\346\265\213\350\257\225\345\274\200\346\272\220\351\241\271\347\233\256.md" +++ "b/doc/manual/\345\246\202\344\275\225\344\275\277\347\224\250compass-ci\346\265\213\350\257\225\345\274\200\346\272\220\351\241\271\347\233\256.md" @@ -1,68 +1,68 @@ -# 使用 compass-ci 平台测试开源项目 - -本文介绍了使用 compass-ci 平台测试开源项目的操作。 - -### 添加待测试仓库 URL 到 upstream-repos 仓库 - -执行以下步骤，将想要测试的代码仓信息以 yaml 文件的方式添加到 upstream-repos 仓库（https://gitee.com/wu_fengguang/upstream-repos ）。 - -1. Fork 要测试的代码仓库并 git clone 到本地，本文以 blacklight 仓库（https://github.com/baskerville/backlight ）为例说明。 - - - -2. 执行以下命令，以首字母和仓库名创建文件路径。 - - ``` - mkdir -p b/backlight - ``` - -3. 执行以下命令，在该目录下新建同名文件 backlight。 - ``` - cd b/backlight - touch backlight - ``` - -4. 执行以下命令，将 backlight 仓库 url 信息写入 backlight 文件。 - - ``` - vim backlight - ``` - 内容格式为 - - ``` - --- - url: - - https://github.com/baskerville/backlight - ``` - - > **注意：** - > - >可参考 upstream-repos 仓库中已有文件格式,请保持格式一致。 - -5. 通过 Pull Request 命令将新增的 backlight 文件提交到 blacklight 仓库。 - - -### 提交测试任务到 compass-ci 平台 - -1. 准备测试用例 - - 测试用例可以自己编写并添加到 lkp-tests 仓库，也可以直接使用 lkp-tests 仓库（https://gitee.com/wu_fengguang/lkp-tests ）的 jobs 目录下已有的测试用例。 - - * 使用仓库中已经适配好的测试用例 - 如果 lkp-tests 仓库中正好有你想要的测试用例，你可以直接使用。以 iperf.yaml 文件为例说明如下： - iperf.yaml 是一个已经适配好的测试用例，它位于 lkp-tests 仓库的 jobs 目录下，其中有一些基本的测试参数。 - - * 编写测试用例并添加到仓库 - - 请参考：[如何添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md ) - -2. 配置 auto_submit.yaml 文件，提交测试任务 - - 你只需要在 compass-ci 仓库下面的 sbin/auto_submit.yaml 文件中添加配置信息，如： - ``` - b/backlight/backlight: - - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml - ``` - 通过 Pull Request 的方式将修改好的 auto_submit.yaml 文件提交到 compass-ci 仓库，就可以使用 compass-ci 测试你的项目了。 - - auto_submit.yaml 文件的参数配置请参考 https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job 。 +# 使用 compass-ci 平台测试开源项目 + +本文介绍了使用 compass-ci 平台测试开源项目的操作。 + +### 添加待测试仓库 URL 到 upstream-repos 仓库 + +执行以下步骤，将想要测试的代码仓信息以 yaml 文件的方式添加到 upstream-repos 仓库（https://gitee.com/wu_fengguang/upstream-repos ）。 + +1. Fork 要测试的代码仓库并 git clone 到本地，本文以 blacklight 仓库（https://github.com/baskerville/backlight ）为例说明。 + + + +2. 执行以下命令，以首字母和仓库名创建文件路径。 + + ``` + mkdir -p b/backlight + ``` + +3. 执行以下命令，在该目录下新建同名文件 backlight。 + ``` + cd b/backlight + touch backlight + ``` + +4. 执行以下命令，将 backlight 仓库 url 信息写入 backlight 文件。 + + ``` + vim backlight + ``` + 内容格式为 + + ``` + --- + url: + - https://github.com/baskerville/backlight + ``` + + > **注意：** + > + >可参考 upstream-repos 仓库中已有文件格式,请保持格式一致。 + +5. 通过 Pull Request 命令将新增的 backlight 文件提交到 blacklight 仓库。 + + +### 提交测试任务到 compass-ci 平台 + +1. 准备测试用例 + + 测试用例可以自己编写并添加到 lkp-tests 仓库，也可以直接使用 lkp-tests 仓库（https://gitee.com/wu_fengguang/lkp-tests ）的 jobs 目录下已有的测试用例。 + + * 使用仓库中已经适配好的测试用例 + 如果 lkp-tests 仓库中正好有你想要的测试用例，你可以直接使用。以 iperf.yaml 文件为例说明如下： + iperf.yaml 是一个已经适配好的测试用例，它位于 lkp-tests 仓库的 jobs 目录下，其中有一些基本的测试参数。 + + * 编写测试用例并添加到仓库 + + 请参考：[如何添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md ) + +2. 配置 auto_submit.yaml 文件，提交测试任务 + + 你只需要在 compass-ci 仓库下面的 sbin/auto_submit.yaml 文件中添加配置信息，如： + ``` + b/backlight/backlight: + - testbox=vm-2p8g os=openEuler os_version=20.03 os_mount=initramfs os_arch=aarch64 iperf.yaml + ``` + 通过 Pull Request 的方式将修改好的 auto_submit.yaml 文件提交到 compass-ci 仓库，就可以使用 compass-ci 测试你的项目了。 + + auto_submit.yaml 文件的参数配置请参考 https://gitee.com/wu_fengguang/compass-ci/tree/master/doc/job 。 diff --git a/doc/tutorial.md b/doc/tutorial.md index 585be6f..e6f8491 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -1,337 +1,337 @@ -# Compass-CI 用户指南 - -## 简介 - - -### 概念 - -Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上游开源软件（来自 Github, Gitee, Gitlab 等托管平台）的测试服务、登录服务、故障辅助定界服务和基于历史数据的分析服务。通过 Compass-CI，社区开发者可以将开源软件快速引入 openEuler 社区，并补充更多的测试用例，共同构建一个开放、完整的开源软件生态系统。 - - -### 功能描述 - -- 测试服务 - - 支持开发者基于本地设备开发，往 github 提交代码，Compass-CI 自动获取代码开展测试，并向开发者反馈测试结果。 - -- 调测环境登录 - - Compass-CI 提供 SSH 登录能力，测试过程中如果测出问题，开发者可根据需要登录环境进行调测 。 - -- 测试结果分析 - - Compass-CI 记录历史测试结果，对外提供 web 及命令行接口，支持开发者针对已有的测试结果进行分析，挖掘影响测试结果的因素。 - -- 辅助定位 - - Compass-CI 在测试过程中可以自动识别错误信息，触发基于 git tree 的测试，找出引入问题模块的变化点。 - -### 应用场景 - -- 聚合开发者测试用例 - - 开发者向代码托管平台提交代码、测试用例、测试工具时，Compass-CI 自动获取提交的代码开展构建测试，同时获取开发者编写到开源软件包的测试用例进行自动化测试，并反馈测试结果。 - -- 登录环境随时调测 - - 测试过程中，发现有 Bug 时，可随时提供调测资源服务，登录到环境进行复现、调试。 - -- 快照数据分析对比 - - 测试过程中，全面监控系统运行信息（CPU/MEM/IO/网络等），对测试过程中的数据做快照归档，提供多次测试之间快照数据分析对比能力，协助开发者对测试结果开展分析，找出影响测试结果的因素。 - -- 辅助定界 - - 测试过程中，发现有 Bug 时，自动触发 Regression 机制，找出首次引入问题 Commit 信息。 - - - -## 使用 Compass-CI -> 当您注册之后，便可以编写 yaml 文件并通过我们的工具上传任务以进行自定义测试，测试功能将尽快上线。 - - -- 注册自己的仓库 - - 如果您想在 `git push` 的时候, 自动触发测试, 那么需要把您的公开 git url 添加到如下仓库 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos)%E3%80%82 - ```bash - git clone https://gitee.com/wu_fengguang/upstream-repos.git - less upstream-repos/README.md - ``` - -### 编写 job yaml 文件 - -#### job yaml 简介 -job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAML_for_ruby.html)%E7%9A%84%E6%A0%BC%E5%BC%8F%E7%BC%96%E5%8... job 文件位于```$LKP_SRC/jobs```路径下。[$LKP_SRC](https://gitee.com/wu_fengguang/lkp-tests) - -#### job yaml 的结构 - -- 标识头部 Header（必选） - - 每一个 job 文件的开始部分，都有一些基础的描述信息，称之为 yaml 的 Header。头部主要有 suite 和 category，suite 是测试套的名称。category 是测试类型，包括 benchmarch（性能测试）/functional（功能测试）/noise-benchmark（不常用）。 - - ```yaml - suite: netperf \测试套名称 - category: benchmark \测试类型：benchmarch|functional|noise-benchmark - ``` - -- 测试脚本和参数（必选） - - - 测试脚本位于```$LKP_SRC/tests```，```$LKP_SRC/daemon```路径下。job yaml 中写入与上述路径中脚本相匹配的文件，将其视为可执行脚本。 - - ``` - $LKP_SRC/setup - $LKP_SRC/monitors - $LKP_SRC/daemon - $LKP_SRC/tests - ``` - - 参数值作为环境变量传递，每个测试脚本将在其文件头的注释中记录可接受的参数。 - - ```yaml - netserver: - netperf: - runtime: 10s - test: - - TCP_STREAM - - UDP_STREAM - send_size: - - 1024 - - 2048 - ``` -- 测试资源相关变量（必选） - - SUT/schduler/部署等资源相关变量。 - ```yaml - testbox: vm-2p8g - os: openeuler - os_version: 20.03 - os_arch: aarch64 - ``` -- 系统设置脚本（可选） - - 位于```$LKP_SRC/setup```路径下。设置脚本将在测试脚本之前运行，主要是用于启停一些依赖性服务或工具，或者配置测试所需的参数等。 - - ```yaml - cgroup2: #$LKP_SRC/setup/cgroup2 executable script - memory.high: 90% - memory.low: 50% - memory.max: max - memory.swap.max: - io.max: - io.weight: - rdma.max: - ``` -- 监控脚本（可选） - - 位于```$LKP_SRC/monitors```路径下。 监视器在运行基准测试时可以捕获性能统计数据，对性能分析和回归根源有价值。 - - ```yaml - proc-vmstats: - nterval: 5 - ``` - -**了解更多 job yaml 相关知识** [Job definition to execution](https://gitee.com/wu_fengguang/lkp-tests/blob/master/jobs/README.md) - - -### 提交测试任务 - -### 查看测试结果 - -### 比较测试结果 - -### 提交 borrow 任务 - -### 提交 bisect 任务 - -## 高级功能 - -### 添加 OS 支持 - -制作一个 initramfs 启动的 cgz 镜像，当系统内核启动时，直接从打包的 cgz 镜像中导出 rootfs，在内存中展开文件系统，利用磁盘的高速缓存机制让系统直接在内存中读写文件，以提升系统的 I/O 性能。 - -#### 操作步骤 - - 1. 获取对应 os 版本的 rootfs（以 openEuler 为例） - - 通过 docker 获取 rootfs - - 1) 下载 openEuler 官方提供的 docker 镜像压缩包 - - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler-... - ``` - 2) 加载 docker 镜像 - ```bash - docker load -i openEuler-docker.aarch64 - ``` - - 3) 启动 openEuler 容器 - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) 拷贝 docker 的 rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - - 通过 qemu.img(qcow2格式)获取 rootfs (openEuler为例) - - - 1) 下载 openEule r官方网站提供的 qcow2 格式镜像 - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/o... - ``` - 2) 使用{compass-ci}/container/qcow2rootfs 制作rootfs - ```bash - cd {compass-ci}/container/qcow2rootfs - ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs - ``` - 2. 定制rootfs - 1. 使用chroot命令切换到 rootfs (此步骤需要 root 权限) - ```bash - chroot openEuler-rootfs - ``` - 2. 根据个人需要安装并配置服务 - - a. 修改 root 密码 - b. 配置 ssh 服务 - c. 检查系统时间 - d. 如果使用 docker 制作 osimage 还需要以下操作： - > 1. 安装所需版本内核 - > 2. 从 centos 官方网站下载内核rpm包 - > 3. 使用 yum 进行安装 - > 4. 删除 docker 环境变量文件 - - 3. 退出 rootfs，并打包 - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` -#### FAQ -1. 日志报错 “Unable to mount root fs on unknown-block” - - 问题现象 - ```bash - [ 0.390437] List of all partitions: - [ 0.390806] No filesystem could mount root, tried: - [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) - ... - [ 0.399404] Memory Limit: none - [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- - ``` - - 解决方法 - - 1）启动内存不足，增加内存可解决。 - 2）内核文件权限不足，给予 644 权限。 - -2. 系统运行缓慢 - - - 问题现象 - - 打包镜像体积过大，会消耗很大内存 - - 解决方法 - - 建议用户根据具体需要对 rootfs 进行裁剪。 - - -### 添加测试用例 - 参考文档详见：[添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) - - -### PKGBUILD 构建 - -1. 概述： - - 使用 PKGBUILD 完成上游项目源代码的编译构建，同一项目可以使用不同 PKGBUILD 完成项目编译构建过程。 - PKGBUILD 相关参考文档：https://wiki.archlinux.org/index.php/PKGBUILD%E3%80%82 - -2. 步骤： - - 1) 注册仓库 - 如果您想在 git push 的时候, 自动触发测试, 那么需要把您的公开 git url 添加到如下仓库 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos)%E3%80%82 - ```bash - git clone https://gitee.com/wu_fengguang/upstream-repos.git - less upstream-repos/README.md - ``` - - 2) 执行构建测试 - 项目代码更新后自动触发构建任务，无需其他操作。 - - 3) 查看结果 - web: https://compass-ci.openeuler.org/jobs%E3%80%82 - - 4) 备注 - 目前提供 archlinux 下已有 PKGBUILD 项目构建测试，暂不支持自主导入 PKGBUILD 文件。 - PKGBUILD 存放路径： /srv/git/archlinux/*/*/PKGBUILD。 - - -### depends - -- 依赖包构建 - 流程: 以下默认使用 debian:sid 版本进行,如果使用其他OS跑测试用例，则需要在 lkp-tests/distro/adaptation/ 下将包名以键值对的形式写入对应 OS 的文件中 - > 例： `echo 'apache2: httpd' >> lkp-tests/distro/adaptation/openeuler` - - 1) 在lkp-tests/distro/depends/下新建一个与测试用例同名的文件 - `vim lkp-tests/distro/depends/test` - 2) 将测试用例所需debian下的包名填入创建的文件中 - `echo "apache2" >> lkp-tests/distro/depends/test` - 3) 生成lkp-aarch64.cgz - `./crystal-ci/container/lkp-initrd/run` - 4) 提交测试用例 - `submit test.yaml` - 5) 查看结果 - https://compass-ci.openeuler.org/jobs - - - -### 本地搭建 compass-ci 服务器节点 - -在 openEuler 系统一键部署 compass-ci 环境，当前已支持 openEuler-aarch64-20.03-LTS 系统环境，以下配置仅供参考。 - -#### 准备工作 -- 硬件 - 服务器类型：ThaiShan200-2280 (建议) - 架构：aarch64 - 内存：>= 32GB - CPU：64 nuclear (建议) - 硬盘：>= 500G (建议划分独立分区) - - >**说明：** - >划分较小独立分区详细操作请参考[添加测试用例](https://gitee.com/wu_fengguang/compass-ci/blob/master/sparrow/1-storage/smal...) - >划分较大独立分区详细操作请参考[添加测试用例](https://gitee.com/wu_fengguang/compass-ci/blob/master/sparrow/1-storage/larg...) - -- 软件 - OS：openEuler-aarch64-20.03 LTS - git：2.23.0版本 (建议) - 预留空间：>= 300G - 网络：可以访问互联网 - - >**说明：** - >openEuler 系统安装详细操作请参考[添加测试用例](https://openeuler.org/zh/docs/20.03_LTS/docs/Installation/%E5%AE%89%E8%A3%85...) - - -#### 操作指导 -1. 登录 openEuler 系统 - -2. 创建工作目录并设置文件权限 - ```bash - mkdir demo && cd demo && umask 002 - ``` -3. 克隆 compass-ci 项目代码到 demo 目录 - ```bash - git clone https://gitee.com/wu_fengguang/compass-ci.git - ``` -4. 执行一键部署脚本 install-tiny - ```bash - cd compass-ci/sparrow && ./install-tiny - ``` - - -## 呼吁合作 - - 增强 git bisect 能力 - - 增强数据分析能力 - - 增强数据结果可视化能力 - -## 下一步计划 - -1. 优化 git bisect 精准率及效率。 -2. 优化数据分析联动能力，让数据分析更加聚焦。 -3. 数据可视化优化，更加友好展示数据比较结果。 -4. 增强登录认证机制，如：GPG 认证。 -5. 优化部署效率。 +# Compass-CI 用户指南 + +## 简介 + + +### 概念 + +Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上游开源软件（来自 Github, Gitee, Gitlab 等托管平台）的测试服务、登录服务、故障辅助定界服务和基于历史数据的分析服务。通过 Compass-CI，社区开发者可以将开源软件快速引入 openEuler 社区，并补充更多的测试用例，共同构建一个开放、完整的开源软件生态系统。 + + +### 功能描述 + +- 测试服务 + + 支持开发者基于本地设备开发，往 github 提交代码，Compass-CI 自动获取代码开展测试，并向开发者反馈测试结果。 + +- 调测环境登录 + + Compass-CI 提供 SSH 登录能力，测试过程中如果测出问题，开发者可根据需要登录环境进行调测 。 + +- 测试结果分析 + + Compass-CI 记录历史测试结果，对外提供 web 及命令行接口，支持开发者针对已有的测试结果进行分析，挖掘影响测试结果的因素。 + +- 辅助定位 + + Compass-CI 在测试过程中可以自动识别错误信息，触发基于 git tree 的测试，找出引入问题模块的变化点。 + +### 应用场景 + +- 聚合开发者测试用例 + + 开发者向代码托管平台提交代码、测试用例、测试工具时，Compass-CI 自动获取提交的代码开展构建测试，同时获取开发者编写到开源软件包的测试用例进行自动化测试，并反馈测试结果。 + +- 登录环境随时调测 + + 测试过程中，发现有 Bug 时，可随时提供调测资源服务，登录到环境进行复现、调试。 + +- 快照数据分析对比 + + 测试过程中，全面监控系统运行信息（CPU/MEM/IO/网络等），对测试过程中的数据做快照归档，提供多次测试之间快照数据分析对比能力，协助开发者对测试结果开展分析，找出影响测试结果的因素。 + +- 辅助定界 + + 测试过程中，发现有 Bug 时，自动触发 Regression 机制，找出首次引入问题 Commit 信息。 + + + +## 使用 Compass-CI +> 当您注册之后，便可以编写 yaml 文件并通过我们的工具上传任务以进行自定义测试，测试功能将尽快上线。 + + +- 注册自己的仓库 + + 如果您想在 `git push` 的时候, 自动触发测试, 那么需要把您的公开 git url 添加到如下仓库 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos)%E3%80%82 + ```bash + git clone https://gitee.com/wu_fengguang/upstream-repos.git + less upstream-repos/README.md + ``` + +### 编写 job yaml 文件 + +#### job yaml 简介 +job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAML_for_ruby.html)%E7%9A%84%E6%A0%BC%E5%BC%8F%E7%BC%96%E5%8... job 文件位于```$LKP_SRC/jobs```路径下。[$LKP_SRC](https://gitee.com/wu_fengguang/lkp-tests) + +#### job yaml 的结构 + +- 标识头部 Header（必选） + + 每一个 job 文件的开始部分，都有一些基础的描述信息，称之为 yaml 的 Header。头部主要有 suite 和 category，suite 是测试套的名称。category 是测试类型，包括 benchmarch（性能测试）/functional（功能测试）/noise-benchmark（不常用）。 + + ```yaml + suite: netperf \测试套名称 + category: benchmark \测试类型：benchmarch|functional|noise-benchmark + ``` + +- 测试脚本和参数（必选） + + - 测试脚本位于```$LKP_SRC/tests```，```$LKP_SRC/daemon```路径下。job yaml 中写入与上述路径中脚本相匹配的文件，将其视为可执行脚本。 + + ``` + $LKP_SRC/setup + $LKP_SRC/monitors + $LKP_SRC/daemon + $LKP_SRC/tests + ``` + - 参数值作为环境变量传递，每个测试脚本将在其文件头的注释中记录可接受的参数。 + + ```yaml + netserver: + netperf: + runtime: 10s + test: + - TCP_STREAM + - UDP_STREAM + send_size: + - 1024 + - 2048 + ``` +- 测试资源相关变量（必选） + + SUT/schduler/部署等资源相关变量。 + ```yaml + testbox: vm-2p8g + os: openeuler + os_version: 20.03 + os_arch: aarch64 + ``` +- 系统设置脚本（可选） + + 位于```$LKP_SRC/setup```路径下。设置脚本将在测试脚本之前运行，主要是用于启停一些依赖性服务或工具，或者配置测试所需的参数等。 + + ```yaml + cgroup2: #$LKP_SRC/setup/cgroup2 executable script + memory.high: 90% + memory.low: 50% + memory.max: max + memory.swap.max: + io.max: + io.weight: + rdma.max: + ``` +- 监控脚本（可选） + + 位于```$LKP_SRC/monitors```路径下。 监视器在运行基准测试时可以捕获性能统计数据，对性能分析和回归根源有价值。 + + ```yaml + proc-vmstats: + nterval: 5 + ``` + +**了解更多 job yaml 相关知识** [Job definition to execution](https://gitee.com/wu_fengguang/lkp-tests/blob/master/jobs/README.md) + + +### 提交测试任务 + +### 查看测试结果 + +### 比较测试结果 + +### 提交 borrow 任务 + +### 提交 bisect 任务 + +## 高级功能 + +### 添加 OS 支持 + +制作一个 initramfs 启动的 cgz 镜像，当系统内核启动时，直接从打包的 cgz 镜像中导出 rootfs，在内存中展开文件系统，利用磁盘的高速缓存机制让系统直接在内存中读写文件，以提升系统的 I/O 性能。 + +#### 操作步骤 + + 1. 获取对应 os 版本的 rootfs（以 openEuler 为例） + - 通过 docker 获取 rootfs + + 1) 下载 openEuler 官方提供的 docker 镜像压缩包 + + ```bash + wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler-... + ``` + 2) 加载 docker 镜像 + ```bash + docker load -i openEuler-docker.aarch64 + ``` + + 3) 启动 openEuler 容器 + ```bash + docker run -id openeuler-20.03-lts + ``` + 4) 拷贝 docker 的 rootfs + ```bash + docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs + ``` + - 通过 qemu.img(qcow2格式)获取 rootfs (openEuler为例) + + + 1) 下载 openEule r官方网站提供的 qcow2 格式镜像 + ```bash + wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/o... + ``` + 2) 使用{compass-ci}/container/qcow2rootfs 制作rootfs + ```bash + cd {compass-ci}/container/qcow2rootfs + ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs + ``` + 2. 定制rootfs + 1. 使用chroot命令切换到 rootfs (此步骤需要 root 权限) + ```bash + chroot openEuler-rootfs + ``` + 2. 根据个人需要安装并配置服务 + + a. 修改 root 密码 + b. 配置 ssh 服务 + c. 检查系统时间 + d. 如果使用 docker 制作 osimage 还需要以下操作： + > 1. 安装所需版本内核 + > 2. 从 centos 官方网站下载内核rpm包 + > 3. 使用 yum 进行安装 + > 4. 删除 docker 环境变量文件 + + 3. 退出 rootfs，并打包 + ```bash + cd $rootfs + find . | coip -o -Hnewc |gzip -9 > $os_name.cgz + ``` +#### FAQ +1. 日志报错 “Unable to mount root fs on unknown-block” + - 问题现象 + ```bash + [ 0.390437] List of all partitions: + [ 0.390806] No filesystem could mount root, tried: + [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) + ... + [ 0.399404] Memory Limit: none + [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- + ``` + - 解决方法 + + 1）启动内存不足，增加内存可解决。 + 2）内核文件权限不足，给予 644 权限。 + +2. 系统运行缓慢 + + - 问题现象 + + 打包镜像体积过大，会消耗很大内存 + - 解决方法 + + 建议用户根据具体需要对 rootfs 进行裁剪。 + + +### 添加测试用例 + 参考文档详见：[添加测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) + + +### PKGBUILD 构建 + +1. 概述： + + 使用 PKGBUILD 完成上游项目源代码的编译构建，同一项目可以使用不同 PKGBUILD 完成项目编译构建过程。 + PKGBUILD 相关参考文档：https://wiki.archlinux.org/index.php/PKGBUILD%E3%80%82 + +2. 步骤： + + 1) 注册仓库 + 如果您想在 git push 的时候, 自动触发测试, 那么需要把您的公开 git url 添加到如下仓库 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos)%E3%80%82 + ```bash + git clone https://gitee.com/wu_fengguang/upstream-repos.git + less upstream-repos/README.md + ``` + + 2) 执行构建测试 + 项目代码更新后自动触发构建任务，无需其他操作。 + + 3) 查看结果 + web: https://compass-ci.openeuler.org/jobs%E3%80%82 + + 4) 备注 + 目前提供 archlinux 下已有 PKGBUILD 项目构建测试，暂不支持自主导入 PKGBUILD 文件。 + PKGBUILD 存放路径： /srv/git/archlinux/*/*/PKGBUILD。 + + +### depends + +- 依赖包构建 + 流程: 以下默认使用 debian:sid 版本进行,如果使用其他OS跑测试用例，则需要在 lkp-tests/distro/adaptation/ 下将包名以键值对的形式写入对应 OS 的文件中 + > 例： `echo 'apache2: httpd' >> lkp-tests/distro/adaptation/openeuler` + + 1) 在lkp-tests/distro/depends/下新建一个与测试用例同名的文件 + `vim lkp-tests/distro/depends/test` + 2) 将测试用例所需debian下的包名填入创建的文件中 + `echo "apache2" >> lkp-tests/distro/depends/test` + 3) 生成lkp-aarch64.cgz + `./crystal-ci/container/lkp-initrd/run` + 4) 提交测试用例 + `submit test.yaml` + 5) 查看结果 + https://compass-ci.openeuler.org/jobs + + + +### 本地搭建 compass-ci 服务器节点 + +在 openEuler 系统一键部署 compass-ci 环境，当前已支持 openEuler-aarch64-20.03-LTS 系统环境，以下配置仅供参考。 + +#### 准备工作 +- 硬件 + 服务器类型：ThaiShan200-2280 (建议) + 架构：aarch64 + 内存：>= 32GB + CPU：64 nuclear (建议) + 硬盘：>= 500G (建议划分独立分区) + + >**说明：** + >划分较小独立分区详细操作请参考[添加测试用例](https://gitee.com/wu_fengguang/compass-ci/blob/master/sparrow/1-storage/smal...) + >划分较大独立分区详细操作请参考[添加测试用例](https://gitee.com/wu_fengguang/compass-ci/blob/master/sparrow/1-storage/larg...) + +- 软件 + OS：openEuler-aarch64-20.03 LTS + git：2.23.0版本 (建议) + 预留空间：>= 300G + 网络：可以访问互联网 + + >**说明：** + >openEuler 系统安装详细操作请参考[添加测试用例](https://openeuler.org/zh/docs/20.03_LTS/docs/Installation/%E5%AE%89%E8%A3%85...) + + +#### 操作指导 +1. 登录 openEuler 系统 + +2. 创建工作目录并设置文件权限 + ```bash + mkdir demo && cd demo && umask 002 + ``` +3. 克隆 compass-ci 项目代码到 demo 目录 + ```bash + git clone https://gitee.com/wu_fengguang/compass-ci.git + ``` +4. 执行一键部署脚本 install-tiny + ```bash + cd compass-ci/sparrow && ./install-tiny + ``` + + +## 呼吁合作 + - 增强 git bisect 能力 + - 增强数据分析能力 + - 增强数据结果可视化能力 + +## 下一步计划 + +1. 优化 git bisect 精准率及效率。 +2. 优化数据分析联动能力，让数据分析更加聚焦。 +3. 数据可视化优化，更加友好展示数据比较结果。 +4. 增强登录认证机制，如：GPG 认证。 +5. 优化部署效率。
-
Wu Zhende