NUTANIX DIAGNOSTICS.PY
This is a specific script runs inside the Nutanix CVM and that creates a new container, one VM per CVM, boot these VMs on an ISO image and mount the new container directly into the VM, bypassing the hypervisor altogether with BMC, IPMI, XCC.
The special purpose of this script is to check whether or not the Nutanix cluster is working fine. Nutanix run this test on their side, and they can then check your results against expected results to tell if something is wrong on the performance side.
Open the Nutanix CVM Putty and enter the below commands.
Usage: /home/nutanix/diagnostics/diagnostics.py [command]
commands:
cleanup --Cleans existing configuration
list_runtime_test_args
run -- run all test
run_iperf --network testFollowing is the output of Lenovo HX HARDWARE
nutanix@NTNX-HXNODE-A-CVM::~/diagnostics$ ./diagnostics.py run
Checking if an existing storage pool can be used …
Using storage pool default-storage-pool-568500421 for the tests.
Checking if the diagnostics container/s exists …
Container NTNX-NFS-DEFAULT-0 does not exist.
Creating a new container NTNX-NFS-DEFAULT-0 for the runs … done.
Preparing 1 UVM/s on host 192.168.10.51 …
Importing diagnostics image … done.
Deploying the UVM on host 192.168.10.51 … done.
Adding disks … done.
Preparing 1 UVM/s on host 192.168.10.52 …
Deploying the UVM on host 192.168.10.52 … done.
Adding disks … done.
Preparing 1 UVM/s on host 192.168.10.53 …
Deploying the UVM on host 192.168.10.53 … done.
Adding disks … done.
Waiting for 1 VM on host 192.168.10.51 to bootup … done. 2 remaining.
Waiting for 1 VM on host 192.168.10.52 to bootup … done. 1 remaining.
Waiting for 1 VM on host 192.168.10.53 to bootup … done. 0 remaining.
Collect executable list.
Collect shared library list.
Start up persistent ssh connection to localhost:17000.
ERROR:root:Error connecting through control master
Connecting to localhost:17000 with password authentication.
Start up persistent ssh connection to localhost:17002.
ERROR:root:Error connecting through control master
Connecting to localhost:17002 with password authentication.
Start up persistent ssh connection to localhost:17004.
ERROR:root:Error connecting through control master
Connecting to localhost:17004 with password authentication.
Prepare environment on SVMs.
Preserve original install in /home/nutanix/builds/stock.
Preserve original install in /home/nutanix/builds/stock.
Preserve original install in /home/nutanix/builds/stock.
Transfer shared libraries.
Transfer dynamic linker.
Transfer executables.
Run patchelf.
Transferring the build across the cluster.
Done!
Start running test prepare_disks
2020-06-06_22-22-03: Running setup "Prepare disks" …
done.
Average SVM CPU: : 63% 10.21.31.63: 64% 10.21.31.62: 64%
SSD usage for disks:
Disk id: 34, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 35, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 36, SSD used : 0 GB, SSD Capacity Bytes: 3087 GB, 0% used
Disk id: 37, SSD used : 0 GB, SSD Capacity Bytes: 3133 GB, 0% used
Disk id: 40, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 41, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 42, SSD used : 0 GB, SSD Capacity Bytes: 3145 GB, 0% used
Disk id: 39, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
Disk id: 44, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 45, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 46, SSD used : 0 GB, SSD Capacity Bytes: 3147 GB, 0% used
Disk id: 47, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
SSD usage for storage pool:
SSD tier usage 0 GB, SSD total capacity 38329 GB, SSD Usage 0%
Duration prepare_disks : 21 secs
Start running test fio_seq_write
Waiting for the hot cache to flush ……….. done.
2020-06-06_22-23-09: Running test "Sequential write bandwidth" …
2031 MBps
Average SVM CPU: : 49% 10.21.31.63: 50% 10.21.31.62: 49%
SSD usage for disks:
Disk id: 34, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 35, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 36, SSD used : 0 GB, SSD Capacity Bytes: 3087 GB, 0% used
Disk id: 37, SSD used : 0 GB, SSD Capacity Bytes: 3133 GB, 0% used
Disk id: 40, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 41, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 42, SSD used : 0 GB, SSD Capacity Bytes: 3145 GB, 0% used
Disk id: 39, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
Disk id: 44, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 45, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 46, SSD used : 0 GB, SSD Capacity Bytes: 3147 GB, 0% used
Disk id: 47, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
SSD usage for storage pool:
SSD tier usage 0 GB, SSD total capacity 38329 GB, SSD Usage 0%
Duration fio_seq_write : 21 secs
Start running test fio_seq_read
Waiting for the hot cache to flush ……………. done.
2020-06-06_22-24-33: Running test "Sequential read bandwidth" …
5122 MBps
Average SVM CPU: : 19% 10.21.31.63: 19% 10.21.31.62: 18%
SSD usage for disks:
Disk id: 34, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 35, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 36, SSD used : 0 GB, SSD Capacity Bytes: 3087 GB, 0% used
Disk id: 37, SSD used : 0 GB, SSD Capacity Bytes: 3133 GB, 0% used
Disk id: 40, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 41, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 42, SSD used : 0 GB, SSD Capacity Bytes: 3145 GB, 0% used
Disk id: 39, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
Disk id: 44, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 45, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 46, SSD used : 0 GB, SSD Capacity Bytes: 3147 GB, 0% used
Disk id: 47, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
SSD usage for storage pool:
SSD tier usage 0 GB, SSD total capacity 38329 GB, SSD Usage 0%
Duration fio_seq_read : 10 secs
Start running test fio_rand_read
Waiting for the hot cache to flush …………… done.
2020-06-06_22-25-42: Running test "Random read IOPS" …
428859 IOPS
Average SVM CPU: : 70% 10.21.31.63: 69% 10.21.31.62: 69%
SSD usage for disks:
Disk id: 34, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 35, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 36, SSD used : 0 GB, SSD Capacity Bytes: 3087 GB, 0% used
Disk id: 37, SSD used : 0 GB, SSD Capacity Bytes: 3133 GB, 0% used
Disk id: 40, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 41, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 42, SSD used : 0 GB, SSD Capacity Bytes: 3145 GB, 0% used
Disk id: 39, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
Disk id: 44, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 45, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 46, SSD used : 0 GB, SSD Capacity Bytes: 3147 GB, 0% used
Disk id: 47, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
SSD usage for storage pool:
SSD tier usage 0 GB, SSD total capacity 38329 GB, SSD Usage 0%
Duration fio_rand_read : 102 secs
Start running test fio_rand_write
Waiting for the hot cache to flush …………….. done.
2020-06-06_22-28-30: Running test "Random write IOPS" …
243193 IOPS
Average SVM CPU: : 61% 10.21.31.63: 60% 10.21.31.62: 60%
SSD usage for disks:
Disk id: 34, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 35, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 36, SSD used : 0 GB, SSD Capacity Bytes: 3087 GB, 0% used
Disk id: 37, SSD used : 0 GB, SSD Capacity Bytes: 3133 GB, 0% used
Disk id: 40, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 41, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 42, SSD used : 0 GB, SSD Capacity Bytes: 3145 GB, 0% used
Disk id: 39, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
Disk id: 44, SSD used : 0 GB, SSD Capacity Bytes: 3247 GB, 0% used
Disk id: 45, SSD used : 0 GB, SSD Capacity Bytes: 3303 GB, 0% used
Disk id: 46, SSD used : 0 GB, SSD Capacity Bytes: 3147 GB, 0% used
Disk id: 47, SSD used : 0 GB, SSD Capacity Bytes: 3079 GB, 0% used
SSD usage for storage pool:
SSD tier usage 0 GB, SSD total capacity 38329 GB, SSD Usage 0%
Duration fio_rand_write : 81 secs
Tests done.
Results archived in /home/nutanix/diagnostics/results/2020-06-06_22-20-07
nutanix@NTNX-HXNODE-A-CVM::~/diagnostics$You can follow this article KB for more details how it works :-
https://next.nutanix.com/how-it-works-22/performance-results-of-diagnostics-py-79