HOW NOT TO PERFORM: TESTING HPE StoreVirtual Storage VSA (Very Slow Appliance) on 2 PHYSICAL NODES

– Attention all Cult of Anarchy units, crime in progress – no performance scaling detected. Suspect: HPE StoreVirtual Storage VSA. Proceed immediately.

– Roger that. Initiating pursuit.

MISSION

So far, one suspect continuously managed to avoid our fair trial. HPE StoreVirtual Storage VSA. Store…Virtual…Storage…Virtual…Storage…Appliance…? What the hwem? Ain’t there too much storage and virtualization? OK, it’s a Software-Defined Storage solution based on LeftHand OS (basically, HPE asks what hand do you use to lgtm off) and delivering scalability and high availability. Goes together with HPE servers or can be used as a software separately. It’s main goal (even after 8 years since it was rolled out) is quite simple: virtualize and pool together local server storage to make it shared for the nodes. According to HPE, it is high-performing (especially on SSDs) and should provide linear scalability. Well, now that we are on SDS, let’s investigate if HPE StoreVirtual Storage VSA can scale or goes to jail.

We are sure that our dear readers are pretty much familiar with our testing methodology. However, since we take all of this extremely seriously, we shall tell you about our strategy and tactics one more time. We’re gonna benchmark the performance of a 2-node HPE StoreVirtual Storage VSA (12.7.00.0226.0) by adding more VMs to increase the workload until we hit the saturation point where performance boost stops.

Testing setup configuration diagram:

setup configuration diagram

HARDWARE ARSENAL

Here is our test configuration for a 2-node HPE StoreVirtual Storage VSA:

ESXi Host 1, ESXi Host 2

Dell R730, CPU 2x Intel Xeon E5-2683 v3 @ 2.00 GHz , RAM 64GB

Storage 1: 1x Intel SSD DC P3700 2TB

Storage 2: 1x Intel SSD DC P3700 2TB

LAN: 2x Mellanox ConnectX-4 100GbE

Hypervisor: VMware ESXi 6.5 Update 2

ATTENTION: Not that we are pointing at something but…while we were running this testing, it was 5 months since the ESXi 6.7 release. BUT HPE StoreVirtual Storage VSA (12.7.00.0226.0) and HPE StoreVirtual Centralized Management Console for Windows (HPE StoreVirtual CMC) still do not support ESXi 6.7. When trying to deploy HPE StoreVirtual Storage VSA on ESXi 6.7 hosts, the StoreVirtual CMC simply did not recognize the hosts. So, we had no other choice than using VMware ESXi 6.5 Update 2. So HPE says “StoreVirtual is tightly integrated with VMware®, Microsoft® and Linux® for ease of storage monitoring and provisioning.” Yeah, right…

HPE StoreVirtual 1TB Registration

Hardware Prerequisites

SOFTWARE EQUIPMENT for HPE StoreVirtual Storage VSA VM

CPU:8xCPU (2x Sockets, 4x Cores per Socket)

Memory: 8GB

Hard disk 1: 32MB (vmdk, SCSI controller 0 – VMware Paravirtual)

Hard disk 2: 1860GB (vmdk, Thick Provisioned Eager Zeroed, SCSI controller 1 – VMware Paravirtual);

Hard disk 3: 1820GB (vmdk, Thick Provisioned Eager Zeroed, SCSI controller 1 – VMware Paravirtual);

Network adapter: 2x VMXNET 3

Virtual SAN: HPE StoreVirtual Storage VSA (12.7.00.0226.0) (download link)

Virtual hardware edit settings

NOTE: the maximum number of CPUs we managed to assign for the VM with HPE StoreVirtual Storage VSA, is 8 since the VM deployment image has VM version 7.

Testing VM configuration:
CPU:
4xCPU (4x Sockets, 1x Cores per Socket)

Memory: 4GB

Hard disk 1: 25GB (vmdk, SCSI controller 0 – LSI Logic SAS), location – ESXi local datastore, OS Windows Server 2016 Datacenter

Hard disk 2: 80GB (vmdk, Thick Provisioned Eager Zeroed, SCSI controller 1 – VMware Paravirtual), location – LEFTHAND iSCSI DISK

Virtual hardware edit settings 1

Testing methodology.

1. Verify that Intel SSD DC P3700 meets the vendor claimed performance level on Windows Server 2016.

2. Deploy our testing cluster with HPE StoreVirtual Storage VSA and since there is a detailed HPE StoreVirtual Storage VSA Installation and Configuration Guide, we won’t be boring you with all that reading. We’re just gonna cover some milestones.

To test HPE StoreVirtual Storage VSA performance, we will deploy a 2-node cluster with no Quorum Witness. Our goal is to measure the StoreVirtual Storage VSA speed so it doesn’t matter if there is a Witness. Following the instruction and using HPE StoreVirtual CMC deployed on a separate Windows VM, create 2 VMs running HPE StoreVirtual Storage VSA on the ESXi hosts. Start the VM and connect to the HPE StoreVirtual Storage VSA VMs from HPE StoreVirtual CMC.

3. Create a Windows Server 2016 VM keeping it pinned to ESXi host 1. VMware Virtual Disk 80GB for testing (VMware Paravirtual) located on HPE VSA Datastore and utilized as a second disk inside the VM.

4. Determine the number of threads and Outstanding I/O value that we’re gonna further use in DiskSPD and FIO testing tools.

5. Test VMware Virtual Disk 80GB speed on a single VM.

6. Clone that VM and pin it to the second ESXi host with a new 80GB VMware Virtual Disk on HPE VSA Datastore. Run performance test simultaneously for two VMs.

7. Repeat the procedure till we hit the ceiling were performance growth stops

8. Benchmark the performance of a single Intel SSD DC P3700 2TB disk on Windows Server 2016. The results we get will be used to compare the performance of HPE StoreVirtual Storage VSA.

So, first things first…let’s check if INTEL SSD DC P3700 can give us what it really should.

VERIFYING IF INTEL SSD DC P3700 2TB MEETS THE VENDOR-CLAIMED PERFORMANCE VALUES.

Below, you can see the table for performance power of P3700 disk series (source)

table for performance power of P3700 disk series

According to Intel, the speed claimed was achieved while measuring with 4 workers and Queue Depth 32:
wp-image-2351

OK, let’s try to prove this on Random Read test with DiskSPD v2.0.20a and Fio v.3.8.

Here are the results:

performance intel SSD DC P3700 2TB (RAW) DiskSPD (4k radom read)

performance intel SSD DC P3700 2TB (RAW) FIO (4k radom read)

Well, the Intel guys don’t seem to be lying much about the Intel SSD DC P3700 2TB speed. Under the 4k random read pattern with 4 workers and Queue Depth 32, DiskSPD showed values that are pretty much close to those 460K IOPS claimed for Intel SSD DC P3700 2TB.

Now that we are sure that our NVMe disk are totally fine, we can move on and set up the software part.

DEPLOYING A 2-NODE HPE STOREVIRTUAL STORAGE VSA CLUSTER

2-NODE HPE STOREVIRTUAL STORAGE VSA CLUSTER

Afterwards, on both HPE StoreVirtual Storage VSA VMs, we set up the NICs:

set up the NICs 1

set up the NICs 2

We do that according to the traffic type:

communication

Since we have already created virtual disks on our VMs over the Intel SSD DC P3700 2TB (mounted as vmdk) during the HPE StoreVirtual Storage VSA deployment, we can move on straight to the cluster part.

Figure 1- creating Virtual Disk during HPE VirtualStore Storage VSA deployment process

Figure 1- creating Virtual Disk during HPE VirtualStore Storage VSA deployment process

Figure 2 - creating Virtual Disk inside the HPE StoreVirtual Storage VSA VM

Figure 2 – creating Virtual Disk inside the HPE StoreVirtual Storage VSA VM

Create a Management Group, Cluster, and Volumes. To do that, run Management Group, Cluster and Volumes Wizard in HPE StoreVirtual CMC. Again, no need to write a full guide here, HPE guys have already done this for us, so just some most important moments:

Management Group
Cluster and Volumes Wizard

Management group time
Create a cluster

Assign Virtual IPs and Subnet Masks
Create volume

So having a free HPE StoreVirtual Storage VSA license to test is great! Except for one thing…the overall volume of all virtual disks cannot be higher than just 1 TB. However, HPE StoreVirtual CMC allows us to step over this barrier during the cluster storage creation. On the screenshot above, you can see that we set the size for 2 TB. BUT we get a notice that YOU SHALL NOT PASS! OK, not really, the notice says that we are about to use an unlicensed feature (hmm…storage size as a feature? Nice one). So they have just cut our free period to 60 days. More than enough.

WARNING

And…since we don’t need this ujkv a second longer than 60 days, let’s just click OK.

And what we get is a cluster with the following parameters:

Summary

Parameters

Add both ESXi hosts as iSCSI initiators

Add both ESXi hosts

Assign cluster storage to the iSCSI initiators:

Assign cluster storage

ISCSI Session

That’s pretty much it for HPE StoreVirtual Storage VSA setup:

HPE StoreVirtual Storage VSA setup

With the release of LeftHand OS 12.0, HP introduced the Multipathing Extension Module (MEM) allowing to connect the iSCSI initiator to an iSCSI target over several sessions. The procedure for the deployment of Multipathing Extension Module (MEM) using VMware ESXi CLI and StoreVirtual MEM VIB file is described in HP StoreVirtual Storage Multipathing Deployment Guide. If you want to download StoreVirtual MEM VIB file, just go here.

Software delivery repository

Now, let us make sure that Multipathing Extension Module (MEM) is installed in ESXi:

esxcli software vib list | grep hpe-lh-mem

wp-image-2372

Connect our iSCSI target to the ESXi hosts by adding the virtual IP of our HPE VSA Cluster into the Dynamic Discovery list.

adding the virtual IP

Check if the disk and iSCSI sessions are there and if Multipathing Policy is set:

Devices

Paths

Multipathing policy

Check on HPE StoreVirtual Storage VSA that iSCSI initiators are connected over multiple sessions:

Check connection

Create a VMware Datastore (HPE-VSA-Datastore) on top of the connected LEFTHAND iSCSI Disk

Create a VMware Datastore

Benchmark the throughput on the physical NICs between the ESXi hosts using iperf. Even that we use Mellanox ConnectX-4 CX465A (100Gbit/s ) connected directly port-to-port (no switches) the throughput is 39-43GbE.

script1

script2

script3

script4

Now keep calm. For the numerous times we saw that this is more than enough for testing. And HPE StoreVirtual Storage VSA on Intel SSD DC P3700 is no exclusion. Unfortunately, we cannot measure the network throughput between HPE StoreVirtual Storage VSA hosts and ESXi hosts since there is no such functionality in HPE StoreVirtual CMC. We also cannot connect to VMs over SSH to install the iperf since HPE banned port 22 for some reason (stop us from testing? Nice try). The most we can do is just t take a look at the TCP Status in HPE StoreVirtual CMC…and that doesn’t give us a ujkv.

TCP Status

The hard part is behind. Now, it’s time to see how fast HPE StoreVirtual Storage VSA can go.

STATUS: SUSPECT DETAINED. PROCEEDING TO TRIAL: TESTING HPE STOREVIRTUAL STORAGE VSA PERFORMANCE

So, as we already said, we can check the network throughput between HPE StoreVirtual Storage VSA and between ESXi and HPE StoreVirtual Storage VSA hosts only using the TCP Status tab which shows 10000Mbps. Let’s check if this value is enough for our tests taking into account that a single SSD gives 460K IOPS under the 4k random read pattern.

Suppose that a single VMXNET3 network interface gives the throughput of 10GbE or 1.25GBE. Accordingly, one HPE StoreVirtual Storage VSA VM will have the limit on one NIC (iSCSI traffic) of (1,25GB/s*1024*1024)/4≈327K IOPS. Two disks equal 2*460K=920K IOPS which is much higher than the 327K IOPS limit on a single HPE StoreVirtual Storage VSA VM. Theoretically, network can be a performance bottleneck for HPE StoreVirtual Storage VSA. But, we will not stop our mission here. Let’s still try testing this fellow. Maybe we will really see how it reaches 327K IOPS limit and asks for more? OR NOT.

NOTE. Yes, we have created our VM testing disks as Provision Eager Zeroed. But we still filled the drives with some junk data to get the nominal disk volume prior to testing. We will also do this prior to each test when creating new Virtual Disk (or changing its size) in a VM. And we us dd.exe utility for this.

Here is the example of launching dd:
dd.exe bs=1M if=/dev/random of=\\?\Device\Harddisk1\DR1 –progress

Create a first testing VM pinned to ESXi host 1. Before we start testing PE StoreVirtual Storage VSA performance, let’s first determine the optimal number of threads and Outstanding I/O value for DiskSPD and Fio.

We’re gonna test 80GB VMware Virtual Disk over HPE virtual device under the 4k random read pattern while changing the number of threads and Outstanding I/O value. This should help us find the saturation point after which the performance growth stops with corresponding threads and Outstanding I/O value.

Testing tools: DiskSPD v2.0.20a, Fio v3.8
DiskSPD testing parameters under threads=1, Outstanding I/O=1,2,4,8,16,32,64,128

diskspd.exe -t1 -b4k -r -w0 -o1 -d60 -Sh -L #1 > c:\log\t1-o1-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o2 -d60 -Sh -L #1 > c:\log\t1-o2-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o4 -d60 -Sh -L #1 > c:\log\t1-o4-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o8 -d60 -Sh -L #1 > c:\log\t1-o8-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\t1-o16-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o32 -d60 -Sh -L #1 > c:\log\t1-o32-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o64 -d60 -Sh -L #1 > c:\log\t1-o64-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o128 -d60 -Sh -L #1 > c:\log\t1-o128-4k-rand-read.txt

timeout 10

FIO testing parameters with threads=1, Outstanding I/O=1,2,4,8,16,32,64,128

[global]

numjobs=1

loops=1

time_based

ioengine=windowsaio

direct=1

runtime=60

filename=\\.\PhysicalDrive1

[4k-rnd-read-o1]

bs=4k

iodepth=1

rw=randread

stonewall

[4k-rnd-read-o2]

bs=4k

iodepth=2

rw=randread

stonewall

[4k-rnd-read-o4]

bs=4k

iodepth=4

rw=randread

stonewall

[4k-rnd-read-o8]

bs=4k

iodepth=8

rw=randread

stonewall

[4k-rnd-read-o16]

bs=4k

iodepth=16

rw=randread

stonewall

[4k-rnd-read-o32]

bs=4k

iodepth=32

rw=randread

stonewall

[4k-rnd-read-o64]

bs=4k

iodepth=64

rw=randread

stonewall

[4k-rnd-read-o128]

bs=4k

iodepth=128

rw=randread

stonewall

Testing results under the 4k random read pattern

VMware Virtual Disk 80GB (RAW) over HPE StoreVirtual Storage VSA 4k random read (DiskSPD)

VMware Virtual Disk 80GB (RAW) over HPE StoreVirtual Storage VSA- 4k random read (DiskSPD)
threads=1 threads=2 threads=4 threads=8
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
QD=1 3188 12 0,31 6280 25 0,32 11209 44 0,36 19841 78 0,40
QD=2 6180 24 0,32 11424 45 0,35 18992 74 0,42 25782 101 0,62
QD=4 11552 45 0,35 19528 76 0,41 24817 97 0,64 29942 117 1,07
QD=8 19683 77 0,41 25824 101 0,62 30085 118 1,06 41327 161 1,55
QD=16 26307 103 0,61 30311 118 1,06 42801 167 1,53 41607 163 3,08
QD=32 29970 117 1,07 41411 162 1,51 42581 166 3,01 43108 168 5,94
QD=64 41475 162 1,54 42709 167 3,00 41877 164 6,11 42351 165 12,09
QD=128 42797 167 2,99 41759 163 6,13 41190 161 12,43 42168 165 24,28

VMware Virtual Disk 80GB (RAW) over HPE StoreVirtual Storage VSA 4k random read (Fio)

VMware Virtual Disk 80GB (RAW) over HPE StoreVirtual Storage VSA- 4k random read (FIO)
threads=1 threads=2 threads=4 threads=8
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
QD=1 3038 12 0,32 5915 23 0,33 11179 44 0,35 18902 74 0,41
QD=2 6040 24 0,32 11094 43 0,35 18500 72 0,42 24856 97 0,63
QD=4 11238 44 0,35 18839 74 0,41 25389 99 0,62 29030 113 1,09
QD=8 18792 73 0,41 25044 98 0,63 29407 115 1,08 41116 161 1,55
QD=16 25744 101 0,61 29265 114 1,08 42055 164 1,54 40886 160 3,12
QD=32 30383 119 1,03 40226 157 1,57 42376 166 3,01 40907 160 6,25
QD=64 41475 162 1,51 43771 171 2,91 42345 165 6,03 41356 162 12,37
QD=128 42479 166 2,99 42033 164 6,07 42394 166 12,06 42064 164 24,33

From the tests, we can see that the optimal parameters for our testing utilities are: threads=4 and Outstanding I/O=16.

STATUS: TRIAL IN PROGRESS

OK, now we’re gonna test the 80GB VMware Virtual Disk performance starting with one VM. After this, we clone that VM with a new 80 GB virtual disk on HPE-VSA-Datastore to the second ESXi host and pin it there and run the test once again. Then, repeat the test for all 80GB VMware Virtual Disks. We will repeat the tests till we hit the saturation point were the increase of VMs number stops showing any things of performance growth.

Testing tools: DiskSPD v2.0.20a, Fio v3.8

Testing patterns:

– 4k random write

– 4k random read

– 64k random write

– 64k random read

8k random 70%read/30%write

1M sequential read

Parameters for working with testing utilities: thread=4, Outstanding I/O=16, time=60sec

DiskSPD

diskspd.exe -t4 -b4k -r -w100 -o16 -d60 -Sh -L #1 > c:\log\4k-rand-write.txt

timeout 10

diskspd.exe -t4 -b4k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\4k-rand-read.txt

timeout 10

diskspd.exe -t4 -b64k -r -w100 -o16 -d60 -Sh -L #1 > c:\log\64k-rand-write.txt

timeout 10

diskspd.exe -t4 -b64k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\64k-rand-read.txt

timeout 10

diskspd.exe -t4 -b8k -r -w30 -o16 -d60 -Sh -L #1 > c:\log\8k-rand-70read-30write.txt

timeout 10

diskspd.exe -t4 -b1M -s -w0 -o16 -d60 -Sh -L #1 > c:\log\1M-seq-red.txt

FIO

[global]

numjobs=4

iodepth=16

loops=1

time_based

ioengine=windowsaio

direct=1

runtime=60

filename=\\.\PhysicalDrive1

[4k rnd write]

rw=randwrite

bs=4k

stonewall

[4k random read]

rw=randread

bs=4k

stonewall

[64k rnd write]

rw=randwrite

bs=64k

stonewall

[64k random read]

rw=randread

bs=64k

stonewall

[OLTP 8k]

bs=8k

rwmixread=70

rw=randrw

stonewall

[1M seq read]

rw=read

bs=1M

stonewall

And here are the results:

Perfomance VMware Virtual Disk 80GB (RAW) 4k random write (IOPS)

Perfomance VMware Virtual Disk 80GB (RAW) 4k random write (MB\s)

Perfomance VMware Virtual Disk 80GB (RAW) 4k random read (IOPS)

Perfomance VMware Virtual Disk 80GB (RAW) 4k random read (MB\s)

Perfomance VMware Virtual Disk 80GB (RAW) 64k random write (IOPS)

Perfomance VMware Virtual Disk 80GB (RAW) 64k random write (MB\s)

Perfomance VMware Virtual Disk 80GB (RAW) 64k random read (IOPS)

Perfomance VMware Virtual Disk 80GB (RAW) 64k random read (MB\s)

Perfomance VMware Virtual Disk 80GB (RAW) 8k random 70% read 30% write (MB\s)

Perfomance VMware Virtual Disk 80GB (RAW) 1M seq read, (IOPS)

Perfomance VMware Virtual Disk 80GB (RAW) 1M seq read, (MB\s)

4k random write 4k random read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 16260 64 3,94 16796 66 3,79 74151 290 0,86 72167 282 0,87
2x VM 20328 79 6,30 19908 78 6,41 81582 319 1,57 81378 318 1,56
3x VM 17470 68 15,81 16859 66 16,21 77712 304 3,59 77538 303 3,51
4x VM 15208 59 16,83 14965 58 17,07 71140 278 3,60 70419 275 3,62
5x VM 15781 62 21,15 16219 63 20,74 71900 281 4,60 65794 257 5,00
6x VM 14920 58 25,74 15396 60 24,90 71451 279 5,37 69605 272 5,50
7x VM 15071 59 30,33 15848 62 29,06 70296 275 6,52 66887 261 6,80
8x VM 17421 68 29,51 16676 65 31,73 67148 262 7,63 65232 255 7,83
9x VM 16679 65 35,23 17030 67 36,50 66057 258 8,82 64711 253 8,96
10x VM 16008 63 40,00 16928 66 35,89 66468 260 9,66 65141 255 9,81
11x VM 16000 63 44,61 16505 64 41,13 66072 258 10,78 64685 253 10,96
12x VM 16311 64 47,09 16940 66 44,59 69008 270 11,13 62929 246 12,24
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16
64k random write 64k random read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 14050 878 4,55 13826 864 4,60 30142 1884 2,12 29799 1863 2,13
2x VM 15820 989 8,09 15688 981 8,13 39526 2470 3,24 39689 2481 3,21
3x VM 13490 843 20,02 13485 843 20,17 38358 2397 5,60 38585 2412 5,57
4x VM 13134 821 19,49 12709 795 20,10 37971 2373 6,74 37586 2350 6,79
5x VM 12871 804 25,88 13149 823 25,12 37578 2349 8,88 36034 2253 9,25
6x VM 13030 814 29,47 12021 752 31,88 37108 2319 10,35 36952 2310 10,37
7x VM 12867 804 35,50 11853 742 38,53 36228 2264 12,67 35791 2238 12,74
8x VM 13479 842 38,00 13002 814 39,36 34658 2166 14,77 34793 2176 14,69
9x VM 13209 826 44,45 12567 787 46,34 34170 2136 17,12 35420 2215 16,40
10x VM 13233 827 48,38 12933 809 49,62 33916 2120 18,87 34875 2181 18,34
11x VM 12953 810 54,76 13159 824 53,99 34539 2159 20,58 34345 2148 20,71
12x VM 12781 799 60,10 13019 815 59,47 35336 2209 21,73 33849 2117 22,73
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16
8k random 70%read/30%write 1M seq read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 38284 299 1,67 38281 299 1,97 1441 1441 44,42 1512 1513 42,21
2x VM 40956 320 3,13 40788 319 4,24 2513 2513 50,96 2576 2582 49,51
3x VM 37953 297 7,01 37736 295 7,98 2552 2552 83,02 2544 2549 83,69
4x VM 34652 271 7,39 35565 278 7,66 2502 2502 102,35 2417 2426 105,44
5x VM 35618 278 9,33 36871 288 9,39 2504 2504 132,70 2504 2510 131,98
6x VM 34755 272 11,05 35350 276 11,35 2493 2493 154,11 2415 2429 157,97
7x VM 35497 277 12,90 35034 274 13,57 2539 2539 179,65 2442 2455 185,86
8x VM 36667 286 13,97 37187 291 14,23 2648 2648 193,98 2449 2465 207,78
9x VM 37249 291 15,78 34768 272 17,31 2675 2675 212,75 2440 2452 237,26
10x VM 37036 289 17,28 36498 285 18,14 2706 2706 237,35 2468 2484 258,18
11x VM 34708 271 20,43 37994 297 19,15 2560 2560 277,76 2435 2462 288,78
12x VM 36571 286 21,00 36430 285 21,70 2502 2502 307,32 2480 2510 308,09
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16

TESTING A SINGLE INTEL SSD DC P3700 2TB DISK PERFORMANCE

We have benchmarked Intel SSD DC P3700 2TB disks on Windows Server 2016 under the same patterns. We need this to measure how the performance changes when 4 Intel SSD DC P3700 disks are united into a single LEFTHAND iSCSI Disk and compare this to the values we should have obtained in theory.

Testing lab configuration for Intel SSD DC P3700 2TB

Dell R730, CPU 2x Intel Xeon E5-2683 v3 @ 2.00 GHz , RAM 64GB

Storage: 1x Intel SSD DC P3700 2TB

OS: Windows Server 2016

To ensure resiliency HPE StoreVirtual Storage VSA resiliency in a 2-node scenario, the Network RAID-10 (2-Way Mirror) is used. Here, block that is written to the VSA Volume is copied to each StoreVirtual Storage VSA node and written to the local RAID 0 (Stripe) comprised of 2 Virtual Disks (2 Intel SSD DC P3700 2TB as vmdk inside the VM). To calculate the theoretical performance value in the most straightforward way, let’s make the following assumptions:

1. Since read operations can be performed from all NVMEs simultaneously, let’s count it as 4x for a single Intel SSD DC P3700 reads.

2. Writes can also go to all NVMes simultaneously. However, since each block written to VSA Volume is then replicated to each HPE StoreVirtual Storage VSA node, we’re gonna use a RAID-1 (Mirroring) logic here. That’s why, the theoretical write speed should be calculated as ((IOPS-Write-one-disk)*N)/2, where N is a number of disks utilized for simultaneous writes (4 in our case).

3. As to 8k random 70%read/30%write pattern, the theoretical performance value is calculated as (IOPS-Read-one-disk*N*0.7)+((IOPS-Write-one-disk*N*0.3)/2), where N is a number of disks utilized simultaneously for reads/writes (yes, it’s still 4).

DiskSPD
Intel SSD DC P3700
( Windows Server 2016)
Theoretical values for use 4x Intel SSD DC P3700 Max Performance for
VMware Virtual Disk
over HPE StoreVirtual Storage VSA
Ratio of measured performance to theoretical value
IOPS MB/s IOPS MB/s IOPS MB/s %
4k random write 409367 1599 818734 3198 20328 79 2
4k random read 423210 1653 1692840 6612 81582 319 5
64k random write 30889 1931 61778 3862 15820 989 26
64k random read 51738 3234 206952 12936 39526 2470 19
8k random 70%read/30%write 403980 3156 1419600 5410 40956 320 3
1M seq read 3237 3237 12948 12948 2706 2706 21
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16
Fio
Intel SSD DC P3700
( Windows Server 2016)
Theoretical values for use 4x Intel SSD DC P3700 Max Performance for
VMware Virtual Disk
over HPE StoreVirtual Storage VSA
Ratio of measured performance to theoretical value
IOPS MB/s IOPS MB/s IOPS MB/s %
4k random write 351489 1373 702978 2746 19908 78 3
4k random read 333634 1303 1334536 5212 81378 318 6
64k random write 31113 1945 62226 3890 15688 981 25
64k random read 37069 2317 148276 9268 39689 2481 27
8k random 70%read/30%write 351240 2744 1178000 4616 40788 319 3
1M seq read 3231 3233 12924 12932 2576 2582 20
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

Comparative Perfomance diagram

THE VERDICT

– COA unit, report the status.

– Emm…well…the suspect tried to avoid the pursuit…but he was going so slow we decided to help him…

– Unit, please clarify, what’s the status?

– I tow him to the department…

It’s not even funny anymore, it seems that HPE StoreVirtual Storage VSA is perfectly suitable for providing storage for…well…just storage for some archival data? OK, it’s good if you don’t want to place any VMs on it. Why? Well… during our investigation, on each testing pattern we noticed that the performance was far from close to those theoretical values (327K IOPS) limited by VMXNET3 virtual network adapter. What this means? Well, basically, that VMXNET3 is no bottleneck for HPE StoreVirtual Storage VSA VM performance benchmarking.

According to the results obtained, the HPE StoreVirtual Storage VSA speed goes higher on all patterns only with 1-2 VMs. The further increase of the VMs number gives no boost while on the contrary, the performance downgrades linearly on 4k and 64k read tests with the addition of new VMs.

4k random read: after we added 2 to 4 VMs, the speed dropped from 81K IOPS to 70K IOPS. What about more VMs? The speed still goes down linearly to 63-65K IOPS (admitted by both DiskSPD and Fio).

What about 64k random read? During scaling from 4 to 12 VMs, we have seen a linear performance decrease from 2500МВ/s to 2200MB/s.

We have managed to get some stable performance of 15K-17K IOPS under the 4k random write pattern and 740-840MB/s under the 4k random write pattern on the range from 2 till 12 VMs.

When it comes to 8k random 70%read/30%write pattern, HPE StoreVirtual Storage VSA scales linearly only with the addition of the second VM (41K IOPS). After scaling up to 12 VMs, we received consistently lower performance at the level of 35K-37K IOPS.

And finally, 1M seq read pattern…here, HPE StoreVirtual Storage VSA manage to hold stable performance of 2400-2500MB/s.

How about comparing HPE StoreVirtual Storage VSA Intel SSD DC P3700 on Windows Server 2016? It’s pretty much obvious that HPE StoreVirtual Storage VSA performance is far behind of that of a single NVMe (not even talking about the overall theoretical values the whole system should give).

To put it in the other way HPE StoreVirtual Storage VSA performance is…

2%(DiskSPD), 3%(Fio) from the theoretical value under the 4k random write pattern

5%(DiskSPD), 6%(Fio) from the theoretical value under the 4k random read pattern

26%(DiskSPD), 25%(Fio) from the theoretical value under the 64k random write pattern

19%(DiskSPD), 27%(Fio) from the theoretical value under the 64k random read pattern

3%(DiskSPD), 3%(Fio) from the theoretical value under the 8k random 70%read/30%write pattern

21%(DiskSPD), 20%(Fio) from the theoretical value under the 1M seq read pattern

And once again, let’s take a look in how many times the overall performance of all VMware Virtual Disk over HPE StoreVirtual Storage VSA is lower than the theoretical values:

Diagram of shame

Well, it looks like HPE needs another hand to get the job done. HPE StoreVirtual Storage VSA can store but not perform. Even on NVMe drives. Now we get it! StoreVirtual Storage Very Slow Appliance (VSA)…this makes sense!

4.71/5 (17)

Please rate this