DataCore Hyperconverged VSAN for vSphere – Could things. Get. Any. Sloooower?

We finally got to challenging DataCore Hyperconverged VSAN performance. Is it a decent VMware vSAN alternative or one more mediocre SDS solution? Let’s find it out!

Line-up

challenging DataCore Hyperconverged VSAN performance

Born to ditch physical SANs, DataCore has built an “enterprise-class” SDS solution for “high-performance, highly available, and cost-effective hyperconverged infrastructure.” Well, that’s dwnnujkv we typically hear from any vendor, but we are here to test those claims…

Wait… Before the ujkv hits the fan, we’d like to say that there are many good things about DataCore Hyperconverged Virtual SAN and DataCore itself. First, these guys are known for working fcop hard on improving their software. Sometimes, they go crazy: UPDATES ARRIVE MORE FREQUENTLY THAN APPS ON A DNQQFA ANDROID SMARTPHONE UPDATE!!! For a big environment, it can be a problem: once you are done with updating the whole infrastructure, the new release comes, and you are to start everything from the very beginning… Second, DataCore Hyperconverged VSAN can be run on only 2 off-the-shelf servers, which seems quite good for guys in small business. No third entity like VMware does it. Just. Two. Nodes! Fourth, DataCore did a really good job on storage tiering: you can tier data not only between hosts but also between servers (and, it’s a truly unique thing, you know)! And, last but not least, DataCore’s VSAN features Parallel I/O, allowing to keep IOPS rocket-high (well, that’s the thing we gonna check).

Now, let’s start looking critically at what DataCore says… These guys promise crazy expandability and awesome performance improvement (up to 5 times!). Well, to start with, it is interesting for us WHAT WAS THE REFERENCE TO WHICH THEY COMPARED THAT GAIN. Was it an old snail? A drunken bear? Nobody knows… Don’t you think that it is far too much for a solution that is deployed inside a VM, anyway? You know, if DataCore Hyperconverged Virtual SAN delivers the performance that we expect (ok, 50% of underlying storage performance [none of vSAN-like solutions got that far…]) and scalability, we gonna send DataCore some beer…

The backstage

Toolkit

The solution was deployed in a “poor man’s” setup consisting only of two nodes. Both hosts (i.e., ESXi Host #1 and ESXi Host #2) are identical on hardware side. Server configuration follows:

  • Dell R730, CPU 2x Intel Xeon E5-2683 v3 @ 2.00 GHz (14 stones per CPU), RAM 64GB
  • Storage 1: 1x Intel SSD DC P3700 2TB
  • Storage 2: 1x Intel SSD DC P3700 2TB
  • LAN: 3x Mellanox ConnectX-4 100Gbit/s
  • Hypervisor: VMware ESXi 6.7 Update 1

Now, to make it clear how we connected everything here, take a look at the interconnection diagram.

interconnection diagram

The toolkit for measuring Intel SSD DC P3700 bare-metal performance

  • Chassis: Dell R730, CPU 2x Intel Xeon E5-2683 v3 @ 2.00 GHz , RAM 64GB
  • Storage: 1x Intel SSD DC P3700 2TB
  • OS: Windows Server 2016

Control VM configuration

  • CPU: 14xCPU (14x Sockets, 1x Cores per Socket)
  • Memory: 12GB
  • Hard disk 1: 100GB (vmdk, Thick Provisioned Lazy Zeroed , SCSI controller 0 – LSI Logic SAS)
  • Hard disk 2: 1768GB (vmdk, Thick Provisioned Lazy Zeroed, SCSI controller 1 – VMware Paravirtual);
  • Hard disk 3: 1768GB (vmdk, Thick Provisioned lazy Zeroed, SCSI controller 1 – VMware Paravirtual);
  • Network adapter: 3x VMXNET 3
  • Virtual SAN: DataCore Hyperconverged Virtual SAN v10 PSP8. Download the solution here: https://www.datacore.com/products/hyper-converged-infrastructure/download/

The image below shows what the control VM configuration looked like in real life.

the control VM configuration looked like in real life

NOTE: Since the default configuration of DataCover Hyperconverged Virtual SAN control VM looks pretty hwemgf up (8 2-GHz vCPUs is not the thing we needed), we decided to fine-tune it a little. And, here the “fun” began… Should you try adding any more vCPU while deploying the control VM… THAT PIECE OF UJKV JUST WON’T START! Fortunately, unlike UjkvMagic StoreMagic SvSAN, DataCore allows playing around with control VM parameters… but a bit later. Below, we provide a guide on how to change the number of vCPUs.

guide on how to change the number of vCPUs

Test VM configuration

  • CPU: 4xCPU (4x Sockets, 1x Cores per Socket)
  • Memory: 4GB
  • Hard disk 1: 25GB (vmdk, SCSI controller 0 – LSI Logic SAS), location – ESXi local datastore, OS – Windows Server 2016 Datacenter. This is a “system” virtual disk – the device where guest OS resides. We do not give a ujkv about its performance here.
  • Hard disk 2: 80GB (vmdk, Thick Provisioned Eager Zeroed, SCSI controller 1 – VMware Paravirtual), location – DataCore iSCSI Disk. It is the “data” virtual disk – the device that keeps virtual machine data. It’s actually the disk that determines VM performance. Therefore, under VM performance in this article, we imply this very disk performance.

edit settings

“Soundcheck”: Testing underlying storage and installing the solution

Testing underlying storage

Before we try running DataCore vSAN at full throttle, let’s check whether our blazing fast NVMes are still that good. We are not going to check each drive since we have them running under the same conditions inside two boxes that stand in one basement. That’s why we assume that if one disk is doing well, others should be fine too. We run tests in Windows Server 2016 bare-metal environment just as usual.

We compare our performance measurement results versus ones Intel provides in its datasheet. The table below is the screenshot from that document. Find what the SSD performance should be like.

We compare our performance measurement results versus ones Intel provides in its datasheet

According to Intel’s testing methodology, they got that performance under Queue Depth = 32 with 4 workers (for random workload).

Queue Depth = 32 with 4 workers

Let’s see now whether our disks are still that fast. We ran tests under the 4k random read pattern. The performance was measured with DiskSPD v2.0.20a and Fio v3.8. Tables and plots below illustrate what drive’s performance was like.

performance Intel SSD DC P3700 2TB (RAW) DiskSPD (4K random read)

performance Intel SSD DC P3700 2TB (RAW) FIO (4K random read)

Mini-conclusion

Our Intel SSD DC P3700 is doing great: its performance matches the value claimed by its vendor. Let’s deploy the solution now!

Installing DataCore Hyperconverged Virtual SAN & fine-tuning the control VM

Today, we run the free version of DataCore Hyperconverged Virtual SAN for vSphere in our cluster. You can download it here: https://www.datacore.com/products/hyper-converged-infrastructure/download/… But you need to register first!

Request a Free Hyperconverged Virtual SAN

Once you are done with registration, check your e-mail. One more time: you’d better use some 10-minute throwaway one than the personal one.

Once you are done with registration, check your e-mail

There should be an email from DataCore, where you can find the download links, license keys, and DataCore Virtual SAN Design and Deployment Guide.

email from DataCore, where you can find the download links, license keys

Since today we are looking for a vSAN alternative, we’ve downloaded DataCore Hyperconverged Virtual SAN for vSphere. Click the link, go to the appropriate website directory, enter your email, and press Download.

Click the link, go to the appropriate website directory, enter your email, and press Download

Once you are done with downloading, copy the file on a Windows host that can talk to both ESXi hosts. First, open the Installation manager.

Once you are done with downloading, copy the file on a Windows host that can talk to both ESXi hosts

Accept the license agreement and press Next to move further.

Accept the license agreement and press Next to move further

Once the wizard finishes, click Finish to exit the wizard.

Once the wizard finishes, click Finish to exit the wizard.

Go to the desktop and launch Installation Manager for vSphere. If you are one of those visual guys, here’s how the installation manager looks like.

Go to the desktop and launch Installation Manager for vSpehere

Here starts an interesting part: changing control VM parameters. Enter vCenter IP and credentials first and press Login.

Enter vCenter IP and credentials first and press Login

Select the vCenter datacenter from the list and click Go.

Select the vCenter datacenter from the list and click Go.

In the Networking for DataCore Hyperconverged Virtual Machines window, add some virtual switches and networks. You can add and remove any element of your infrastructure using those ugly + and – buttons at the top. Assign virtual switches and networks to the physical adapters. You can switch between ESXi hosts by choosing them from the ESXi host dropdown menu. Once you are done with setting up the networking in the DataCore Hyperconverged Virtual Machines interface, click OK.

Once you are done with setting up the networking in the DataCore Hyperconverged Virtual Machines interface, click OK

Now, do some changes in DataCore Hyperconverged Virtual Machine Settings. Just press Edit and do some magic. DataCore, thanks for providing us with that functionality!

do some changes in DataCore Hyperconverged Virtual Machine Settings. Just press Edit and do some magic.

Enter the desired control VM parameters and press Ok to return to the wizard.

Enter the desired control VM parameters and press Ok to return to the wizard

Press Deploy to deploy control VMs on both hosts.

Press Deploy to deploy control VMs on both hosts

Press Finish once the deployment process is over.

Press Finish once the deployment process is over

Network check

RDP to both control VMs and benchmark network bandwidth with iperf.

RDP to both control VMs and benchmark network bandwidth with iperf (1)
RDP to both control VMs and benchmark network bandwidth with iperf (2)
RDP to both control VMs and benchmark network bandwidth with iperf (3)
RDP to both control VMs and benchmark network bandwidth with iperf (4)

RDP to both control VMs and benchmark network bandwidth with iperf (5)
RDP to both control VMs and benchmark network bandwidth with iperf (6)

For the pre-configured control VMs, network bandwidth uwemu. Oh, come on, 11.1-12.7 Gbit/s… are you kidding, DataCore? Ok, let’s do some quick math to see whether we still can access underlying storage performance regardless such a ujkvvay bandwidth.

A single VMXNET3 vNIC provides 12.7 Gbit/s (1.58 GB/s) network bandwidth. Converted into IOPS, each VMXNET3 vNIC can provide (1,58GB/s*1024*1024)/4≈414K IOPS at best. There are also two DataCore Hyperconverged Virtual SAN for vSphere instances running in the cluster. This means that at certain point two SSDs are going to be accessed through one vNIC, so that thing should potentially deliver 2*460K = 920K IOPS. However, such a poor networking bandwidth won’t let to full throttle those NVMes. All these calculations prove that VMXNET3 vNICs of control VMs are very likely to be the bottleneck due to which the overall cluster performance gonna be hwemgf up… You know, we start “loving” that solution!

Tuning the guitars (again): How the solution is deployed

Go to any control VM and start DataCore Management Console on it.

Here’s how the shortcut looks like.

Here’s how the shortcut looks like

For more details on how to use DataCore Hyperconverged Virtual SAN for vSphere, look through what these guys write on their solution https://docs.datacore.com/SSV-WebHelp/welcome.htm. Today, we focus only on the key points of setting up that thing.

Since we started DataCore Management Console from one of control VMs, just keep the default server name and IP (localhost).

we started DataCore Management Console from one of control VMs, just keep the default server name and IP (localhost)

Dear lord… we’ve heard that DataCore Management Console is a piece of ujkv, but NOW WE ARE “ENJOYING” THE HANDS-ON EXPERIENCE!

After connecting the Console to control VMs, you’ll get informed that your license expired.

your license expired

Go to the License tab and press Activate Product Key.

Go to the License tab and press Activate Product Key

Select the automatic authentication and activate DataCore Hyperconverged Virtual SAN for vSphere on both nodes. Press Next once you are done.

Select the automatic authentication and activate DataCore Hyperconverged Virtual SAN for vSphere on both nodes. Press Next

One more license agreement is to be accepted, blah-blah-blah… just check the checkbox below and click Activate!

license agreement is to be accepted, just check the checkbox below and click Activate

Here’s what we got after automatic licensing.

Here’s what we got after automatic licensing

Create a disk pool for each control VM.

Create a disk pool 1 for each control VM

Create a disk pool 2 for each control VM

After the pool creation, let’s create a virtual disk (Virtual Disk 1). First, select the virtual disk type (we opted for Mirrored here) and size.

select the virtual disk type (we opted for Mirrored here) and size

Next, specify the nodes where Mirrored virtual disk resides. Click Next to move on.

specify the nodes where Mirrored virtual disk resides. Click Next

Once you are done with virtual disk creation, let’s move to something that is called “Advanced Options”. Namely, we set up a storage profile (Critical) and mirror path. You can find out more about storage profiles here.

let’s move to something that is called “Advanced Options”

Next, we created the second virtual disk (Virtual Disk2).

Connecting virtual disks to the hosts

Now, let’s see how to connect a virtual disk to the host. The process looks fairly similar for both virtual disks, so we discuss that procedure only for one disk.

In the Common Actions tab, press Serve Virtual Disks.

In the Common Actions tab, press Serve Virtual Disks.

Next, select the host to serve the virtual disk to. Click Next once you select one.

select the host to serve the virtual disk to., click Next

Select the virtual disk.

Select the virtual disk.

Select Redundant path for the path mode. DataCore Management Console should automatically set up the path from ESXi hosts to DataCore servers. In our case, though, Mirror Network 1 (the network responsible for sync) was selected as a Target port. Hwem, we’ve reserved it for sync between DataCore Server instances at the very beginning while tinkering the control VMs! DataCore, could you just be normal? Considering that quirk, we had to configure ports manually.

Select Redundant path for the path mode

Next, we repeated this procedure for Virtual Disk2.

Setting up ESX hosts

As we are done with setting up DataCore, let’s finally configure the hosts themselves. In Static Discovery, add both DataCore Server instances and press Rescan Adapter afterward.

In Static Discovery, add both DataCore Server instances and press Rescan Adapter afterward

After doing adapter rescan, see whether DataCore iSCSI Disk is there and everything is Ok with path to it.

 doing adapter rescan, see whether DataCore iSCSI Disk is there and everything is Ok with path to it

 doing adapter rescan, see whether DataCore iSCSI Disk is there and everything is Ok with path to it

 doing adapter rescan, see whether DataCore iSCSI Disk is there and everything is Ok with path to it

Next, create a datastores on both hosts connected to DataCore iSCSI Disk.

create a datastores on both hosts connected to DataCore iSCSI Disk
create a datastores on both hosts connected to DataCore iSCSI Disk

Picking up the optimal testing parameters

So, let’s start with a single test VM pinned to an ESXi host. We’ve decided on the test VM parameters at the very beginning of this article. The thing we are going to do now is picking the optimal test utility settings (i.e., number of threads and outstanding IOs).

To pick the test utilities’ parameters that ensure the best possible performance, we ran a bunch of tests under a varying number of threads and outstanding IO. Once VM performance saturates, this means that we’ve found the optimal test utility parameters.

DiskSPD (threads=1, Outstanding I/O=1,2,4,8,16,32,64,128)

diskspd.exe -t1 -b4k -r -w0 -o1 -d60 -Sh -L #1 > c:\log\t1-o1-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o2 -d60 -Sh -L #1 > c:\log\t1-o2-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o4 -d60 -Sh -L #1 > c:\log\t1-o4-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o8 -d60 -Sh -L #1 > c:\log\t1-o8-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\t1-o16-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o32 -d60 -Sh -L #1 > c:\log\t1-o32-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o64 -d60 -Sh -L #1 > c:\log\t1-o64-4k-rand-read.txt

timeout 10

diskspd.exe -t1 -b4k -r -w0 -o128 -d60 -Sh -L #1 > c:\log\t1-o128-4k-rand-read.txt

timeout 10
FIO (threads=1, Outstanding I/O=1,2,4,8,16,32,64,128)

[global]

numjobs=1

loops=1

time_based

ioengine=windowsaio

direct=1

runtime=60

filename=\\.\PhysicalDrive1

[4k-rnd-read-o1]

bs=4k

iodepth=1

rw=randread

stonewall

[4k-rnd-read-o2]

bs=4k

iodepth=2

rw=randread

stonewall

[4k-rnd-read-o4]

bs=4k

iodepth=4

rw=randread

stonewall

[4k-rnd-read-o8]

bs=4k

iodepth=8

rw=randread

stonewall

[4k-rnd-read-o16]

bs=4k

iodepth=16

rw=randread

stonewall

[4k-rnd-read-o32]

bs=4k

iodepth=32

rw=randread

stonewall

[4k-rnd-read-o64]

bs=4k

iodepth=64

rw=randread

stonewall

[4k-rnd-read-o128]

bs=4k

iodepth=128

rw=randread

stonewall

Measurement results

WMvare Virtual Disk 80GB (RAW) over DataCOre Hyperconverged Virtual San 4k random read (DiscSPD)

VMware Virtual Disk 80GB (RAW) over DataCore Hyperconverged Virtual SAN for vSphere
– 4k random read (DiskSPD)
threads=1 threads=2 threads=4 threads=8
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
QD=1 4721 18 0.21 9680 38 0.21 18418 72 0.22 32733 128 0.24
QD=2 11285 44 0.18 22553 88 0.18 41778 163 0.19 59427 232 0.27
QD=4 24318 95 0.16 45404 177 0.18 60668 237 0.26 81267 317 0.39
QD=8 45924 179 0.17 61552 240 0.26 83505 326 0.37 85801 335 0.75
QD=16 62270 243 0.26 84045 328 0.38 85384 334 0.77 82887 324 1.54
QD=32 84299 329 0.38 85073 332 0.75 82829 324 1.55 87499 342 2.93
QD=64 84729 331 0.74 85411 334 1.50 86103 336 2.97 86175 337 5.94
QD=128 82976 324 1.54 85635 335 2.99 83288 325 6.15 86280 337 11.87

WMvare Virtual Disk 80GB (RAW) over DataCOre Hyperconverged Virtual San 4k random read (FIO)

VMware Virtual Disk 80GB (RAW) overFree DataCore Hyperconverged Virtual SAN for vSphere – 4k random read (FIO)
threads=1 threads=2 threads=4 threads=8
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
QD=1 4204 16 0.23 8537 33 0.23 16741 65 0.23 29271 114 0.26
QD=2 8301 32 0.23 16447 64 0.24 30921 121 0.25 46835 183 0.33
QD=4 15857 62 0.24 30374 119 0.25 55075 215 0.28 59081 231 0.53
QD=8 29072 114 0.26 54192 212 0.28 81477 318 0.38 62534 244 1.01
QD=16 54605 213 0.28 83619 327 0.37 83930 328 0.75 62208 243 2.05
QD=32 83039 324 0.37 86609 338 0.72 85470 334 1.49 62104 243 4.11
QD=64 84179 329 0.74 87334 341 1.45 85408 334 2.99 62259 243 8.21
QD=128 85791 335 1.47 83823 327 3.04 80631 315 6.34 61589 241 16.62

Mini-conclusion

According to the plots above, the optimal parameters are threads=4 and Outstanding I/O=16.

The show finally starts: Testing

Now, as we have set up everything, we can finally start performance and scalability tests!

Scalability

We judged on solution scalability based on the overall performance of the cluster that was gradually swarmed by VMs. Starting with a lonely VM in the cluster, we kept on populating it with VMs, assigning VMs to host 1 and 2 in turns. The testing stopped once there was no performance gain after spawning another VM in the cluster.

So, here’s the list of patterns:

  • 4k random write
  • 4k random read
  • 64k random write
  • 64k random read
  • 8k random 70%read/30%write
  • 1M sequential read

Here are test utilities’ listings:

DiskSPD

diskspd.exe -t4 -b4k -r -w100 -o16 -d60 -Sh -L #1 > c:\log\4k-rand-write.txt

timeout 10

diskspd.exe -t4 -b4k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\4k-rand-read.txt

timeout 10

diskspd.exe -t4 -b64k -r -w100 -o16 -d60 -Sh -L #1 > c:\log\64k-rand-write.txt

timeout 10

diskspd.exe -t4 -b64k -r -w0 -o16 -d60 -Sh -L #1 > c:\log\64k-rand-read.txt

timeout 10

diskspd.exe -t4 -b8k -r -w30 -o16 -d60 -Sh -L #1 > c:\log\8k-rand-70read-30write.txt

timeout 10

diskspd.exe -t4 -b1M -s -w0 -o16 -d60 -Sh -L #1 > c:\log\1M-seq-red.txt
FIO

[global]

numjobs=4

iodepth=16

loops=1

time_based

ioengine=windowsaio

direct=1

runtime=60

filename=\\.\PhysicalDrive1

[4k rnd write]

rw=randwrite

bs=4k

stonewall

[4k random read]

rw=randread

bs=4k

stonewall

[64k rnd write]

rw=randwrite

bs=64k

stonewall

[64k random read]

rw=randread

bs=64k

stonewall

[OLTP 8k]

bs=8k

rwmixread=70

rw=randrw

stonewall

[1M seq read]

rw=read

bs=1M

stonewall

And, results

Perfomance WMvare Virtual Disk 80GB (RAW) 4k random write (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 4k random write (MB/s)

Perfomance WMvare Virtual Disk 80GB (RAW) 4k random read (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 4k random read (MB/s)

Perfomance WMvare Virtual Disk 80GB (RAW) 64k random read (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 64k random write (MB/s)

Perfomance WMvare Virtual Disk 80GB (RAW) 64k random read (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 64k random read (MB/s)

Perfomance WMvare Virtual Disk 80GB (RAW) 8k random 70%read/30% write (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 8k random 70%read/30% write (MB/s)

Perfomance WMvare Virtual Disk 80GB (RAW) 1M seq read (IOPS)

Perfomance WMvare Virtual Disk 80GB (RAW) 1M seq read (MB/s)

4k random write 4k random read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 21389 84 2.99 20246 79 3.15 85117 332 0.76 83311 325 0.97
2x VM 36071 141 3.55 30744 120 4.15 125596 491 1.02 84491 330 1.56
3x VM 34496 135 7.02 30034 117 7.15 122389 478 1.74 88403 345 2.96
4x VM 32866 128 7.85 29031 113 8.84 90751 355 2.83 79129 309 3.25
5x VM 30246 118 10.83 30110 118 10.94 95762 374 3.58 77645 303 4.39
6x VM 29017 113 13.27 29732 116 12.92 94186 368 4.22 75968 297 5.04
7x VM 27265 107 16.46 30239 118 15.15 111688 436 4.23 73320 286 6.23
8x VM 29492 115 17.38 26317 103 19.89 106546 416 4.98 77088 301 6.65
9x VM 29866 117 19.58 28767 112 20.12 98531 385 5.96 80045 313 7.34
10x VM 29125 114 22.02 27865 109 22.93 98143 383 6.57 92642 362 7.08
11x VM 27119 106 27.46 29348 115 24.01 98694 386 7.32 86599 338 8.74
12x VM 27567 108 29.04 28857 113 26.69 94876 371 8.10 78487 307 9.83
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16
64k random write 64k random read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 7372 461 8.70 8312 520 7.68 22988 1437 2.78 19921 1245 3.19
2x VM 9174 573 14.06 11744 734 11.06 40269 2517 3.24 39637 2478 3.20
3x VM 10168 636 22.18 11781 737 19.19 37420 2339 5.98 37626 2352 5.87
4x VM 9765 610 26.27 11688 731 21.87 34336 2146 7.54 32283 2018 7.91
5x VM 12893 806 25.88 11345 710 29.29 32392 2025 10.39 32588 2037 10.24
6x VM 12205 763 31.95 11528 721 33.57 33509 2094 11.49 32458 2029 11.81
7x VM 11769 736 40.68 10793 676 41.67 31838 1990 14.48 31868 1993 14.50
8x VM 11252 703 46.06 11341 710 45.55 30535 1908 16.77 31424 1965 16.29
9x VM 11692 731 50.99 10980 687 55.80 30107 1882 19.45 30291 1894 19.23
10x VM 11465 717 56.18 11621 728 55.05 29375 1836 21.79 29941 1872 21.34
11x VM 12188 762 59.05 12415 777 59.04 29334 1833 24.24 29641 1854 23.92
12x VM 12376 773 62.81 12270 767 60.01 29604 1850 25.95 29580 1850 25.92
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16
8k random 70%read/30%write 1M seq read
DiskSPD FIO DiskSPD FIO
IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms) IOPS MB/s Latency (ms)
1x VM 38154 298 1.68 29782 233 2.40 1548 1548 41.34 1476 1479 43.20
2x VM 55047 430 2.34 54367 425 2.62 3128 3128 40.92 2958 2963 43.10
3x VM 49042 383 4.27 50340 393 4.33 3054 3054 70.00 2980 2988 70.86
4x VM 47605 372 7.41 48754 381 5.35 3020 3020 84.84 3011 3021 84.64
5x VM 48360 378 6.94 48452 379 7.08 3038 3038 109.76 2975 2987 111.83
6x VM 49678 388 7.77 49818 389 7.83 2991 2991 128.71 2950 2963 129.50
7x VM 50281 393 9.31 49365 386 9.52 2981 2981 153.38 2920 2940 155.64
8x VM 47663 372 10.76 49394 386 10.49 2919 2919 175.58 2908 2920 175.20
9x VM 48688 380 12.13 49083 384 12.24 2875 2875 202.63 2900 2916 199.65
10x VM 48462 379 13.23 49311 385 13.11 2919 2919 219.75 2880 2900 220.70
11x VM 49209 384 14.57 49459 387 14.82 2939 2939 242.45 2949 2967 239.69
12x VM 50510 395 15.26 50525 395 15.33 3004 3004 257.77 2914 2937 261.57
threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16 threads=4 Outstanding I/O=16

Performance

Now, let’s get some numbers for performance. But, numbers themselves are hwemkpi useless unless you have some reference! Our reference today is Intel SSD DC P3700 bare-metal performance in Windows Server 2016 environment. Let’s see how pooling four these crazy fast drives and running Hyperconverged Virtual SAN on top will affect the overall cluster performance.

To build a fault-tolerant 2-node DataCore Hyperconverged Virtual SAN cluster, we’ve set up synchronous mirroring. Every block of data written to iSCSI Disk is subsequently written to local DataCore Pools (DataCore Server instances) comprised of 2 virtual disks (each Intel SSD DC P3700 2 TB disks is presented as a VMDK). Data is striped between those disks (RAID 0) So, generally speaking, data is transmitted much as in RAID 0+1.

Data is striped between those disks (RAID 0) So, generally speaking, data is transmitted much as in RAID 0+1

This being said, there are a bunch of assumptions made to predict the performance:

  1. Data is read from all disks in the capacity tier. As we have 4 drives in the underlying storage today, we expect the overall reading performance to be 4x Intel SSD DC P3700.
  2. Writing can be described with this formula:

    N stands here for the total number of disks in the setup (4 in our case), ½ is a coefficient that includes the number of mirrors.
  3. Under 8k random 70%read/30%write workloads, the overall performance fits this formula:
    8k random 70%read/30%write workloads

One more time, N here represents the total number of disks in the setup.

Now, we can go to the measurement results.

DiskSPD
Intel SSD DC P3700
( Windows Server 2016)
Theoretical values for use 4x Intel SSD DC P3700 Max Performance for
VMware Virtual Disk
over DataCore Hyperconverged Virtual SAN
Ratio of measured performance to the theoretical value
IOPS MB/s IOPS MB/s IOPS MB/s %
4k random write 409367 1599 818734 3198 36071 141 4
4k random read 423210 1653 1692840 6612 125596 491 7
64k random write 30889 1931 61778 3862 12893 806 21
64k random read 51738 3234 206952 12936 40269 2517 19
8k random 70%read/30%write 403980 3156 1419600 5410 55047 430 4
1M seq read 3237 3237 12948 12948 3128 3128 24
threads=4

Outstanding I/O=16

threads=4

Outstanding I/O=16

Fio
Intel SSD DC P3700
( Windows Server 2016)
Theoretical values for use 4x Intel SSD DC P3700 Max Performance for
VMware Virtual Disk
over DataCore Hyperconverged Virtual SAN
Ratio of measured performance to the theoretical value
IOPS MB/s IOPS MB/s IOPS MB/s %
4k random write 351489 1373 702978 2746 30744 120 4
4k random read 333634 1303 1334536 5212 92642 362 7
64k random write 31113 1945 62226 3890 12415 777 20
64k random read 37069 2317 148276 9268 39637 2478 27
8k random 70%read/30%write 351240 2744 1178000 4616 54367 425 5
1M seq read 3231 3233 12924 12932 3011 3021 23
threads=4

Outstanding I/O=16

threads=4

Outstanding I/O=16

Measurements with DiskSPD

comperative perfomance diagram (diskSPD test utility)

comperative perfomance diagram (diskSPD test utility)

Measurements with FIO

comperative perfomance diagram (FIO test utility)

comperative perfomance diagram (FIO test utility)

The show is over (because it uwemu)

Generally speaking, DataCore Hyperconverged Virtual SAN for vSphere is one more outstanding example of SDS… Ujkv Defined Storage! Of course, one might argue that it has higher performance than StorMagic SvSAN and HPE StoreVirtual Storage VSA. Furthermore, the solution even beats VMware vSAN under some patterns (i.e., 4k random writing, 64k random writing, 8k random 70%read/30%write)! Nevertheless, it is still far from what we call a GOOD solution [therefore, no beer to DataCore]! Let us show what we are talking about.

First, that thing doesn’t scale. “Increased VM density”? DWNNUJKV!!! DataCore Hyperconverged VSAN ujkvu into its pants under all patterns as soon as 2 VMs get on board! Interestingly, adding more virtual machines to the cluster somehow makes random reading even slower (4k random reads and 64 random reads)!

Under mixed workloads (8k random 70%read/30%write), DataCore Hyperconverged VSAN performance reached 54K IOPS but later, when 12 VMs were spawned in the cluster, it dropped to 48-50K IOPS. Under 1M sequential reads, performance saturated once 2nd VM got on board.

In terms of performance, DataCore Hyperconverged Virtual SAN looks like a mediocre piece of ujkv. The solution delivers less IOPS than a single Intel SSD DC P3700 can ever provide… Welcome to the Hall of Shame, u ewpv!

Now comes the traditional Diagram of Shame [SLOW heavy metal music playing]! Here, values on the How hwemkpi disappointed we are axis represents how many times the overall performance is lower than we expected.

diagram of shame

3.69/5 (16)

Please rate this