Setting up the lab environment – Deduplication

The next step for the lab or so-called home data center: Installing and Configuring Deduplication

I was going to use a USB stick for the Windows Server 2016 OS.
The main reason for this: DEDUPLICATION.

I did start out with a USB stick, but due to performance issues this was changed – read the follow-up post (https://blog.thomasmarcussen.com/follow-up-on-the-home-datacenter-hardware/)

The reason for having the OS on a separate volume: Deduplication is not supported on system or boot volumes. Read more about Deduplication here: About Data Deduplication

Let’s get started

Installing and Configuring Deduplication

  1. Open an elevated PowerShell prompt
  2. Execute: Import-Module ServerManager
  3. Execute: Add-WindowsFeature -Name FS-Data-Deduplication
  4. Execute: Import-Module Deduplication

Installing Deduplication

Now we installed data Deduplication and it’s ready for configuration.

My Raid 0 volume is D:
The volume will primarily hold Virtual Machines (Hyper-V)
I’m going to execute the following command: Enable-DedupVolume D: -UsageType HyperV

Enable Deduplication for volume

You can read more about the different usage types here: Understanding Data Deduplication

Some quick info for the usage type Hyper-V:

  • Background optimization
  • Default optimization policy:
    • Minimum file age = 3 days
    • Optimize in-use files = Yes
    • Optimize partial files = Yes
  • “Under-the-hood” tweaks for Hyper-V interop

You can start the optimization job and limited (if needed) the amount of consumed memory for the process: Start-DedupJob -Volume “D:” -Type Optimization -Memory 50

 

 

 

You can get the deduplication status with the command: Get-DedupStatus

 

 

 

 

The currently saved space on my volume is 46.17 GB
That is for a 2 ISO files and a reference machine for Windows Server 2016 and the reference disks copied to separate folder.

More usefull powershell cmdlets here: Deduplication Cmdlets in Windows PowerShell

I do love deduplication especially for virtual machines, hence most of the basic data is the same.
The disks are also rather expensive so getting the most out of them is preferred.

 

Follow up on the home datacenter hardware

It’s time for a small update – the previous post is available here: https://blog.thomasmarcussen.com/new-lab-home-datacenter/

The datacenter has been running for about a week now – quite good…. but…..

I’ve been using the Samsung USB as OS drive – Samsung USB 3.0 Flash Drive FIT 32GB
It does have fast read, and a not that slow write, according to Samsung: Up to 130 MB/s

The week passed with setting up and installing VMs – using the actual VMs etc.
But when installing Windows Updates on the Hyper-V host, installing Features/Roles or anykind of configuration, it seems to slow down to useless/freeze.

Running a full Windows Update took about 2 days to reach fully patched level.
During that time it was useless as in no respondig.

I ran a WinSat drive test on the Samsung USB Flash Drive:

Random 16.0 Read: 8.87 MB/s
Random 16.0 Write: 5.45 MB/S

Random reads and writes seems pretty low.

The sequential seems a bit better:

Sequential 64.0 Read: 76.89 MB/s
Sequential 64.0 Write: 86.95 MB/s

The Commands used with winsat:
Winsat disk -drive C: -ran -write (Random 16.0 Write)
Winsat disk -drive C: -ran -read (Random 16.0 Read)
Winsat disk -drive C: -seq -write (Sequential 64.0 Read)
Winsat disk -drive C: -seq -read (Sequential 64.0 Write)

So I decided to replace to Samsung USB 3.0 Flash Drive FIT as a OS Drive.

The new hardware choosen ended up being:

1 x StarTech.com USB 3.0 to M.2 SATA External SSD Enclosure with UASP
1 x Samsung 850 EVO M.2 2280 SSD – 250GB

SM2NGFFMBU33 - StarTech.com USB 3.0 to M.2 SATA External SSD Enclosure with UASPMZ-N5E250BW - Samsung 850 EVO M.2 2280 SSD - 250GB
NOTE: the StarTech.com enclosure does not support NVMe, so did choose a m.2 SSD.

I know that StarTech also have USB 3.1, but i really do want to keep the USB 3.1 port free for an additional RAID enclosure when/if needed. Properly a StarTech enscloure but not sure yet.. (USB 3.1 (10Gbps) External Enclosure for Dual 2.5″ SATA Drives) still looking for a nice USB 3.1 enclosure that supports m.2 NVMe…

Samsung states the specs for the new disk as:

  • Up to 500MB/s Sequential Write
  • Up to 540/s Sequential Read

The actual performance test on the Samsung 850 EVO M.2 2280 SSD:

Random 16.0 Read: 276.51 MB/s
Random 16.0 Write: 271.37 MB/S

Sequential 64.0 Read: 388.85 MB/s
Sequential 64.0 Write: 383.71 MB/s

So in any case it’s quite a performance boost for the OS disk.