I just finished implementing the new Windows Deduplication on some of my backup volumes and decided to share the results. I’m pretty impressed with the numbers. Its pretty easy to set up, just install the Role Service, then right click a volume in server manager to turn it on. It creates scheduled tasks that you can adjust.
Drive E consists primarily of MS SQL bak files. I went from a total used space of 969gb down to 547gb. Thats a 44% dedupe ratio. I saved 434gb.
The initial dedupe job took just over 13 hours.
Drive F consists primarily of VM archives. Mostly vhd’s but also some vmdk’s. I went from a total used space of 1.75tb down to 468gb. That’s a 74% dedupe ratio.
The initial dedupe job took almost 25 hours.
Drive D consists of Veeam backup files and Acronis backup files (vbk’s and tib’s). I went from a total used space of 1.07tb down to 562gb. That’s a 50% dedupe ratio.
The initial dedupe job took 14.5 hours.
Here are the results from another server. This drive has ONLY Veeam backups on it. Keep in mind that this is after Veeam has already applied its own dedupe (and compression) from within the backup repository.
Capacity : 1.57 TB
FreeSpace : 1 TB
UsedSpace : 583.38 GB
UnoptimizedSize : 960.54 GB
SavedSpace : 377.16 GB
SavingsRate : 39 %
OptimizedFilesCount : 21
OptimizedFilesSize : 834.38 GB
OptimizedFilesSavingsRate : 45 %
InPolicyFilesCount : 21
InPolicyFilesSize : 834.38 GB
So, after 2 dedupes and compression I have a total of 4.64tb of backed up data with 583.38gb used space!
Some things to consider:
- Dedupe is supported on NTFS only, not ReFS which confused me because I just assumed it was a function of the new file system.
- When using RAID 1 mirrors it’s beneficial to use Windows Storage Spaces instead of a hardware RAID because the dedupe engine can pull corrupted “chunks” from the mirrored disk.
- The fsdmhost.exe process was using almost 8 gigs of ram while running 2 simulatenous jobs. It queued the third. Cpu was 10-20% on an X3440 CPU
Useful Powershell commands
Manually start the dedupe job at normal priority. Replace X with your drive letter.
Start-DedupJob –Full –Path x: -Type Optimization
Check the status of the job with this command.
Finally, check the stats with this command.
To get even more details add | fl
Get-DedupStatus | fl