+- Antwise community forums (https://forums.antwise.com)
+-- Forum: Vimalin (https://forums.antwise.com/forumdisplay.php?fid=4)
+--- Forum: Vimalin for Windows (https://forums.antwise.com/forumdisplay.php?fid=9)
+--- Thread: CertUtil Slow? Backing up large VM slow. (/showthread.php?tid=23)
CertUtil Slow? Backing up large VM slow. - bleomycin - 2019-07-24
Just found Vimalin and it's exactly what I've been looking for! In my initial testing backing up a large VM (~760GB) running in vmware workstation 15 Pro a large portion of time is spent running CertUtil.exe to generate the MD5. It's processing the file at ~210MB/s which means CertUtil spends an hour each time a backup is run. The ssd the VM is stored on is just a normal sata SSD but I suspect the bottleneck here isn't the SSD but CertUtil itself? The host machine has a 4Ghz (all core) 8-core CPU and certutil seems to use only about 2-3% when it's running.
Anyways, is there any way to make this process go faster? I'm planning to swap out for a much faster nvme drive soon partially to increase backup speeds as the NAS I backup to is capable of being written to at sustained 10Gbit/s and sata isn't cutting it.
Thanks for any help!
CertUtil Slow? Backing up large VM slow. - wila - 2019-07-24
Hi,
Glad to hear that Vimalin is just what you are looking for.
At the moment I'm not sure what you can do to speed up certutil. The utility not using all your available cores is both a blessing and a curse.
It's a blessing because running a backup won't impact your workflow much when using the VM.
It's a curse because you have more CPU capacity available, but the utility isn't using it.
Replacing certutil with another utility might be a workaround, but the problem is which utility to use.
As certutil comes with the OS it has my preference, it also is a utility I trust to work well with files over 2GB in size.
Another option would be to disable the md5 hashing, but that reduces the reliability as Vimalin won't be able to guarantee that the backup is correct.
For the moment you cannot disable this step.
--
Wil
CertUtil Slow? Backing up large VM slow. - bleomycin - 2019-07-24
Thanks for the reply I appreciate it. I understand I'm probably a super niche use case but it would be great to have the option as you said to either use a significantly faster alternative to certutil (I don't know what that is) or to have a hidden advanced option to disable the MD5 checking. Disabling it is a risk i'd be willing to take, I utilize zfs snapshots on the backup machine so if one or 2 backups were to somehow have an issue the snapshots would mostly insulate me from that.
CertUtil Slow? Backing up large VM slow. - wila - 2019-07-24
Note that Vimalin will still allow you to restore your backup even if the md5sum is wrong (or missing), so it's not that it would block you there.
But it would tell you that there's a problem when restoring the VM.
Unfortunately it is not configurable at the moment. I have logged it at the moment as a feature request.
--
Wil
CertUtil Slow? Backing up large VM slow. - wila - 2019-08-08
Hello,
Starting with version 2.0.404 you can now disable the MD5 sum feature.
There is no user interface for this, you will have to manually edit a configuration file.
This is by intent as I do not want to complicate the user interface or make it too easy for users to shoot themselves in the foot.
In order to disable the feature that Vimalin creates an md5sum hash for each file it backups, follow these steps:
Shut down the Vimalin application if it runs.
Go to the following location on your disk:
C:\Users\<username>\AppData\Local\Vimalin\Config
Where <username> is your Windows username such as bleomycin
Then open the file Vimalin.ini with a plain text editor such as notepad and add the following line in the [base] section.
Code:
vimalin.DisableFileHash=trueIt might take up to 15 minutes before that change is read by the worker process as changes in this particular file are not immediately read.
But after that it should no longer try to generate MD5 hashes.
--
Wil