Handling ShadowProtect and file systems with high-delta

ShadowProtect is an awesome tool, but one of it's weaknesses is that it does not allow you to specify exclusions during the backup process. So if you know that a particular directory is going to generate a lot of files that you don't need during the day, then right now you still have to back this up.

Common reaons for not wanting / needing to backup include:

  • It's temporary content - e.g. files and directories generated during installations
  • Log files that you are not relying upon for long term analysis / system rebuild (Exchange / SQL logs probably don't fit this category)
  • The content is generated, and can be regenerated easily in the event of a disaster recovery.

Some common situations where this might apply include:

  • Antivirus programs - commonly store several revisions of their definitions and the installation programs. Some packages rebuild these at each update, which can result in many Gb per day of delta that you don't actually need to backup because they regenerate. Note - you need to backup the program but not the definitions.
  • Proxy caches - up to you. You may or may not need to backup the content of your proxies.
  • Temporary folders - self explanatory really...
  • Logs (but be careful about what kind of logs you're not backing up...)
  • WSUS Updates
  • Any content that you do not want to backup

In Unix/Linux you generally store frequently changing content in /var . In the Windows world we don't really have an equivalent, but with block based backup software you need it.

How do you use it?

  • Allocate a specific partition as VAR in Windows
  • Configure programs that meet the above criteria. This might mean making use of mount points or symbolic links (refer to this article I wrote), or simply installing an entire program in this location.
  • If you're configuring a program you will probably have to do a fair amount of manual work tweaking folders, locations, possibly the registry etc. It's not for the faint hearted. If you're really lucky the program will have a sensible config file or options menu where you can specify all of these things.
  • Put your page file in here.
  • Configure ShadowProtect **to not backup this partition**. And voila. You have sensible backups, where you're only capturing the delta of data, as opposed to the delta of processes that don't need to be captured.

None of this is straightforward - there is unfortunately no "one size fits all" for this type of thing, so you will need to backup, test, test, test.

My last comment on this matter is for Storage Craft. Their current position on these difficulties is "this is a block based backup, so therefore we can't see and therefore exclude directories or files". I beg to differ. If you know I'm wrong then please tell me, but this is my laypersons understanding...

  • ShadowProtect tracks changed blocks - this is how it achieves it's remarkably fast backup times. It already knows which blocks need backing up, rather than scanning everything.
  • When you mount a ShadowProtect backup or restore from it, you can view the file system, directories, files etc.
  • You can retrieve a particular file(s), directories etc.
  • So obviously, ShadowProtect is aware of how the blocks that it backed up relate to actual files and directories.
  • Consequently, it must therefore be technically possible for it to determine a list of "banned blocks". I imagine it would work this way:
    • User defines folders / files they wish to exclude;
    • ShadowProtect determines which blocks these relate to;
    • The existing file system hook that they have in place tracks changes to blocks... so it also knows whether a change has occured in any of the "banned blocks";
    • If the file system moves a banned block - for example a defrag, the file system hook is aware of this, so can track what block(s) are now relevant;
    • When it's conducting a backup it excludes the banned blocks.

Now I'm sure that there will be things about this that are trickier than my description makes it seem. If so tell me about it and I'll put it up. But don't just ignore your customers and claim that it can't be done.

Troubleshooting Shadow Protect and Symantec Endpoi...
Creating a route automatically when a VPN tunnel i...

Related Posts


No comments made yet. Be the first to submit a comment