Page 1 of 1
ln - command line hard links
Posted: Fri Jul 21, 2023 11:31 pm
by Andrew Lee
Never thought I'd get excited over a CLI program, but here it is:
https://schinagl.priv.at/nt/ln/ln.html
It does exactly what I have been looking for, and possibly more.
So what I wanted to do is this:
- Suppose I have a source folder to backup called c:\source
- Initially I copy everything to a backup location called d:\backup
- Then after things change in c:\source, I want to copy everything to d:\backup2, with these conditions:
- if a file has not been modified, it creates a hard link back to the same file in d:\backup
- if a file has been modified or is new, it copies that file instead
- After some time, I can create a new backup called d:\backup3, but basing off d:\backup2, and so on
I had no idea this method of backup is called
Delorean Copy, so the way to do this via command line is:
Code: Select all
ln --copy c:\source d:\backup
ln --delorean c:\source d:\backup d:\backup2
ln --delorean c:\source d:\backup2 d:\backup3
etc.
The nice thing about this way of copying is that each backup folder always contain a full copy of
c:\source, but the amount of disk space used by each subsequent backup is only increased by the changed files.
I have been doing this by hacking together something using rsync and Linux Subsystem for Windows, but this is far simpler and cleaner using a native Win64 CLI.
Re: ln - command line hard links
Posted: Sat Jul 22, 2023 4:41 am
by __philippe
@Andrew
Typo correction for above link
https://schinagl.priv.at/nt/ln/ln.html
(...for the lack of an
"l" an "ERROR 404" came to the fore...
)
Re: ln - command line hard links
Posted: Sat Jul 22, 2023 10:23 pm
by Andrew Lee
Corrected in original post. Thanks!
Re: ln - command line hard links
Posted: Mon Jul 24, 2023 3:34 am
by Midas
Cool example for DeLorean backups -- and having a CLI utility is great for scripting.
FTR,
LinkShellExtension by the same author isn't really portable but has nice GUI for the same purpose...
Re: ln - command line hard links
Posted: Mon Jul 24, 2023 11:32 pm
by Specular
Andrew Lee wrote: ↑Fri Jul 21, 2023 11:31 pm
I had no idea this method of backup is called Delorean Copy, so the way to do this via command line is:
Interesting term
I normally see this method referred to as snapshotting. For example rsnapshot for Linux (a popular utility that leverages rsync) uses this same method, where hardlinks are used for unchanged files in the next snapshot. It's very useful for a NAS backup system (particularly if set up to handle NTFS-compatible attributes and where the snapshots have read-only permissions).
The benefit, as you've seen, is all files are visible for each snapshot (rather than merely just the changed files), so whatever the latest version is can be restored as-is.
Downside is any corruption of a file the hardlinks are based on will propagate to all hardlinks. Though drives have their own internal ECC and it's hard to guestimate the likelihood such corruption would occur (seems more likely a drive would die).
Re: ln - command line hard links
Posted: Tue Jul 25, 2023 3:46 pm
by Andrew Lee
Specular wrote: ↑Mon Jul 24, 2023 11:32 pm
Downside is any corruption of a file the hardlinks are based on will propagate to all hardlinks. Though drives have their own internal ECC and it's hard to guestimate the likelihood such corruption would occur (seems more likely a drive would die).
I think what you are referring to is bit-rot, and the way I deal with it is to:
1) Keeps a CRC of all backed up files using
RapidCRC, and check from time-to-time
2) Only keep a max of say 7 snapshots before starting all over again with a fresh master.
Yeah, I know it's tedious, but we've all been there before when the hard drive dies on ya...
Re: ln - command line hard links
Posted: Thu Jul 27, 2023 4:14 am
by Andrew Lee
Midas wrote: ↑Mon Jul 24, 2023 3:34 am
Cool example for DeLorean backups -- and having a CLI utility is great for scripting.
FTR,
LinkShellExtension by the same author isn't really portable but has nice GUI for the same purpose...
I had no idea! I thought it was just a cool shell extension for visualizing symlinks/hardlinks. I didn't know it could perform Delorean backups as well.
Will still stick to
ln for portability + scripting, but it is great to know a GUI alternative is available!