Rsync portable

[Hydaral]

I was going to post this as a reply to the Deltacopy topic (http://www.portablefreeware.com/forums/ ... 96&start=0), but thought that it warranted its own post.

I've been using Rsync to backup ~2 GB every night to an overseas server for the last 4 years, I only send about 30 MB of actual data though. I was using the somewhat hacked Win32 version that, as modussn has mentioned, used Cygwin. It took some time to initially set up, but it worked fine.

I just found out about a month ago that it has been improved quite a bit, the Cygwin component is well integrated now, so there is very little configuration that you have to do. In my previous backup script I had to do a lot of setting of environment variables so that Cygwin knew where to find everything, this has been reduced to just one: %HOME% (plus %PATH% too, if you want).

I downloaded the cwrsync server version, you have to manually extract the NSIS installers, it's about 10MB installed, 15 MB if you add the CopSSH component. If anyone wants more instructions, just ask.

http://www.itefix.no/i2/cwrsync

It's not 100% portable, Cygwin seems to use some registry entries and it's CLI based.

[Hydaral]

I just checked out the registry entries, Cygwin writes them, but they are not actually re-read on subsequent runs, it gets the settings from the configuration files and overwrites the old registry keys. Since Rsync will most likely be run from a batch file anyway, you could just add a "reg delete" command to the end.

HKEY_CURRENT_USER\Software\Cygwin\Program Options

HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\Installations
HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\Program Options

[webfork]

Great stuff. The package program is GPL'd and seems to support other command-line functions for greater customization.

If anyone wants more instructions, just ask.

I haven't tested the other rsync program much, but I have no doubt a need for this will come up sooner or later so, yes, please post any information you've got.

[Hydaral]

1. Download cwrsync server, I recommend you get copssh too, this gives you a whole lot more apps and you will need it if you want to sync with a Linux machine.

2. Extract the zips.

3. Extract the installers with Uniextract

4. You also need to extract the two additional NSIS installers in the $PLUGINSDIR directory for both apps, use Uniextract.

5. Now you should have the following directories:

For Rsync:

Code: Select all

ICW_Base
ICW_RsyncServer

For CopSSH

Code: Select all

ICW_Base
ICW_OpenSSHServer

6. The "Base" directories should be exactly the same, so throw one of those away.

7. Open the $INSTDIR directories in all three remaining directories and copy everything in them to a single place, there are no duplicates files in subdirectories so you can answer yes to the Windows folder replacement dialog.

8. You also want to get the HTML documentation from the ICW_OpenSSHServer directory, on my system it was in a directory called "ýŸ€", in the same place as $INSTDIR. Copy the contents to the doc directory, you can discard the skel directory.

9. Now you should have:

Code: Select all

Bin
doc
etc
fstab.txt
rsyncd.conf

10. The one thing that does not get configured by manually extracting is the etc\fstab file. Create a text file named fstab in the etc directory and paste the following into it, white space can be spaces, tabs, whatever, just as long as there is a gap (read the fstab.txt file too):

Code: Select all

#
# see fstab.txt in the installation directory for more information

none					/cygdrive					cygdrive	binary,posix=0,user,noacl	0	0

This creates a Cygwin compatible link for the root directory. Cygwin requires this!

Phew, that's the install done! I said the configuration is easy, not the install.

The commands in the Bin directory should all run now, you should be able to access all your drives using the standard Windows notation, use the ls command (short for list, Linux equivalent of Windows dir) to try it out:

Code: Select all

ls.exe c:\

Cygwin doesn't normally like this notation, but it will detect it and internally convert it to Cygwin-ese, Cygwin paths look like this, for C:\temp:

Code: Select all

/cygdrive/c/temp

That's what that line in fstab is for.

You can use UNC paths too:

Code: Select all

ls.exe \\[computername]\[share]\

This will run, but Cygwin will show you a warning:

Code: Select all

  MS-DOS style path detected: \\[computername]\[share]
  Preferred POSIX equivalent is: /srv/[computername]/[share]
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames

To use Cygwin paths properly you need to add a line to the fstab file:

Code: Select all

//[computername]/[share]	/srv/[computername]/[share]		ntfs		binary,posix=0,user,noacl	0	0

You can replace "srv" with whatever you like, just remember what you changed it to when you refer to it. Now you can use this:

Code: Select all

ls.exe /srv/[computername]/[share]

And you don't get a warning anymore.

To add the Bin directory to your path, you can run this at the command prompt, add it to your permanent enviroment variable or use it in a batch file (this is what I do):

Code: Select all

set path=[Bin directory];%path%

It you want to connect to a remote Linux machine you need to use SSH. SSH needs to store some "stuff" about the connection so it needs to know your home directory, make a directory named "home" in the same directory as Bin and etc, and set the home variable:

Code: Select all

set home=[path to home directory]

It's best to use the Cygwin path notation for that.

That's it, make sure to read the docs if you haven't used Rsync and SSH before. I'll do another post soon about making public and private keyfiles so you don't have to keep typing the password whenever you connect to a remote machine.

[webfork]

Excellent, thanks Hydaral.

[webfork]

Evidently the program went for-pay:

https://www.itefix.no/i2/node/12990

[Hydaral]

Disappointing. I don't think they actually modified any of the binaries though, otherwise they would have to supply the source. I think the only thing they did was create a user-friendly installer.

Theoretically, you should be able to create a portable Rsync manually using selected Rsync, SSH and Cygwin files. I might take a look at my current cwRsync file and folder layout and see if I can duplicate it using the available GPL apps.

[webfork]

Theoretically, you should be able to create a portable Rsync manually using selected Rsync, SSH and Cygwin files. I might take a look at my current cwRsync file and folder layout and see if I can duplicate it using the available GPL apps.

That would be a cool trick. If I could get a basic batch file setup with some common settings, I could probably adapt it to my own use.

I'll also check out some other tools in the next few days.

[Hydaral]

Getting an Rsync environment running seems to be extremely simple actually!

Go to your nearest Cygwin mirror: http://cygwin.com/mirrors.html

Get the following packages:
/release/cygwin/
/release/libiconv/libiconv2/
/release/rsync/

You just need the latest bz2 file out of each directory. Extract them all into the same directory.

You should now have two directories: etc and usr. /usr/bin is the only one you actually need, so move that somewhere and delete the rest.

Rsync (inside /bin) should run now, I've only tested it locally, but it appears to sync fine. You probably need to download OpenSSH too if you want to sync remotely to a Linux machine, just copy the files out of /usr/bin as before.

Most of my previous instructions regarding fstab and environment variables are still applicable.

If you want all the base Linux commands too, ls, rm, mv, etc, get the /release/coreutils/ package.

Edit: Looks like you'll need to get some extra files if you want to use coreutils, It pops up an error stating which file it needs. Searching the web for "[required file] package cygwin" should tell you which package you need to get.

[webfork]

Thanks for the breakdown.

I was really hoping to find a UI to help make some of the finer points of rsync a little smoother, but its looking like I'm just going to have to learn it the command-line way.

[Hydaral]

Rsync is one of those things that you usually only setup once for each case, rather than ad-hoc, so a GUI is not really required. If you need any help with commands, just ask.

GRSync is a GUI, there's a Windows port:

http://sourceforge.net/projects/grsync-win/

Edit: Just found this one too: http://www.yinter.net/

Not a GUI for RSync, but related: http://www.sepiola.org/en/home/

[webfork]

I was going to post why RSYNC is important here, but I decided to give it its own to avoid a thread hijack.

[SYSTEM]

Found another rsync frontend.

https://code.google.com/p/licielrsync/

Requires .NET Framework 2.0 & portability not tested.

(By the way, quite bad project homepage. Looks like the author put everything he/she could think of in there.)