It is currently Sun May 19, 2013 2:54 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: Rsync portable
PostPosted: Mon Nov 08, 2010 5:47 pm 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
I was going to post this as a reply to the Deltacopy topic (viewtopic.php?f=4&t=5196&start=0), but thought that it warranted its own post.

I've been using Rsync to backup ~2 GB every night to an overseas server for the last 4 years, I only send about 30 MB of actual data though. I was using the somewhat hacked Win32 version that, as modussn has mentioned, used Cygwin. It took some time to initially set up, but it worked fine.

I just found out about a month ago that it has been improved quite a bit, the Cygwin component is well integrated now, so there is very little configuration that you have to do. In my previous backup script I had to do a lot of setting of environment variables so that Cygwin knew where to find everything, this has been reduced to just one: %HOME% (plus %PATH% too, if you want).

I downloaded the cwrsync server version, you have to manually extract the NSIS installers, it's about 10MB installed, 15 MB if you add the CopSSH component. If anyone wants more instructions, just ask.

http://www.itefix.no/i2/cwrsync

It's not 100% portable, Cygwin seems to use some registry entries and it's CLI based.


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Mon Nov 08, 2010 6:03 pm 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
I just checked out the registry entries, Cygwin writes them, but they are not actually re-read on subsequent runs, it gets the settings from the configuration files and overwrites the old registry keys. Since Rsync will most likely be run from a batch file anyway, you could just add a "reg delete" command to the end.

HKEY_CURRENT_USER\Software\Cygwin\Program Options

HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\Installations
HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\Program Options


Top
 Profile  
 
 Post subject: cwrsync - Rsync for Windows
PostPosted: Wed Nov 10, 2010 11:00 am 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
Great stuff. The package program is GPL'd and seems to support other command-line functions for greater customization.

Hydaral wrote:
If anyone wants more instructions, just ask.

I haven't tested the other rsync program much, but I have no doubt a need for this will come up sooner or later so, yes, please post any information you've got.

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Wed Nov 10, 2010 10:43 pm 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
1. Download cwrsync server, I recommend you get copssh too, this gives you a whole lot more apps and you will need it if you want to sync with a Linux machine.

2. Extract the zips.

3. Extract the installers with Uniextract

4. You also need to extract the two additional NSIS installers in the $PLUGINSDIR directory for both apps, use Uniextract.

5. Now you should have the following directories:

For Rsync:
Code:
ICW_Base
ICW_RsyncServer
For CopSSH
Code:
ICW_Base
ICW_OpenSSHServer
6. The "Base" directories should be exactly the same, so throw one of those away.

7. Open the $INSTDIR directories in all three remaining directories and copy everything in them to a single place, there are no duplicates files in subdirectories so you can answer yes to the Windows folder replacement dialog.

8. You also want to get the HTML documentation from the ICW_OpenSSHServer directory, on my system it was in a directory called "ýŸ€", in the same place as $INSTDIR. Copy the contents to the doc directory, you can discard the skel directory.

9. Now you should have:
Code:
Bin
doc
etc
fstab.txt
rsyncd.conf
10. The one thing that does not get configured by manually extracting is the etc\fstab file. Create a text file named fstab in the etc directory and paste the following into it, white space can be spaces, tabs, whatever, just as long as there is a gap (read the fstab.txt file too):
Code:
#
# see fstab.txt in the installation directory for more information

none               /cygdrive               cygdrive   binary,posix=0,user,noacl   0   0
This creates a Cygwin compatible link for the root directory. Cygwin requires this!

Phew, that's the install done! I said the configuration is easy, not the install. :)

The commands in the Bin directory should all run now, you should be able to access all your drives using the standard Windows notation, use the ls command (short for list, Linux equivalent of Windows dir) to try it out:
Code:
ls.exe c:\
Cygwin doesn't normally like this notation, but it will detect it and internally convert it to Cygwin-ese, Cygwin paths look like this, for C:\temp:
Code:
/cygdrive/c/temp
That's what that line in fstab is for.

You can use UNC paths too:
Code:
ls.exe \\[computername]\[share]\
This will run, but Cygwin will show you a warning:
Code:
  MS-DOS style path detected: \\[computername]\[share]
  Preferred POSIX equivalent is: /srv/[computername]/[share]
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
To use Cygwin paths properly you need to add a line to the fstab file:
Code:
//[computername]/[share]   /srv/[computername]/[share]      ntfs      binary,posix=0,user,noacl   0   0
You can replace "srv" with whatever you like, just remember what you changed it to when you refer to it. Now you can use this:
Code:
ls.exe /srv/[computername]/[share]
And you don't get a warning anymore.

To add the Bin directory to your path, you can run this at the command prompt, add it to your permanent enviroment variable or use it in a batch file (this is what I do):
Code:
set path=[Bin directory];%path%
It you want to connect to a remote Linux machine you need to use SSH. SSH needs to store some "stuff" about the connection so it needs to know your home directory, make a directory named "home" in the same directory as Bin and etc, and set the home variable:
Code:
set home=[path to home directory]
It's best to use the Cygwin path notation for that.

That's it, make sure to read the docs if you haven't used Rsync and SSH before. I'll do another post soon about making public and private keyfiles so you don't have to keep typing the password whenever you connect to a remote machine.


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Thu Nov 11, 2010 2:33 pm 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
Excellent, thanks Hydaral.

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Thu Jan 26, 2012 5:03 pm 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
Evidently the program went for-pay:

https://www.itefix.no/i2/node/12990

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Thu Jan 26, 2012 6:22 pm 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
Disappointing. I don't think they actually modified any of the binaries though, otherwise they would have to supply the source. I think the only thing they did was create a user-friendly installer.

Theoretically, you should be able to create a portable Rsync manually using selected Rsync, SSH and Cygwin files. I might take a look at my current cwRsync file and folder layout and see if I can duplicate it using the available GPL apps.


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Thu Jan 26, 2012 6:31 pm 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
Hydaral wrote:
Theoretically, you should be able to create a portable Rsync manually using selected Rsync, SSH and Cygwin files. I might take a look at my current cwRsync file and folder layout and see if I can duplicate it using the available GPL apps.

That would be a cool trick. If I could get a basic batch file setup with some common settings, I could probably adapt it to my own use.

I'll also check out some other tools in the next few days.

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Fri Jan 27, 2012 12:53 am 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
Getting an Rsync environment running seems to be extremely simple actually!

Go to your nearest Cygwin mirror: http://cygwin.com/mirrors.html

Get the following packages:
/release/cygwin/
/release/libiconv/libiconv2/
/release/rsync/

You just need the latest bz2 file out of each directory. Extract them all into the same directory.

You should now have two directories: etc and usr. /usr/bin is the only one you actually need, so move that somewhere and delete the rest.

Rsync (inside /bin) should run now, I've only tested it locally, but it appears to sync fine. You probably need to download OpenSSH too if you want to sync remotely to a Linux machine, just copy the files out of /usr/bin as before.

Most of my previous instructions regarding fstab and environment variables are still applicable.

If you want all the base Linux commands too, ls, rm, mv, etc, get the /release/coreutils/ package.

Edit: Looks like you'll need to get some extra files if you want to use coreutils, It pops up an error stating which file it needs. Searching the web for "[required file] package cygwin" should tell you which package you need to get.


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Wed Feb 01, 2012 11:36 am 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
Thanks for the breakdown.

I was really hoping to find a UI to help make some of the finer points of rsync a little smoother, but its looking like I'm just going to have to learn it the command-line way.

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Wed Feb 01, 2012 6:30 pm 
Offline

Joined: Tue Mar 09, 2010 7:36 pm
Posts: 197
Rsync is one of those things that you usually only setup once for each case, rather than ad-hoc, so a GUI is not really required. If you need any help with commands, just ask.

GRSync is a GUI, there's a Windows port:

http://sourceforge.net/projects/grsync-win/

Edit: Just found this one too: http://www.yinter.net/

Not a GUI for RSync, but related: http://www.sepiola.org/en/home/


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Thu Feb 23, 2012 6:06 pm 
Offline
User avatar

Joined: Wed Apr 11, 2007 8:06 pm
Posts: 3473
Location: US, Texas
I was going to post why RSYNC is important here, but I decided to give it its own to avoid a thread hijack.

_________________
Supporting the Electronic Frontier Foundation | DuckDuckGo user | My GPG key | Projects donated to: VLC, CubicExplorer, Ditto, Greenshot, TrueCrypt


Top
 Profile  
 
 Post subject: Re: Rsync portable
PostPosted: Mon Feb 18, 2013 9:52 am 
Offline
User avatar

Joined: Sat Jul 31, 2010 1:19 am
Posts: 805
Location: Helsinki, Finland
Found another rsync frontend.

https://code.google.com/p/licielrsync/

Requires .NET Framework 2.0 & portability not tested.

(By the way, quite bad project homepage. Looks like the author put everything he/she could think of in there.)

_________________
My YouTube channel | Release date of my fourth playlist: April 11, 2013


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Protected by Anti-Spam ACP Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group