Hurray, another fine way to control data on Windows boxes from Linux machines. We are using cygwin, openssl, openssh, vi and bash.

Remote Backups using Rsync

rsync can be used in a number of ways. This documents how to use rsync to backup a windows box to a linux box.

For information about rsync see http://samba.anu.edu.au/rsync/

Rsync was created by the same man who began samba, so the configuration files are very similar.

Background: from http://lists.svlug.org/pipermail/svlug/2004-February/046680.html, the following information is important.

    If the machine that you installed rsync on is firewalled (I hope so!), just block incoming access to TCP port 873. That's the port that the rsync daemon listens on. Also, you should be able to configure the rsync daemon to only listen on the loopback interface. Of course, port 22 for ssh will need to be open.

    Then, from your backup host, you'll need to log into ssh like this:

    ssh -l -L 873:localhost:873

    That will set up a tunnel from port 873 on your local machine to port 873 on the loopback interface of the remote machine. Then you can use rsync on your local machine to connect to its local port 873. ssh will take of the rest and forward the connection to the remote machine for you.

    The command line might look like this:

    rsync –delete -ravz user at localhost::BackupDir BackupDir

Note that this applies IFF rsync is used in daemon mode which will be documented as well. However, it is important enough to be mentioned twice.

Also, important information about tunneling rsync through ssh can be found at http://www.jdmz.net/ssh/ .

The general syntax is:

rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

This documents a PUSH routine, i.e. when the client (the machine to be backed up) pushes its data to the server (the machine storing the backup). This appears to be the easiest to implement on windows because you don't need an ssh daemon running on the windows box.

You will need to install cygwin on the windows box and include openssl, openssh, bash, vi, and rsync as minimal packages.

We assume there is a user called backupuser that exists on the windows box. This user must have minimally backup operators members for permission to read the file system. Its probably a good idea NOT to create this as an administrative user since it seems to work fine as a member of backup operators. Why have extra priveledge?

We also assume you have created ssh keys and copied the ssh public key from the client (the windows box) to the server and placed it in the proper authorized_keys file. This is an important piece becuase you want the backup to run unattended. The jdmz site mentioned above describes how to create these.

The bash script below may be used as a template on the windows server.

This script could be called snapshot.sh

PATH=/bin:/usr/bin
HOME=/home/backupuserdir
desthost="rsyncserver.asu.edu"
exclude="– exclude *.tmp –exclude 'dnscache/*'"
srcdir="/cygdrive/c"
destdir="/home/backupuser/clientmachinename"
rsyncopts="-e -ssh -avp "
user="remotuser"
rsync $rsynopts $exclude $srcdir $user@$desthost:$destdir > $HOME/synclog.txt

Notes to the script above.

   1. its a good idea to set the path and home directory since this script will be called from the windows AT scheduler and may not get it otherwise
   2. exclusions are good to avoid things like temp files
   3. the rsync options imply that rsync will tunnel through ssh and use the archive switches to perform a full backup sync.

This bash script must be sourced and called properly to recieve a standard cygwin bash environment. Since the calling environment is really windows the following windows batch file sets the proper variables.

The following file could be named wrapper.bat.

echo on
c:
setlocal
chdir c:\cygwin\home\backupuser
set HOME=/home/backupuser
c:\cygwin\bin\bash –login -c %HOME%/snapshot.sh
endlocal

The windows AT command may be used to schedule this batch file to run at regular intervals.

CYGWIN SSH

http://pigtail.net/LRP/printsrv/cygwin-sshd.html is the source for this

Advanced Disclaimer

cygwin-openssh

How to install a ssh server (called sshd, from OpenSSH) on a Windows 2000 or XP How to install a sftp server on a Windows 2000 or XP

The ssh server is an emulation of the UNIX environment and OpenSSH for Windows, by Redhat, called cygwin

(1a) Login as Administrator (Windows XP – login as a user with Administrator privilege).

(1b) Make sure the current admin/user has a Windows password set. If not, use g Control Panel…User Accounts to create a password.

(2a) Create a folder c:\cygwin

(2b) g Download cygwin's setup.exe from http://www.cygwin.com/ and save setup.exe in c:\cygwin Click Start…Run…and type c:\cygwin\setup.exe

If you are asked about "Just Me" or "All Users", choose "All Users"

When it asks for "Local Package Directory", type c:\cygwin When a selection screen comes up, (you can resize the windows to see better) click the little View button for "Full" view g, find the package "openssh", click on the word "skip" so that an x appears in Column B, see this illustration.

Click next to start installing cygwin and ssh. Size of the basic cygwin system is about 40 Meg, this may take a while.

Take a coffee break. g

(3) Right click My Computer, Properties, Advanced, Environment Variables See this illustration (red dots) Click the "New" new button to add a new entry to system variables: variable name is CYGWIN variable value is ntsec

(4) Right click My Computer, Properties, Advanced, Environment Variables See this illustration (green dots) Select the Path variable and click the "Edit" edit button: append ;c:\cygwin\bin to the end of the existing variable string.

(5) Open a cygwin window (by double clicking theg icon), a black screen pops open, type ssh-host-config (on slower computers, it may take several minutes to generate the dsa keys) When the script asks you about "privilege separation", answer yes When the script asks about "create local user sshd", answer yes When the script asks you about "install sshd as a service", answer yes When the script asks you for "CYGWIN=" your answer is ntsec

See Note 5 below if you need to run ssh-host-config again.

(6) While you are still in the (black) cygwin screen, start the sshd service net start sshd or cygrunsrv –start sshd

Note: if you need to stop the sshd service, pop open a g cygwin window net stop sshd or cygrunsrv –stop sshd

(7) Make sure every Windows user has a password set, if not, go to g Control Panel….User Accounts and create a password.

(8) important Pop a cygwin gwindow, harmonize Windows user information with cygwin, otherwise they cannot login mkpasswd –local > /etc/passwd mkgroup –local > /etc/group Test to see if sshd is working, pop a cygwin gwindow (note: the command below is case sensitive) whoami ssh localhost or ssh "$USERNAME@127.0.0.1"

If you get a prompt without error messages, type ls -lh /cygdrive/c if you see a directory listing, success! g g g (type exit to end the cygwin ssh session)

If you have a Windows username that contain space, expand the space into \ [space], e.g. if the Windows login name is mickey mouse ssh mickey\ mouse@127.0.0.1

Thanks to Jared Kilgour for above $USERNAME variable substitution. Thanks to Justin Kerk for the tip on quotes around $USERNAME to allow for spaces in username. g Windows XP SP2 – open the Windows Firewall to allow TCP port 22 through

Click Start…Control Panel….Security Centre….Manage Security Settings for Windows Firewall….Exceptions tab….Add Port… "Name of port" is ssh "Port number" is 22 (Thanks to Stefano of Sardegna, Italy for his Windows XP Firewall reminder)

If you previously used Windows XP SP1 and installed sshd service, then upgraded to Windows XP SP2, note that the SP2 disables the sshd service and deletes the CYGWIN environment variable Re-enter the environment variables and path. Click Start…Control Panel….Security Centre….Manage Security Settings for Windows Firewall….Exceptions tab….Add Port… "Name of port" is ssh "Port number" is 22 (Thanks to Chris Davitt of New Zealand for this SP1 SP2 glitch) Multiple Windows users g

Create other Windows users using the g Control Panel…User Accounts. After you created (or removed) Windows users pop a g cygwin windows to harmonize Windows user information with cygwin, otherwise they cannot login mkpasswd –local > /etc/passwd mkgroup –local > /etc/group

g Don't get too carry away with multiple users, if a user successfully ssh into the box, he or she can cd to just about any directory.

Users from the internal network (geeks call this a LAN) can ssh usersname@ip_address (e.g. ssh john@192.168.0.100)

On Unix/Linux systems, user names generally do not contain spaces. On Windows system, user names can have spaces. If you have a Windows username that contain spaces, expand each space into \ [space], e.g. if the Windows username is mickey mouse ssh mickey\ mouse@192.168.0.100

g

If you have a NAT firewall, port forward (D-link calls it Virtual Server) TCP port 22 to the IP address of the Windows box with the sshd server. See above diagram.

Users from the outside (geeks call this a WAN) can ssh username@external_ip_address (e.g. ssh john@64.64.64.64 ) ssh mickey\ mouse@external_ip_address (e.g. ssh mickey\ mouse@64.64.64.64 ) (assuming you have an IP address that is accessible from the outside world, some ISP do not give an outside accessible IP address) (assuming your firewall opens TCP port 22 and forwards it to the box running ssh server)

g

g sshd includes sftp and sftp-server for encrypted file transfers. These two programs function like the familiar ftp-client and ftp-server.

g For example, from a remote laptop, you can do file transfers to your home base (see above diagram). sftp username@ip_address (e.g. sftp john@32.97.166.74 ) sftp username@hostname (e.g. sftp john@supercompuer.ibm.com ) openssh [which uses openssl] has strong encryption capability. The encryption used by openssh can be either AES-128, AES-192, AES-256, 3DES, Blowfish, cast-128

After you establish a ssh or sftp connection into the Windows box, changing directory is a bit more difficult, for example, to change to "my documents" cd "/cygdrive/c/documents and settings/$USERNAME/my documents"

Similarly, to change directory to d: drive cd /cygdrive/d

Where else can you find a sftp client ? (1) Putty has psftp.exe, it runs from a command console. (2) Commercial software vendors such as VanDyke Software. (3) Filezilla, a free, GNU (GPL) licensed, ftp and sftp client.

If you prefer to use a graphical client to do sftp file transfers, purchase a high quality commercial software called SecureFX from VanDyke Software in Albuquerque, New Mexico, USA or use Filezilla, a free, GNU (GPL) licensed, ftp and sftp client, or use WinSCP, a free, GNU (GPL) licensed sftp and scp client. Also, ftp.ssh.com in their /pub/ssh directory, there is a Windows version of ssh and sftp client for non-commercial use, thanks to Stephan of Rutgers State University of New Jersey g for the link.

Other very, very useful things you can do with ssh is to tunnel (wrap) pure tcp applications under the ssh protocol, giving them a strong cryptographic protection while traveling over the insecure public network. The encryption used by openssh can be either AES-128, AES-192, AES-256, 3DES, Blowfish, cast-128 g

Below are some popular plain-text TCP protocols that can benefit from the protection of a ssh tunnel: POP3 (tcp port 110) IMAP (tcp port 143) SMTP (tcp port 25) TELNET (tcp port 21) VNC (tcp port 5900) Print server traffic (tcp port 9100)

Note: The world is moving away from plain text protocols by hardening them with TLS or SSL: newer versions of POP3 has TLS support at port 110, and SSL support at port 995 newer versions of IMAP has TLS support at port 143, and SSL support at port 993 newer versions of SMTP has TLS support at port 25 a version of "smtps" uses port 465 with SLL support, now becomes legacy newer versions of telnet has SSL support at port 992

See this page on how to tunnel VNC traffic under ssh. See this page on how to tunnel TCP applications under ssh.

Sometimes, there are applications such as mid-night back-up of files to a remote Linux server using "rsync encrypted with ssh", you want to be able to ssh from one machine to another machine without a person sitting at the console to type a password. See this page on how to ssh from one machine into another machine without typing a password, i.e, use public-key authentication.

public-key-authentication

How to install a ssh client (called ssh) Click here for a tutorial on how to setup a ssh client on Windows 2000 or Windows XP g

How to install a smtp server [exim] on a Windows machine Click here for a tutorial on how to setup exim, a mail transfer agent on Windows 2000 or Windows XP g as a learning exercise.

Note 5: if you run ssh-host-config when sshd is installed, ssh-host-config will not ask for CYGWIN value. In that case, stop and remove the sshd service, then run the ssh-host-config script again.

cygrunsrv –stop sshd cygrunsrv –remove sshd ssh-host-config cygrunsrv –start sshd

Reference: http://cygwin.com/cygwin-ug-net/

Thanks to Mike Skallas for his tips on "privilege separation" during the setup script. Thanks to Jan Haul of Hamburg, Germany g for his WinSCP link. Thanks to Brad Erdman, Institute for Advanced Computer Studies, University of Maryland, USA g for his confirmation of cygwin-sshd working on Windows Server 2003 Thanks to Richard Goodman of UK g for his tips on the order of CYGWIN variable. You can also use sshwindows from Sourceforge to install ssh and sshd on Windows without the full cygwin package. http://www.lns.cornell.edu/public/COMP/cygwin/cyg_inst_net.html is the link to install Cygwin with Xfree86. Thanks to Richard Ward for this link.

Disclaimer

© 2003-2006 Nicholas Fong HTML Editor used to generate this page is Nvu.

Document made with Nvu

Last revised: April 11, 2006

RSync Pull script

use this to rsync copy on a unix system

This script illustrates how one might use a pull method to rsync updates from a remote system running an rsync daemon.

Assumptions: 1) pubkey auth is already setup and running so that one my do ssh user@host without being prompted for a password 2) rsync is running in daemon mode on the remotesystem, exporting a module named home

#!/bin/sh
#
# rsync -av rsync://localhost:8730/home remotesystem/home/
# ssh -L 8730:localhost:873 remoteuser@remotesystem.pp.asu.edu
# ps ax | grep "sshuser@remotesystem" | grep -v "grep" | cut -f1 -d "
# batch script to rsync all items in homes module
# by db
# date: 20060306
#
sshuser="asurite-id"
rsyncuser="any user if implemented"
rhost="remotesystem.pp.asu.edu"
pforward="8730"
rport="873"
sshoptions="-fN -L $pforward:localhost:$rport "
module="home"
dest="remotesystem/home/"
/usr/bin/ssh $sshoptions $sshuser@$rhost
/usr/bin/rsync -av rsync://localhost:$pforward/$module $dest
echo "all done rsyncing, now cleaning up backgrounded shell"
/bin/kill -9 `/bin/ps ax | /usr/bin/grep "$sshuser@$rhost" | /usr/bin/grep -v "grep" |
/usr/bin/cut -f1 -d " "`
echo "background shells killed "

Yuck, there must be a better way to kill the backgrounded ssh session but I couldn't find a switch to return a pid, nor a cleaner way to just grab the running pid of ssh and send it a kill command.