Rsync is one of the most useful tools for a systems administrator. Regardless of what your specific roll or responsibility is. At some point you are going to need to copy the data from one place to another. And Rsync is the tool which will help ensure you quickly and accurately make a copy of your data. So in this post I hope to convey how to use Rsync, but focusing on the basic uses that I find most helpful each day.
What is Rsync
Rsync was initially built as a basic clone of “rcp” (Remote Copy) but with a handful of additional features. That handful of additional features has expanded over the years and made Rsync an indispensable tool. This simple tool can be used to copy files between directories on a local computer. Or you can use it to copy files to and from remote systems. My favorite part of Rsync is its ability to quickly compare the source and target locations. This ensures that only new, updated, or other file changes are transferred. Helping you save time and bandwidth when copying larger numbers of files.
So How do I Use Rsync?
The basic command is pretty simple, rsync [options] [source] [destination], and in this simple form you can easily copy data between local directories. ie:
rsync /home/bdoga/source/file /home/bdoga/destination/
This command will take “file” and place it inside the “/home/bdoga/destination/” directory. If you instead would like to copy all of the contents of one directory into another you simply need to add the “-r” (recursive) option. ie:
rsync -r /home/bdoga/source/ /home/bdoga/destination/
Thus all of the contents of “/home/bdoga/source/” will now be copied into “/home/bdoga/destination”. But it is important to note, that if a file with an identical name exists in the destination, it will be overwritten. In addition the “-r” option does not preserve ownership, permissions, or access/modification timestamps. But that is where the next option comes in “-a” (archive).
It is also important to note that if you want to copy just the contents of the source directory, you must end with a trailing “/”. If you fail to add the trailing “/” Rsync will copy the specified directory as well as the contents into the destination. Rather than just the contents of the directory.
The most useful options
The archive option not only copies the files recursively, but also preserves the file permissions and timestamps. I find this the most useful option because when I want to copy a source directory I typically want to be able to restore it with the permissions intact.
Another option that is sometimes useful, depending on the scenario is the “-z” (zip) option. It instructs Rsync to compress the files being copied to ensure they use less bandwidth. Not always useful when copying files over a Gigabit or faster lan, but can be helpful over a slower internet connection.
The next most useful option I frequently use is “-v” (verbose) which tells Rsync to give you more information about the files being transferred. This can be useful to see exactly what is being transferred. It also lets you know exactly what was and was not copied if there is an issue.
And then there is the “-h” (Human Readable) option which makes sure that all numbers/sizes are printed in an easily readable format. For instance rather than reporting that 856342348 bytes were transferred, it would report 816.67 MB were transferred.
And all of these options can be used together as needed. As in this example which will recursively transfer the files while preserving their permissions and timestamps. Also giving verbose output and zipping the files during transfer.
rsync -avzh /home/bdoga/source/ /home/bdoga/destination/
Sample Command Output
root@bdoga:~/test# ls -lah test1 total 4.0K drwxr-xr-x 3 root root 76 Dec 21 18:33 . drwxr-xr-x 4 root root 32 Dec 21 17:45 .. -rw-r--r-- 1 bdoga bdoga 7 Dec 21 17:47 bob -rw-r--r-- 1 bdoga bdoga 0 Dec 21 17:46 doug drwxr-xr-x 2 root root 18 Dec 21 17:46 subdir -rw-r--r-- 1 bdoga bdoga 10M Dec 21 18:33 test.img -rw-r--r-- 1 bdoga bdoga 100M Dec 21 18:33 test2.img root@bdoga:~/test# rsync -avh ./test1/ ./test2 sending incremental file list ./ bob doug test.img test2.img subdir/ subdir/file sent 115.37M bytes received 122 bytes 46.15M bytes/sec total size is 115.34M speedup is 1.00 root@bdoga:~/test# rm -rf test2/* root@bdoga:~/test# rsync -avzh ./test1/ ./test2 sending incremental file list ./ bob doug test.img test2.img subdir/ subdir/file sent 112.61K bytes received 122 bytes 25.05K bytes/sec total size is 115.34M speedup is 1,023.21 root@bdoga:~/test# ls -lah test2 total 111M drwxr-xr-x 3 root root 76 Dec 21 18:33 . drwxr-xr-x 4 root root 32 Dec 21 17:45 .. -rw-r--r-- 1 bdoga bdoga 7 Dec 21 17:47 bob -rw-r--r-- 1 bdoga bdoga 0 Dec 21 17:46 doug drwxr-xr-x 2 root root 18 Dec 21 17:46 subdir -rw-r--r-- 1 bdoga bdoga 10M Dec 21 18:33 test.img -rw-r--r-- 1 bdoga bdoga 100M Dec 21 18:33 test2.img
The above command output shows the contents of the source and destination directories. And also shows the difference between running rsync with and without the “-z” option.
Conclusion
Rsync will become a super useful part of your systems administration toolkit. Now that you have a basic understanding of how to use Rsync you are ready to see how to connect to a remote computer. Or learn how other programs like Rdiff-backup build upon it to create an awesome tools. And a big thanks to some other sites which we have referenced over the years. Check them out here, and here.
