sync2cd: an incremental archiving tool to CD/DVD

Description | Download | Usage | Configuration | FAQ | To do | Feedback | Links

Current version: 0.9 (2005.05.16)

Description

sync2cd is an incremental archiving tool. It allows backing up complete filesystem hierarchies to multiple backup media (e.g. CD-R). Files are archived incrementally, i.e. only new or changed files are stored during an archive operation.

All entity types are supported: directories, files, symlinks, named pipes, sockets, block and character devices.

Download

sync2cd is released under the GNU General Public License. The current version can be downloaded here.

sync2cd requires at least Python 2.2, and provides an installer based on distutils. This means that installation to the default location (/usr) is as simple as: 

   python setup.py install
To install sync2cd to a specific location, e.g. /usr/local, enter: 
   python setup.py install --prefix=/usr/local

sync2cd has been tested with Python 2.3 on Linux. It might work with other version and platform combinations, YMMV. If you were able to make it work with another configuration, please drop me a line (and, if necessary, send me a patch ;-)

Usage

Basic usage information is output with sync2cd.py --help: 

    [joe@pat sync2cd]$ sync2cd --help
    sync2cd 0.9   Incremental archiving tool to CD/DVD
    Copyright (C) 2005  Remy Blank

    Usage: sync2cd [commands] [options] config_file

    Commands: -c, --create               Create a new archive descriptor
              -g, --graft-list           Output a graft list for an archive
              -h, --help                 Show this text
              -p, --print                Print archive information
              -r, --restore              Restore from archives
              -s, --status               Print current synchronization status

    Options:  -a N, --archive N          Operate on archive number N
              -b GLOB, --glob GLOB       Add glob GLOB to pattern list
              -d DIR, --destination DIR  Restore into directory DIR
              -m N, --medium-size N      Set archive medium size to N
              -n CMD, --mounter CMD      Mount media using CMD for restore
              -v, --verbose              Be more verbose
              -x EXP, --regexp EXP       Add regular expression EXP to pattern list

Commands define what will be done and what will be output to stdout. Several commands can be specified at the same time, and will be executed in a sensible order (e.g. --create before --graft-list).

Options allow passing arguments to the selected commands. If the same option is specified on the command line and in the configuration file, the command line takes precedence.

-a N, --archive N
Specify that the selected command(s) should be executed on archive N. Note that this option will have no effect with --create and will be overridden by the newly created archive.

-b GLOB, --glob GLOB
Add the shell pattern GLOB to the list of patterns for items to restore. The pattern format is specified in the Exclude() configuration file function description. If the pattern matches a directory, all items below it will be matched as well.

-c, --create
Create a new archive, containing the files with the oldest modification date in the inputs specified in the configuration file, up to the capacity of one archive medium. This will create a new archive descriptor, with the same base name as the configuration file and a numbered extension, starting at 1 if no descriptor exists yet.

-d DIR, --destination DIR
Specify the base directory for a restore operation. All files and directories will be restored below this directory. It must exist before restoring.

-g, --graft-list
Output a list of graft-points to stdout, in the format expected by mkisofs with the -graft-points option.

-h, --help
Show some basic usage information and exits.

-m, --medium-size
Set the maximum size of an archive to N. Corresponds to the function MediumSize() in the configuration file.

-n CMD, --mounter CMD
Specify the command to be executed to mount a backup medium for restoring. It will be called with the path to the archive descriptor as an additional command-line parameter, and must print the mount point of the backup medium to stdout. Additionally, it will be called after the restore operation with a "dummy" descriptor with number 0 to allow ejecting the last media. For an example, see the sync2cd_mounter.sh script provided with sync2cd.

-p, --print
Print information about an archive to stdout. If --verbose is also specified, the list of files contained in the archive is also output.

-r, --restore
Restore the items matching the patterns specified with --glob and --regexp, or all items if no pattern was specified.

-s, --status
Print the status for a backup set, i.e. the total size of all files that need to be archived. If --verbose is also specified, the list of files that need to be archived is also output.

-v, --verbose
Output more information to stdout for various commands.

-x EXP, --regexp EXP
Add the regular expression EXP to the list of patterns for items to restore. If the regular expression matches a directory, all items below it will be matched as well.

Here are a few basic examples:

Configuration file format

The configuration file is actually Python code calling functions defined in sync2cd.py and passing configuration information. The functions available are described below.

BaseDir(Dir)

Set the current working directory to Dir before starting. All paths specified with Input() are relative to this directory. This option corresponds to the -C or --directory option of tar.

Default:. (current directory)
Example:BaseDir("/home")

Exclude(Pattern)

Exclude files matching the shell pattern Pattern from the archive. Several exclude patterns can be specified. The pattern matching is done against the path relative to BaseDir(). If a directory matches an exclude pattern, it is not recursed into.

As usual with shell patterns, a * wildcard matches zero or more characters except path separators (e.g. "/" on *nix). A new wildcard, **, matches zero or more characters, including path separators.

Example:ExcludeGlob("Music/Country/**.mp3")

This excludes all mp3 files in Music/Country and in all subdirectories.

ExcludeRegexp(Pattern)

Exclude files matching the regular expression Pattern from the archive. Several exclude patterns can be specified. The pattern matching is done against the path relative to BaseDir(). If a directory matches an exclude pattern, it is not recursed into.

For more information about regular expression syntax in Python, see this page.

Example:ExcludeRegexp("Music/Country/.*\\.mp3")

This excludes all mp3 files in Music/Country and in all subdirectories (note escaping of "\").

HashFunction(Hash)

Specify the hash function to be used to check files for content modification. Currently supported: md5 (128 bits), sha1 (160 bits).

Default:md5
Example:HashFunction("sha1")

Input(Path)

Add a file or directory to be archived. Several inputs can be specified. The use of a directory name always implies that the subdirectories below should be included in the archive. Path must be a relative path specification, and is interpreted relative to BaseDir().

Example:Input("Music")

MediumSize(Value)

Set the maximum size of an archive to Value. This is typically used to span a backup over multiple media.

Value is an integer giving the size in bytes, or a string containing a floating-point value optionally followed by the suffix k, M, G, T, P, E.

Default:0 (no limit)
Example:MediumSize("4.3G")

Here are a few examples of configuration files.

Frequently asked questions

(Not that so many people actually asked...)

To do

The following features will be added to sync2cd as time permits.

Feedback

If you are using or trying to use sync2cd, I would be happy to hear from you! I'm especially interested in the following:

In any case, just drop me an e-mail.

Links


Copyright (C) 2005 Remy Blank