Design goals of asd:
- Completely transparent user experience.
- Reduced wear to physical discs (particularly SSDs).
Since the sync targets is relocated into tmpfs (RAM disk), the corresponding onslaught of I/O associated with system usage of them is also redirected from the physical disc to RAM, thus reducing wear to the physical disc and also improving speed and responsiveness.
NOTE: edits made to /etc/asd.conf while asd is running will be applied only after asd has been restarted from the init service.
- At a minimum, define the sync targets to be managed by asd in the WHATTOSYNC array. Syntax below.
- Optionally uncomment and define the location of your distro's tmpfs* in the VOLATILE variable.
- Optionally enable the use of overlayfs to improve sync speed even further and use a smaller memory footprint. Do this in the USE_OVERLAYFS variable. Note that this option requires your kernel to be configured to use either the 'overlay' or 'overlayfs' module. See the FAQ below for additional details on this feature.
- Optionally disable the use of crash-recovery snapshots. Do this in the USE_BACKUPS variable.
- Optionally define the number of crash-recovery snapshots to keep. Do this in the BACKUP_LIMIT variable.
*Note that the default value of "/tmp" should work just fine for the VOLATILE setting. If using bleachbit, do NOT invoke it with the '--clean system.tmp' switch or you will remove a key dot file (.foo) from /tmp that asd needs to keep track of sync status. Also note that using a value of "/dev/shm" can cause problems with systemd's NAMESPACE spawning only when users enable the overlayfs option.
WHATTOSYNC=('/var/lib/monitorix' '/srv/http' '/foo/bar') or WHATTOSYNC=( '/var/lib/monitorix' '/srv/http' '/foo/bar' )
$ asd p Anything-sync-daemon on Arch Linux. Systemd service is currently active. Systemd resync service is currently active. Overlayfs v23 is currently active. Asd will manage the following per /run/asd.conf settings: owner/group id: root/0 target to manage: /srv/http/serve sync target: /srv/http/.serve-backup_asd tmpfs target: /tmp/asd-root/srv/http/serve dir size: 21M overlayfs size: 15M recovery dirs: 2 <- delete with the c option dir path/size: /srv/http/.serve-backup_asd-crashrecovery-20141105_124948 (17M) dir path/size: /srv/http/.serve-backup_asd-crashrecovery-20150124_062311 (21M) owner/group id: facade/100 target to manage: /home/facade/logs sync target: /home/facade/.logs-backup_asd tmpfs target: /tmp/asd-facadey/home/facade/logs dir size: 1.5M overlayfs size: 480K recovery dirs: none
Note that if a sync target is owned by root or another user, and if you call asd to clean, it will throw errors based on the permissions of your sync targets.
$ asd c Anything-sync-daemon on Arch Linux. Deleting 2 crashrecovery dirs for sync target /srv/http/serve /srv/http/.serve-backup_asd-crashrecovery-20141105_124948 /srv/http/.serve-backup_asd-crashrecovery-20150124_062311
The role of the timer is update the tmpfs copies back to the disk. This occurs once per hour by default. The timer is started automatically with asd.service.
# systemctl [option] asdAvailable options: start stop enable disable
Note that for these init systems, the supplied cron script (installed to /etc/cron.hourly) will run the resync option to keep the tmpfs copies sync'ed. Of course, the target system must have cron installed and active for this to happen.
- Arch Linux
- Debian (6+)
- Mint (14+)
- Ubuntu (10.04+)
A1: Overlayfs is a simple union file-system mainlined in the Linux kernel version 3.18.0. Starting with asd version 5.54, overlayfs can be used to reduce the memory footprint of asd's tmpfs space and to speed up sync and unsync operations. The magic is in how the overlay mount only writes out data that has changed rather than the entire sync target. See Example 1 below. The same recovery features asd uses in its default mode are also active when running in overlayfs mode. Overlayfs mode is enabled by uncommenting the USE_OVERLAYFS= in /etc/asd.conf followed by a restart of the daemon.
There are several versions of overlayfs available to the Linux kernel in production in various distros. Versions 22 and lower have a module called 'overlayfs' while newer versions (23 and higher) have a module called 'overlay' -- note the lack of the 'fs' in the newer version. Asd will automatically detect the overlayfs available to your kernel if it is configured to use one of them.
See the example in the PREVIEW MODE section above which shows a system using overlayfs to illustrate the memory savings that can be achieved. Note the "overlayfs size" report compared to the total "dir size" report for each sync target. Be aware that these numbers will change depending on just how much data is written to the sync target, but in common use cases, the overlayfs size will always be less than the dir size.
Q2: Why do I see another directory ".foo-back-ovfs" when I enable overlayfs?
A2: The way overlayfs works is to mount a read-only base copy (so-called lower dir) of the target, and manage the new data on top of that. In order to avoid resyncing to the read-only file system, a copy is used instead. So using overlayfs is a trade off: faster initial sync times and less memory usage vs. disk space.
Q3: My system crashed and asd didn't sync back. What do I do?
A3: The "last good" backup of your sync targets is just fine still sitting happily on your filesystem. Upon restarting asd (on a reboot for example), a check is preformed to see if the symlink to the tmpfs copy of your sync target is valid. If it is invalid, asd will snapshot the "last good" backup before it rotates it back into place. This is more for a sanity check that asd did no harm and that any data loss was a function of something else.
Q4: Where can I find this snapshot?
A4: You will find the snapshot in the same directory as the sync target and it will contain a date-time-stamp that corresponds to the time at which the recovery took place. For example, a /foo/bar snapshot will be /foo/.bar-backup_asd-crashrecovery-20141221_070112 -- of course, the date_time suffix will be different for you.
Q5: How can I restore the snapshot?
A5: Follow these steps:
- Stop asd.
- Confirm that there is no symlink to the sync target. If there is, asd did not stop correctly for other reasons.
- Move the "bad" copy of the sync taget to a backup (don't blindly delete anything).
- Copy the snapshot directory to the expected sync target.
Example using /foo/bar:
- mv /foo/bar /for/bar-bad
- cp -a /foo/.bar-backup_asd-crashrecovery-20141221_070112 /foo/bar
At this point, check that everything is fine with the data on /foo/bar and, if all is well, it is safe to delete the snapshot.
Q6: Can asd delete the snapshots automatically?
A6: Yes, run asd with the "clean" switch to delete snapshots.
- Currently, asd cannot handle open files on a sync target so if a hung process has something open there, it can be messy.
- If syncing a path where pacman (Arch Linux package manager) is expected to install files, pacman will stop the update since version 4.2 of pacman will refuse to install to a symlink. If you are syncing a path like this, you will need to stop asd prior to the package update.
- Project page: https://github.com/graysky2/anything-sync-daemon
- Wiki page: https://wiki.archlinux.org/index.php/Anything-sync-daemon
|26 November 2016|