Category Archives: Linux

Linux image backups with UrBackup 2.5.y

Posted on by
1

So I’ve resisted the idea of Linux image backups for a while. After all, one can just backup all the files, right? True, even with Linuxs fast file access this might take a bit longer than doing an image backup if one has a a lot of files and one wants to back them all up…

Image backups also make backups and restores a lot easier. One doesn’t have to think about what to backup, instead it’s just the whole thing. When restoring, it’s just one step: Copy back the volume.

Advantages:

  • Everything is backed up
  • Content does not affect backup or restore performance (e.g. many small files don’t slow down backups and restores)
  • 1:1 backups and restores of volumes

Disadvantages:

  • Can’t select subset to backup (e.g. exclude temporary file data)
  • Can’t migrate to different file system on restore
  • File level deduplication doesn’t work
  • E.g. defragmentation causes changes to backup

The following are step-by-step instructions on how to backup a Linux VPS.

Backup VPS (Step-by-step)

Assumption: VPS is running Debian 10/buster. 2.5.x server + client is used.

1. Add new client on server web interface:

2. Add active/Internet client

3. Copy & paste Linux client installation command and follow setup:

4. Reboot client/VPS (if not using dattobd)

5. Create image backup (“C” gets automatically translated to the Linux root volume)

Restore VPS (Step-by-step)

1. Browse to image backup in “Backups” tab on web interface

2. Click on restore Linux image

3. Copy & paste into new Debian 10/buster instance and follow instructions:

Raspberry PI 4 Backup Appliance update

Posted on by
1

An update for the Raspberry PI 4 backup appliance (last post). Raspberry PI specific changes:

  • Upgrade Linux kernel to 5.4.y. This allows e.g. a swap file on the system disk
  • Removed the USB 3.0 UAS driver, since it doesn’t seem to work (with the disks I tested it with — see also this thread). This makes USB disks slower, but at least it works reliably
  • Use xchacha12+adiantum as system disk encryption method since RPI4 doesn’t have AES CPU instructions. This significantly increases performance

    If you want to use it manually outside the appliance use
    cryptsetup luksFormat --cipher xchacha12,aes-adiantum-plain64 --type luks2 --sector-size 4096
    to format the encrypted disk

Download the image for the SD card.

Write the image to the SD card using e.g. Win32 Disk Imager.

Discussion

Discuss the Infscape UrBackup Appliance here.

Building completely static Linux binaries via Android NDK

Posted on by
Reply

The Android NDK can be used to build completely static Linux binaries which run on any Linux.

Advantages:

  • Runs on every Linux distribution be it RHEL, Debian or Alpine
  • Linux (i.e. Linus Torvalds) is pretty strict about backwards compatibility so it’ll continue to run on future Linux kernels without problems
  • Theoretically it’ll even work without a Linux distribution, e.g. in a minimal Docker container
  • All the dependencies are static and confirmed to work together

Disadvantages:

  • Might use more memory at runtime since libraries (such as libc.so) aren’t shared with other programs
  • Behavior might deviate from other programs in the distribution. E.g. when it doesn’t respect /etc/nsswitch.conf, parses /etc/resolv.conf differently or when OpenSSL looks for the root certificates in the wrong place

To mitigate the disadvantages the UrBackup client Linux binary installer first tries to use a glibc (non-static) build on amd64/x86_64 (the most common platform). Only if that doesn’t run (e.g. because the glibc is too old), does it fall back to the NDK build.

Previously this was done with ELLCC, but that doesn’t support C++ exceptions and doesn’t seem to get updated anymore.

Usage

Since UrBackup uses autotools to build, cross compilation is automatically present and can be used simply by setting a few environment variables before building:

export NDK=/path/to/android/ndk/android-ndk-r20
export HOST_TAG=linux-x86_64
export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/$HOST_TAG
export TARGET=x86_64-linux-android
export TARGET2=${TARGET}29
export AR=$TOOLCHAIN/bin/$TARGET-ar
export AS=$TOOLCHAIN/bin/$TARGET-as              
export CC=$TOOLCHAIN/bin/$TARGET2-clang                
export CXX=$TOOLCHAIN/bin/$TARGET2-clang++                
export LD=$TOOLCHAIN/bin/$TARGET-ld                
export RANLIB=$TOOLCHAIN/bin/$TARGET-ranlib                
export STRIP=$TOOLCHAIN/bin/$TARGET-strip
export NDK_CPUFLAGS=""
 ./configure --enable-headless --enable-c-ares --enable-embedded-cryptopp --enable-embedded-zstd LDFLAGS="-static -Wl,--gc-sections -O2 $NDK_CPUFLAGS -flto" --host $TARGET --with-zlib=$TOOLCHAIN/sysroot/usr --with-crypto-prefix=$TOOLCHAIN/sysroot/usr --with-openssl=$TOOLCHAIN/sysroot/usr CPPFLAGS="-DURB_THREAD_STACKSIZE64=8388608 -DURB_THREAD_STACKSIZE32=1048576 -DURB_WITH_CLIENTUPDATE -ffunction-sections -fdata-sections -ggdb -O2 -flto $ARCH_CPPFLAGS" CFLAGS="-ggdb -O2 -flto $NDK_CPUFLAGS" CXXFLAGS="-ggdb -O2 -flto $NDK_CPUFLAGS -I$NDK/sources/android/cpufeatures/ -DOPENSSL_SEARCH_CA" LIBS="-ldl"

See also the script that builds the Linux client installer.

The advantage UrBackup has here, is that many dependencies are already bundled with the source code like crypto++, zstd, lua and sqlite. All the dependencies that are not bundled need to be compiled to a static library (for every architecture) and in my case I have put them into $TOOLCHAIN/sysroot/usr/.

Complications

The Android NDK is of course made to build programs for Android. There are significant differences between Android and other Linux distributions. Here is two I found:

If one wants to resolve a DNS name such as example.com to an IP address one usually uses getaddrinfo(). This won’t work with the Android NDK libc (bionic libc), because it uses the Android resolver by calling some Android runtime java code that is obviously not present on non-Android systems. The solution for this problem was to use c-ares instead of the libc to resolve addresses. If you are using a library that resolves addresses that needs to have the option of using c-ares as well (such as cURL).

A call to system() or popen(), calls the shell (usually /bin/sh). The Android libc, however, calls /system/bin/sh instead, which is of course not present on non-Android distributions. The solution was to replace all those calls to an own version. Again, if any library one uses does use those, they’ll need to be replaced.

On x86 and amd64/x86_64 the Android NDK automatically uses SSE4 CPU instructions which older CPUs do not support. Users complained about that and that client should run on as many systems as possible.
To disable SSE4 the Android NDK compiler needs to be passed “-mno-sse4a -mno-sse4.1 -mno-sse4.2 -mno-popcnt” (that was the only way I found). The problem is that some SSE4 instructions are in the libc and libc++. So, the libc and libc++ need to be recompiled with those flags. The bionic libc source code is (unfortunately) NOT part of the Android NDK source code, it is part of the Android source code.
So after downloading 50GB of Android source code for half a day, one needs to change the Go source code of Androids custom build tool (soong). Adjust e.g. build/soong/cc/config/x86_64_device.go, select the correct architecture to build, then fish out the libc.a (and libc++.a) from the output directory and replace the more then dozens of libc.a occurrences in the NDK (no idea which one it actually uses).

One final complication was that crypto++ does actually feature test and then use SSE4 instructions, so they need to be enabled for some crypto++ compile units. If one specifies both “-mno-sse4” and “-msse4”, “-mno-sse4” seems to take precedence. So the solution was to have a compiler wrapper script that removes “-mno-sse4” in such a case.

Raspberry Pi 4 backup appliance

Posted on by
22

Rapsberry PI 4 is (finally) relatively powerful hardware and suited to host a UrBackup appliance. Specifically:

  • 4 GB RAM
  • 64 bit support
  • USB 3
  • Gigabit ethernet
  • Build-in Wifi

The RAM is of course not ECC-RAM which makes it unsuited for large UrBackup appliance instances or serious use (you could always risk it, see notes on ECC RAM section on the bottom).

The main advantages of Raspberry Pis are that they are widely available, have good support and merely by having so many people focused on them, having few bugs.

A few gotchas when setting up a UrBackup server:

  • Use the SD card as a read-only device for booting the operating system. Regularly writing to the SD card will break it because SD cards are not designed for such use. There are “industrial” SD cards designed for regular (random) writes (e.g. SwissBit) but they cost way too much (an external USB SSD is cheaper)
  • I had a lot of problems getting the 64bit Linux kernel to boot, but I guess this will improve
  • Power supply is still an issue. The USB SSD drive I was using caused issues because (I think) the power supplied by the Raspberry wasn’t enough. I’d guess the Raspberry + USB SDD need a 4A power supply at least.

The system plus costs (w.o. taxes):

Raspberry PI 4 Model B 4GB50 €
Intenso SSD 256GB37 €
32GB microSD SDHC3 €
Raspberry Pi 4 Case6 €
Powered USB 3 hub 5V/4A (20W)20 €
USB 3 to USB C cable7 €
Sum (w.o. taxes)123 €

Once you have the base system, you can add USB disks as backup storage, use those 2-bay docking stations + disks or use larger 4-bay USB external disk enclosures. Adding single USB disks is probably cheapest.
Alternatively, if using the Infscape UrBackup Appliance, you can store your backups directly to the cloud (Amazon S3 or compatible) using the SSD as local cache.

Software

I’ll describe how to setup the Infscape UrBackup Appliance. You can of course also install Raspbian and install UrBackup on that.

Download the image for the SD card.

Write the image to the SD card using e.g. Win32 Disk Imager.

For setup, you’ll have to attach the Raspberry PI to your wired ethernet network. After setup, you can run it via Wifi only. Put the SD card into the Raspberry and start it up. After boot it’ll show its IP address on its display. Alternatively, you can look for new devices on your router, or using an IP range scanning tool.

Browse to http://YOURRASPBERRYIP and follow the setup wizard.

Make sure the SSD is attached and click on “+Use device” on the status screen. Then use the SSD as external system disk (UrBackup database, logs and temporary files will be stored on the SSD instead of the SD card).

If you want to store the backups to S3, select that you want to use the system disk as cache (or attach an additional SSD as cache only disk). If you want to store the backups to a local (USB) disks I’d recommend to also use the system disk as cache for the local USB disks (cached, auto-layout RAID).

ECC-RAM

If you are unlucky a single bit error in your Raspberry PI RAM can make the whole backup storage unwritable. In that case you’ll have to either make a new backup server and replicate everything (that is still readable) to that one, or simply start with a new backup server. The worst case is that it corrupts data in such a way that it does not detect the corruption, in which case you’ll only find there is a problem when you restore/actually want to use the backup.
That said, the probability of the RAM having problems is perhaps very low. We can probably trust the Raspberry PI foundation to have selected reliable RAM plus settings that minimize the risk of corruption.
The cloud/RAID backup storage has an advanced option “Size of in-memory write-back cache”. Per default it is disabled and you probably shouldn’t use it if you don’t have ECC-RAM as it increases the time data is kept in memory before it is written to checksum protected storage.

Discussion

Discuss the Infscape UrBackup Appliance here.

Btrfs file system stability status update

Posted on by
Reply

I wrote before in a blog entry about btrfs stability. That was almost exactly seven years ago.The issue I had back then (about a hard link limit of hard links in the same directory) was fixed a long time ago.

Since then btrfs has come a long way. The most significant development was that SUSE started using btrfs as root file system and put several developers to work on finding and fixing bugs. Simultaneously only few new features were added. This significantly improved stability (at least for the root file system use case).

As just hinted, the main problem is that each developer just looks after their set of use cases. So, developers from Facebook mainly look at their use case of using btrfs on their HDFS storage nodes. They have redundancy etc. on a higher level, so they are probably using btrfs only with single disks and focusing on making btrfs performant and stable within this use case. SuSE, as said, uses it as root file system (system files), where there are single disks or, at most, mirrored disks. File system repair isn’t too important, as those file systems don’t get very large and contain non-unique data.

Compared to that when using UrBackup with btrfs:

  • Storage is for backup and/or long term archival (RAID6 etc. would be appropriate)
  • Can have a lot of threads writing data to the file system at once
  • Can have combination of threads writing lots of small files and large files (image backups)
  • Lots of snapshots (backups) of different sub-volumes, some of which can persist for a long time
  • Deduplication (at least file level)
  • Can’t reboot the server, when it gets stuck like you’d be able to with e.g. a HDFS node
  • UrBackup imposes a database style workload (db for deduplication), where latency is important as well as the backup workload, where throughput is important
  • File system can get very large and it is very time consuming to move data from a damaged (read-only mounted) file system to a new one

This, combined, was guaranteed to cause problems when seriously using btrfs as UrBackup storage. The most persistent issue was premature ENOSPC, where btrfs has an error saying it is out of space, remounting the file system read-only, even though there is still a lot of free space (both for data and metadata) available on the disk(s). The problem seems to be solved (on the systems I observe) with Linux 5.1 or Linux 4.19.x with this patch. RAID56 is still officially “unstable”.

Btrfs isn’t alone with having issues with this work load. When using UrBackup seriously (large number of clients with many simultaneous backups) e.g. with ZFS, I experienced a lot of different issues with ZFSOnLinux (especially in connection with memory/thread management), making ZFSOnLinux unusable. ZFS on FreeBSD was a little better, but there also were issues that occurred about once per week causing hangs.

Btrfs also isn’t the sole area where Linux was needing improvement in the storage area. For example, writeback throttling was significantly improved a few years ago. This example improves the memory management, making Linux more able to handle the mixed database and archival workload mentioned above. Not to say that all errors are fixed. For example, I recently learned that a call to synchronize the file system to make sure that data is actually written to disk does not return errors on Linux. There is now a work-around for that, at least for btrfs, in UrBackup but there hasn’t been a fix on the Linux side in general, yet.

Another important consideration is performance. One thing w.r.t. btrfs to keep in mind is that there is a feature, or better trade-off, where there is a back-reference from each disk data location to each file/item using that disk data location. So, you can ask btrfs to list all files having data at e.g. disk offset 726935388160 and it will do so fast compared to other file systems like EXT, ZFS, NTFS, etc. which would have to scan through all files. Managing the backref metadata comes with the cost of having more metadata in general, though. Operations such as writing new data to files, deleting files, deleting snapshots, etc. become a bit slower as the backref metadata has to be added or deleted, in addition to the (forward) metadata pointing from file to data. Btrfs goes to great lengths making this fast, though (delayed backrefs etc.). There is a number of unique features of btrfs which could be implemented because of the presence of back-references:

  • File system shrinking
  • Device removal
  • Balancing
  • If there is a corruption, it can show all the files (and offsets into files) affected
  • Quotas
  • Btrfs send

The performance problem with backrefs is that if a file has a lot of reflinks or snapshots, it may have a lot of backrefs. Running one of the operations above then involves a lot of iteration through all backrefs, making this operation (unusably) slow. UrBackup naturally creates a lot of snapshots and may reflink certain files many times. The work-around (at least for me) is to avoid the above operations at all cost and patch out the reflink walking in btrfs send.

Conclusion

Should you use btrfs with UrBackup? It is certainly the file system with which UrBackup works the best. Outside of that you’d have to see if you can have:

  • Linux kernel at least 4.19.x (I suggest using the latest LTS kernel). If you have ENOSPC issues with 4.19.x apply patch.
  • Able to avoid above operations such as file system shrinking or device removal
  • Able to avoid btrfs RAID56 (i.e. use RAID1(0) instead or use btrfs in single mode on top of a storage layer that provides integrity such as Ceph or the UrBackup Appliance)
  • I’d suggest using ECC memory, as btrfs repair once it is damaged is mostly not possible

Connect clients with a HTTPS CONNECT web proxy

Posted on by
1

With 2.4.x you can use UrBackup with a HTTPS proxy. This way you can have the web interface and the clients connecting at the same port, secured by the same transport encryption (SSL). This post shows how to do this in combination with the Apache web server.

The idea is that the client connects to the web server and issues a HTTP CONNECT request to the actual UrBackup server.

First Enable CONNECT proxy module in apache. On debian via

a2enmod proxy_connect

Then allow connections to the UrBackup server Internet port by adding

AllowConnect 55415

to your apache configuration.

Next in your apache virtual host configuration, set proxy options such as the timeout, allow proxy connections to the UrBackup server, and disallow them to every other host:

ProxyTimeout 600
ProxyRequests On  
<Proxy 127.0.0.1:55415>
</Proxy>
<ProxyMatch ^(?!127.0.0.1:55415$).*$>
    Order Deny,Allow
    Deny from all
</ProxyMatch>

Then, go to your UrBackup server web interface and setup your web server URL as Internet client proxy (https://example.com) and the Internet server name/IP as 127.0.0.1. Internet clients should then start connecting via your web server to your UrBackup server. Once all clients connect this way you could turn off UrBackup’s build in Internet transfer encryption and rely on SSL.

Fixing client IP addresses

You may notice that on the status page all Internet clients now show the IP address of your web server as their IP address. Fixing this is a bit difficult, as there is no standard way to forward the client IP address information from the web server (compared to a normal HTTP proxy where there is a X-Forwarded-For header). So, a bit of hacking to fix this is in order. I modified the mod_proxy_connect apache plugin to forward the client IP information in a 50 byte buffer to the backend: mod_proxy_connect.c
On debian you could replace your original mod_proxy_connect with the modified one via the following commands:

apt install apache2-dev
wget https://gist.githubusercontent.com/uroni/143c0d7ed6169e89f2d6c59a870dd4cc/raw/28dd30b1f82938777c504f2afdc5f162fd91b3fd/mod_proxy_connect.c
apxs -i -a -c mod_proxy_connect.c

Then in the UrBackup server advanced global settings set “List of server IPs (proxys) from which to expect endpoint information (forwarded for) when connecting to Internet service (needs server restart)” to include your web server IP (127.0.0.1 in the example here). After a server restart you should be able to see the actual client IP instead of the web server IP on the status screen.

Fixing SNI errors

If you have multiple virtual hosts with SSL there is an issue with SNI. Apache2 automatically compares the hostname in the CONNECT request with the server name in the SSL connection (SNI) and rejects the request if they differ. The only solution (or ugly hack) I found to fix this was to add the hostname with the target IP to /etc/hosts and then use the hostname instead of the IP in the CONNECT request. I.e., add “127.0.0.1 example.com” to /etc/hosts, then replace 127.0.0.1 with example.com in all the configuration above.

Additional proxy authentication

As additional security layer, one can require proxy authentication. Clients need to know a username+password to get through the web server to the UrBackup server. With apache2 e.g.:

 htpasswd -c -b /etc/apache2/urbackup_password urbackup passw0rd

Then modify the proxy section to:

<Proxy 127.0.0.1:55415>
    AuthType Basic
    AuthName "Restricted UrBackup"
    AuthBasicProvider file
    AuthUserFile "/etc/apache2/urbackup_password"
    Require user urbackup
</Proxy>

Afterward add username+password to the proxy url, that is e.g. https://urbackup:password@example.com

Image mounting in UrBackup Server 2.1.x

Posted on by
6

UrBackup 2.1.x can now mount image backups. That is, it lists the image backups it has on the web interface, you can browse into them and e.g. download files or directories as a ZIP file.
I am particularly proud that this works on Linux as well as on Windows with both raw image files and VHD(z) files. On FreeBSD it only works with raw image files currently. The screenshots are from a Windows UrBackup server.

On Linux it uses libguestfs-tools to mount images in a sandboxed virtual machine. On Windows/FreeBSD mounting a hostile image may be a dangerous operation.

How I backup LVM volumes on my Xen server

Posted on by
1

I’ve got a Xen server which runs a couple of Linux and Windows VMs. The VMs are stored in LVM volumes on a LVM volume group which is on a bcache device. The bcache device consists of a mirrored SSD pair (using mdraid) as cache and a mirrored HDD pair (also using mdraid) as backing storage. The SSD caching gives a nice performance boost, but nowadays I would go with SSD storage only, because bcache caused some problems (did not play nice with udev during boot).
The Windows VMs are backed up by installing the UrBackup client in the VMs. To restore I’d need to boot the restore CD in Xen or restore the Windows images via command line in the hypervisor.
The Linux VMs are backed up at hypervisor level in the Xen dom0 (which is Debian in this case) using LVM snapshots. To create and remove LVM snapshots I have following snapshot creation and removal script (the volume group on which the volumes are is mirror-vg).

Snapshot creation script at /usr/local/etc/urbackup/create_filesystem_snapshot:

#!/bin/bash
set -e
SNAP_UID=$1
VOLNAME="$5"
VGNAME="mirror-vg"
if [[ $VOLNAME == "" ]]; then
        echo "No volume name specified"
        exit 1
fi
if [[ $VOLNAME == "other-data" ]]; then
        VGNAME="data2-vg"
fi
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
fi
export LVM_SUPPRESS_FD_WARNINGS=1
lvcreate -l100%FREE -s -n $SNAP_UID /dev/$VGNAME/$VOLNAME
SUCCESS=0
trap 'test $SUCCESS = 1 || lvremove -f /dev/$VGNAME/$SNAP_UID' EXIT
mkdir -p /mnt/urbackup_snaps/${SNAP_UID}
mount -o ro /dev/$VGNAME/$SNAP_UID /mnt/urbackup_snaps/${SNAP_UID}
SUCCESS=1
echo "SNAPSHOT=/mnt/urbackup_snaps/$SNAP_UID"
exit 0

Snapshot removal script at /usr/local/etc/urbackup/remove_filesystem_snapshot:

#!/bin/bash
set -e
SNAP_UID=$1
SNAP_MOUNTPOINT="$2"
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
fi
if [[ "$SNAP_MOUNTPOINT" == "" ]]; then
        echo "Snapshot mountpoint is empty"
        exit 1
fi
if ! test -e $SNAP_MOUNTPOINT; then
        echo "Snapshot at $SNAP_MOUNTPOINT was already removed"
        exit 0
fi
if ! df -T -P | egrep "${SNAP_MOUNTPOINT}\$" > /dev/null 2>&1; then
        echo "Snapshot is not mounted. Already removed"
        rmdir "${SNAP_MOUNTPOINT}"
        exit 0
fi
if lsblk -r --output "NAME,MOUNTPOINT" --paths > /dev/null 2>&1; then
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" --paths | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
else
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
        VOLNAME="/dev/mapper/$VOLNAME"
fi
if [ "x$VOLNAME" = x ]; then
    echo "Could not find LVM volume for mountpoint ${SNAP_MOUNTPOINT}"
    exit 1
fi
if [ ! -e "$VOLNAME" ]; then
    echo "LVM volume for mountpoint ${SNAP_MOUNTPOINT} does not exist"
    exit 1
fi
echo "Unmounting $VOLNAME at /mnt/urbackup_snaps/${SNAP_UID}..."
if ! umount /mnt/urbackup_snaps/${SNAP_UID}; then
        sleep 10
        umount /mnt/urbackup_snaps/${SNAP_UID}
fi
rmdir "${SNAP_MOUNTPOINT}"
echo "Destroying LVM snapshot $VOLNAME..."
export LVM_SUPPRESS_FD_WARNINGS=1
lvremove -f "$VOLNAME"

The snapshot scripts are specified via the file /usr/local/etc/urbackup/snapshot.cfg:

create_filesystem_snapshot=/usr/local/etc/urbackup/create_filesystem_snapshot
remove_filesystem_snapshot=/usr/local/etc/urbackup/remove_filesystem_snapshot
volumes_mounted_locally=0

Then I have a virtual client for each LVM volume that needs to be backed up. I have put those virtual clients in a settings group with the default path to backup “/|root/require_snapshot”.

For restore I need to recreate the LVM volume. Create a file system on it (e.g. with mkfs.ext4) mount it in the hypervisor and then restore via.

urbackupclientctl restore-start --virtual-client VOLUMENAME -b last –map-from / --map-to /mnt/localmountpoint

New in UrBackup 2.0.x

Posted on by
3

Wactivitieseb interface modernization. The web interface was a little bit utilitarian which gave many people the wrong impression. With the help of mombojuice the web interface was improved such that it looks much more modern and professional. Many small improvements were made as well. For example the dates are now formatted according to browser locale, backups can be started via drop-down menu and the live log of a running backup can be directly accessed from the activities screen.

Improved file deduplication. Completely reworked the file deduplication and file backup statistics calculation. This should be much faster, scalable and reliable now.

File lastmodifiedbackup improvements. File meta-data such as last modified time and file permissions are now backed up on all supported client systems (Windows, Linux, Mac OS X). Supporting more exotic file system features such as sparse files UrBackup is now a fully featured file backup solution.

File backup restore. To restore the file meta-data UrBackup has now an integrated file restore. The file restore reuses client-side hashes, if present, and transfers only differences, such that restoring folders with only few changes since the restored backup is fast.

access_backupsDirect backup access. If configured, the backed up file permissions are used to allow clients direct access to their files with only minimal configuration. On Windows there is a shortcut in Explorer which directly opens the relevant/file folder in the browser. There is a new list view which shows a file/folder in all backups. For files, hashes are used to show when the file content changed (versions).

image_backup_settingsImage backup improvements. UrBackup supports GPT formatted disks now and the restore CD boots on UEFI firmware devices (also with secure boot enabled). In combination with btrfs, UrBackup supports an incremental forever style image backup and image backups over 2TB. For VHD/VHDZ UrBackup has now settings to base incremental backups on the last or last full image backup. Full image backups can be configured to be synthetic full backups transferring only changes since the last image backup.

Significant security improvements. Forward secrecy for Internet clients via ECDH and Internet client security improvement by using AES-GCM. Switch from DSA to ECDSA for client update and server identity signatures. Web server/restore CD login now uses PBKDF2.

Mac OS X client. There is nowmac_backup_running a UrBackup Client for Mac OS X. This client is fully featured, excluding image backup (like Linux client). The Mac OS X client can be used as a technically superior backup solution to Time Machine.

Improved command line. Mainlttyy for Linux all command line usage has been significantly improved. This includes the command line client (urbackupclientctl), the server command line (urbackupsrv) and the restore client.

Linux file system snapshotting. Snapshotting now also works on Linux and is fully integrated. A portable Linux client includes snapshot scripts for LVM, dattobd and btrfs which work without changes in most cases.

Lots of other changes. Proper symbolic link backup. Virtual clients allow you to backup different sets of files at different intervals and max/min amounts. Simultaneous image and file backups. Different backup speeds and backup intervals at different times. Improved Internet transfer compression. New hashing method where the server only needs to hash changed parts of a file.