UrBackup Windows Installer creator

If you wanted to create a generic UrBackup client installer for your internet server you’d have to create one yourself following instructions here .

I have made https://installercreator.urbackup.org/ to make this process easier. Given the correct parameters it’ll create a executable for you that’ll create a client on your server, then download the client installer for this newly created client and then run this installer. It’ll use the computer name as default client name, optionally appending a random string, such that name collisions do not occur.

RAID calculator/simulator

Go to RAID calculator/simulator

This is a RAID calculator/simulator. Given a desired failure percentage and space overhead, it simulates a few disk arrangements and then outputs the one where the failure percentage is lower or equal to the desired one.

So, you don’t have to read up on when it is best practice to use RAID5 vs. RAID6 or how many disks groups to arrange your RAID50 into.

A few notes:

  • The default disk failure percentage per year is 5%. This is the upper end. Depending on disk model, how much they are used and things like temperature and vibration isolation one would probably set it to more like 1% (See e.g. the statistics here)
  • The Unrecoverable Read Error (URE) rate is 1e14 per default. As mentioned in the Wikipedia article this often also is 1e15. You can get this info from the disk data sheets for example here for Seagate IronWolf.

What can still be improved:

  • Currently it assumes that disk failures are independent (“Independent and identically distributed random variables” from probability theory). I.e. one disk failing doesn’t make it more likely that others will fail. This may not be the case if you put more than one disk of the same model or even production batch into the same RAID group, because those models may all have the same flaws which make the disks die about the same time after identical usage (taking your RAID down with it). So, try to avoid putting identical disks into the same RAID group
  • As one can see it only uses one failure probability per year. A more accurate simulation can probably be had if the failure probability is adjusted for age. Disks fail often early on, then fail less, then start to fail more often when they are older.

Infscape UrBackup Appliance uses this RAID calculator/simulator to automatically arrange the RAID configuration according to the storage failure percentage and overhead you specify. This has a few advantages compared to manually setting up a RAID:

  • It is easier because you don’t have to manually arrange the disks. And it also avoids accidental misconfiguration
  • When a disk fails it can simply recalculate the RAID configuration with one disk less and use this configuration to write new data. After waiting a bit for the disk to be replaced it can also automatically reshape the data into this new configuration
  • When a disk becomes full it can calculate a RAID configuration without the full disk and write more data using this new configuration allowing the RAID to fill disks with different sizes while keeping the overall failure guarantees of the storage

Skewed Color

In which I debug why the tray icon on my laptop doesn’t turn yellow during backups anymore and just cannot find the issue:

Turns out I have f.lux on and it is night time and f.lux makes the yellow look like white. Now is that a bug or not?

Image mounting in UrBackup Server 2.1.x

UrBackup 2.1.x can now mount image backups. That is, it lists the image backups it has on the web interface, you can browse into them and e.g. download files or directories as a ZIP file.
I am particularly proud that this works on Linux as well as on Windows with both raw image files and VHD(z) files. On FreeBSD it only works with raw image files currently. The screenshots are from a Windows UrBackup server.

On Linux it uses libguestfs-tools to mount images in a sandboxed virtual machine. On Windows/FreeBSD mounting a hostile image may be a dangerous operation.

How I backup LVM volumes on my Xen server

I’ve got a Xen server which runs a couple of Linux and Windows VMs. The VMs are stored in LVM volumes on a LVM volume group which is on a bcache device. The bcache device consists of a mirrored SSD pair (using mdraid) as cache and a mirrored HDD pair (also using mdraid) as backing storage. The SSD caching gives a nice performance boost, but nowadays I would go with SSD storage only, because bcache caused some problems (did not play nice with udev during boot).
The Windows VMs are backed up by installing the UrBackup client in the VMs. To restore I’d need to boot the restore CD in Xen or restore the Windows images via command line in the hypervisor.
The Linux VMs are backed up at hypervisor level in the Xen dom0 (which is Debian in this case) using LVM snapshots. To create and remove LVM snapshots I have following snapshot creation and removal script (the volume group on which the volumes are is mirror-vg).

Snapshot creation script at /usr/local/etc/urbackup/create_filesystem_snapshot:

#!/bin/bash
set -e
SNAP_UID=$1
VOLNAME="$5"
VGNAME="mirror-vg"
if [[ $VOLNAME == "" ]]; then
        echo "No volume name specified"
        exit 1
fi
if [[ $VOLNAME == "other-data" ]]; then
        VGNAME="data2-vg"
fi
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
fi
export LVM_SUPPRESS_FD_WARNINGS=1
lvcreate -l100%FREE -s -n $SNAP_UID /dev/$VGNAME/$VOLNAME
SUCCESS=0
trap 'test $SUCCESS = 1 || lvremove -f /dev/$VGNAME/$SNAP_UID' EXIT
mkdir -p /mnt/urbackup_snaps/${SNAP_UID}
mount -o ro /dev/$VGNAME/$SNAP_UID /mnt/urbackup_snaps/${SNAP_UID}
SUCCESS=1
echo "SNAPSHOT=/mnt/urbackup_snaps/$SNAP_UID"
exit 0

Snapshot removal script at /usr/local/etc/urbackup/remove_filesystem_snapshot:

#!/bin/bash
set -e
SNAP_UID=$1
SNAP_MOUNTPOINT="$2"
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
fi
if [[ "$SNAP_MOUNTPOINT" == "" ]]; then
        echo "Snapshot mountpoint is empty"
        exit 1
fi
if ! test -e $SNAP_MOUNTPOINT; then
        echo "Snapshot at $SNAP_MOUNTPOINT was already removed"
        exit 0
fi
if ! df -T -P | egrep "${SNAP_MOUNTPOINT}\$" > /dev/null 2>&1; then
        echo "Snapshot is not mounted. Already removed"
        rmdir "${SNAP_MOUNTPOINT}"
        exit 0
fi
if lsblk -r --output "NAME,MOUNTPOINT" --paths > /dev/null 2>&1; then
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" --paths | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
else
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
        VOLNAME="/dev/mapper/$VOLNAME"
fi
if [ "x$VOLNAME" = x ]; then
    echo "Could not find LVM volume for mountpoint ${SNAP_MOUNTPOINT}"
    exit 1
fi
if [ ! -e "$VOLNAME" ]; then
    echo "LVM volume for mountpoint ${SNAP_MOUNTPOINT} does not exist"
    exit 1
fi
echo "Unmounting $VOLNAME at /mnt/urbackup_snaps/${SNAP_UID}..."
if ! umount /mnt/urbackup_snaps/${SNAP_UID}; then
        sleep 10
        umount /mnt/urbackup_snaps/${SNAP_UID}
fi
rmdir "${SNAP_MOUNTPOINT}"
echo "Destroying LVM snapshot $VOLNAME..."
export LVM_SUPPRESS_FD_WARNINGS=1
lvremove -f "$VOLNAME"

The snapshot scripts are specified via the file /usr/local/etc/urbackup/snapshot.cfg:

create_filesystem_snapshot=/usr/local/etc/urbackup/create_filesystem_snapshot
remove_filesystem_snapshot=/usr/local/etc/urbackup/remove_filesystem_snapshot
volumes_mounted_locally=0

Then I have a virtual client for each LVM volume that needs to be backed up. I have put those virtual clients in a settings group with the default path to backup “/|root/require_snapshot”.

For restore I need to recreate the LVM volume. Create a file system on it (e.g. with mkfs.ext4) mount it in the hypervisor and then restore via.

urbackupclientctl restore-start --virtual-client VOLUMENAME -b last –map-from / --map-to /mnt/localmountpoint

Windows Backup API support in UrBackup 2.1.x

UrBackup 2.1.x has more completselect_windows_componentse Windows Backup API support. Previously the backup API was only used to create snapshots of the specified paths to backup (volume shadow copy snapshots). Now it does a so called component level backup, if configured to do so. You can select the components to backup via the client user interface and then the selected components automatically communicate to UrBackup which files need to be backed up, and on restore it communicates where the files should be restored and if e.g. services should be restarted during/after restore.select_restore_components

This works with applications which in turn support the Windows Backup API, such as for example Microsoft Exchange, Microsoft SQL Server, Microsoft Hyper-V, Oracle DB on Windows.

Testing backup and resrestore_componentstore with those different applications is now the big item on the to-do list. Every help and pointers to applications where backup or restore is broken will be helpful.

Visual Studio 2015 runtime and MSI installer

If you are using the MSI installer to install either UrBackup Client or Server on Windows there is a potential problem you might run into.

Starting with Visual Studio 2015, with which UrBackup Client/Server 2.x are compiled, Microsoft decided to split the Visual Studio runtime into a operating system level component (Universal Runtime) and another “normal” runtime component, wheras earlier it was only a “normal” runtime component.
The operating system component is installed via Windows Update and cannot be installed by a MSI installer. With Windows 10 it is always installed, but with Windows Vista or 8.1 the system needs to be up to date in order for the system component to be present (KB2999226), otherwise UrBackup will not start.

Another work-around is to use the .EXE (NSIS) Installer which includes the operating system compontent. Installation depends on Windows Update functioning correctly (which it may not).

Let’s just say this change does not make the software developer’s and user’s life easier.

New in UrBackup 2.0.x

Wactivitieseb interface modernization. The web interface was a little bit utilitarian which gave many people the wrong impression. With the help of mombojuice the web interface was improved such that it looks much more modern and professional. Many small improvements were made as well. For example the dates are now formatted according to browser locale, backups can be started via drop-down menu and the live log of a running backup can be directly accessed from the activities screen.

Improved file deduplication. Completely reworked the file deduplication and file backup statistics calculation. This should be much faster, scalable and reliable now.

File lastmodifiedbackup improvements. File meta-data such as last modified time and file permissions are now backed up on all supported client systems (Windows, Linux, Mac OS X). Supporting more exotic file system features such as sparse files UrBackup is now a fully featured file backup solution.

File backup restore. To restore the file meta-data UrBackup has now an integrated file restore. The file restore reuses client-side hashes, if present, and transfers only differences, such that restoring folders with only few changes since the restored backup is fast.

access_backupsDirect backup access. If configured, the backed up file permissions are used to allow clients direct access to their files with only minimal configuration. On Windows there is a shortcut in Explorer which directly opens the relevant/file folder in the browser. There is a new list view which shows a file/folder in all backups. For files, hashes are used to show when the file content changed (versions).

image_backup_settingsImage backup improvements. UrBackup supports GPT formatted disks now and the restore CD boots on UEFI firmware devices (also with secure boot enabled). In combination with btrfs, UrBackup supports an incremental forever style image backup and image backups over 2TB. For VHD/VHDZ UrBackup has now settings to base incremental backups on the last or last full image backup. Full image backups can be configured to be synthetic full backups transferring only changes since the last image backup.

Significant security improvements. Forward secrecy for Internet clients via ECDH and Internet client security improvement by using AES-GCM. Switch from DSA to ECDSA for client update and server identity signatures. Web server/restore CD login now uses PBKDF2.

Mac OS X client. There is nowmac_backup_running a UrBackup Client for Mac OS X. This client is fully featured, excluding image backup (like Linux client). The Mac OS X client can be used as a technically superior backup solution to Time Machine.

Improved command line. Mainlttyy for Linux all command line usage has been significantly improved. This includes the command line client (urbackupclientctl), the server command line (urbackupsrv) and the restore client.

Linux file system snapshotting. Snapshotting now also works on Linux and is fully integrated. A portable Linux client includes snapshot scripts for LVM, dattobd and btrfs which work without changes in most cases.

Lots of other changes. Proper symbolic link backup. Virtual clients allow you to backup different sets of files at different intervals and max/min amounts. Simultaneous image and file backups. Different backup speeds and backup intervals at different times. Improved Internet transfer compression. New hashing method where the server only needs to hash changed parts of a file.

 

Can you use the last modified file time on Windows as change indicator for incremental file backups?

Short answer: No.

If your backup software is using the last modified time on Windows as the only indication if a file changed or not the backup software will miss some changed files.

The first issue is that Windows does not update the last modified time until the file is closed by the program writing to it. To work around that your program will need to track which files are open when a shadow copy is created.

The second less known issue is that last modified times are not changed at all if a file is modified by memory mapping it and then modifying it via memory mapping. This isn’t just an edge case – Windows itself creates such files, as well as e.g. Microsoft Exchange and other performance sensitive applications.

UrBackup handles both cases during incremental backups, many other backup software does not.