Automatic notarization for macOS

During the effort of building the UrBackup client for macOS the problem of automatically notarizing the build came up. There is only sparse documentation and disparate sources for that. The current build script does this in pure bash now. You can see this and use it as reference here:

Building completely static Linux binaries via Android NDK

The Android NDK can be used to build completely static Linux binaries which run on any Linux.


  • Runs on every Linux distribution be it RHEL, Debian or Alpine
  • Linux (i.e. Linus Torvalds) is pretty strict about backwards compatibility so it’ll continue to run on future Linux kernels without problems
  • Theoretically it’ll even work without a Linux distribution, e.g. in a minimal Docker container
  • All the dependencies are static and confirmed to work together


  • Might use more memory at runtime since libraries (such as aren’t shared with other programs
  • Behavior might deviate from other programs in the distribution. E.g. when it doesn’t respect /etc/nsswitch.conf, parses /etc/resolv.conf differently or when OpenSSL looks for the root certificates in the wrong place

To mitigate the disadvantages the UrBackup client Linux binary installer first tries to use a glibc (non-static) build on amd64/x86_64 (the most common platform). Only if that doesn’t run (e.g. because the glibc is too old), does it fall back to the NDK build.

Previously this was done with ELLCC, but that doesn’t support C++ exceptions and doesn’t seem to get updated anymore.


Since UrBackup uses autotools to build, cross compilation is automatically present and can be used simply by setting a few environment variables before building:

export NDK=/path/to/android/ndk/android-ndk-r20
export HOST_TAG=linux-x86_64
export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/$HOST_TAG
export TARGET=x86_64-linux-android
export TARGET2=${TARGET}29
export AR=$TOOLCHAIN/bin/$TARGET-ar
export AS=$TOOLCHAIN/bin/$TARGET-as              
export CC=$TOOLCHAIN/bin/$TARGET2-clang                
export CXX=$TOOLCHAIN/bin/$TARGET2-clang++                
export LD=$TOOLCHAIN/bin/$TARGET-ld                
export RANLIB=$TOOLCHAIN/bin/$TARGET-ranlib                
export STRIP=$TOOLCHAIN/bin/$TARGET-strip
export NDK_CPUFLAGS=""
 ./configure --enable-headless --enable-c-ares --enable-embedded-cryptopp --enable-embedded-zstd LDFLAGS="-static -Wl,--gc-sections -O2 $NDK_CPUFLAGS -flto" --host $TARGET --with-zlib=$TOOLCHAIN/sysroot/usr --with-crypto-prefix=$TOOLCHAIN/sysroot/usr --with-openssl=$TOOLCHAIN/sysroot/usr CPPFLAGS="-DURB_THREAD_STACKSIZE64=8388608 -DURB_THREAD_STACKSIZE32=1048576 -DURB_WITH_CLIENTUPDATE -ffunction-sections -fdata-sections -ggdb -O2 -flto $ARCH_CPPFLAGS" CFLAGS="-ggdb -O2 -flto $NDK_CPUFLAGS" CXXFLAGS="-ggdb -O2 -flto $NDK_CPUFLAGS -I$NDK/sources/android/cpufeatures/ -DOPENSSL_SEARCH_CA" LIBS="-ldl" 

See also the script that builds the Linux client installer.

The advantage UrBackup has here, is that many dependencies are already bundled with the source code like crypto++, zstd, lua and sqlite. All the dependencies that are not bundled need to be compiled to a static library (for every architecture) and in my case I have put them into $TOOLCHAIN/sysroot/usr/.


The Android NDK is of course made to build programs for Android. There are significant differences between Android and other Linux distributions. Here is two I found:

If one wants to resolve a DNS name such as to an IP address one usually uses getaddrinfo(). This won’t work with the Android NDK libc (bionic libc), because it uses the Android resolver by calling some Android runtime java code that is obviously not present on non-Android systems. The solution for this problem was to use c-ares instead of the libc to resolve addresses. If you are using a library that resolves addresses that needs to have the option of using c-ares as well (such as cURL).

A call to system() or popen(), calls the shell (usually /bin/sh). The Android libc, however, calls /system/bin/sh instead, which is of course not present on non-Android distributions. The solution was to replace all those calls to an own version. Again, if any library one uses does use those, they’ll need to be replaced.

On x86 and amd64/x86_64 the Android NDK automatically uses SSE4 CPU instructions which older CPUs do not support. Users complained about that and that client should run on as many systems as possible.
To disable SSE4 the Android NDK compiler needs to be passed “-mno-sse4a -mno-sse4.1 -mno-sse4.2 -mno-popcnt” (that was the only way I found). The problem is that some SSE4 instructions are in the libc and libc++. So, the libc and libc++ need to be recompiled with those flags. The bionic libc source code is (unfortunately) NOT part of the Android NDK source code, it is part of the Android source code.
So after downloading 50GB of Android source code for half a day, one needs to change the Go source code of Androids custom build tool (soong). Adjust e.g. build/soong/cc/config/x86_64_device.go, select the correct architecture to build, then fish out the libc.a (and libc++.a) from the output directory and replace the more then dozens of libc.a occurrences in the NDK (no idea which one it actually uses).

One final complication was that crypto++ does actually feature test and then use SSE4 instructions, so they need to be enabled for some crypto++ compile units. If one specifies both “-mno-sse4” and “-msse4”, “-mno-sse4” seems to take precedence. So the solution was to have a compiler wrapper script that removes “-mno-sse4” in such a case.

UrBackup Windows Installer creator

If you wanted to create a generic UrBackup client installer for your internet server you’d have to create one yourself following instructions here .

I have made to make this process easier. Given the correct parameters it’ll create a executable for you that’ll create a client on your server, then download the client installer for this newly created client and then run this installer. It’ll use the computer name as default client name, optionally appending a random string, such that name collisions do not occur.

Skewed Color

In which I debug why the tray icon on my laptop doesn’t turn yellow during backups anymore and just cannot find the issue:

Turns out I have f.lux on and it is night time and f.lux makes the yellow look like white. Now is that a bug or not?

How I backup LVM volumes on my Xen server

I’ve got a Xen server which runs a couple of Linux and Windows VMs. The VMs are stored in LVM volumes on a LVM volume group which is on a bcache device. The bcache device consists of a mirrored SSD pair (using mdraid) as cache and a mirrored HDD pair (also using mdraid) as backing storage. The SSD caching gives a nice performance boost, but nowadays I would go with SSD storage only, because bcache caused some problems (did not play nice with udev during boot).
The Windows VMs are backed up by installing the UrBackup client in the VMs. To restore I’d need to boot the restore CD in Xen or restore the Windows images via command line in the hypervisor.
The Linux VMs are backed up at hypervisor level in the Xen dom0 (which is Debian in this case) using LVM snapshots. To create and remove LVM snapshots I have following snapshot creation and removal script (the volume group on which the volumes are is mirror-vg).

Snapshot creation script at /usr/local/etc/urbackup/create_filesystem_snapshot:

set -e
if [[ $VOLNAME == "" ]]; then
        echo "No volume name specified"
        exit 1
if [[ $VOLNAME == "other-data" ]]; then
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
lvcreate -l100%FREE -s -n $SNAP_UID /dev/$VGNAME/$VOLNAME
trap 'test $SUCCESS = 1 || lvremove -f /dev/$VGNAME/$SNAP_UID' EXIT
mkdir -p /mnt/urbackup_snaps/${SNAP_UID}
mount -o ro /dev/$VGNAME/$SNAP_UID /mnt/urbackup_snaps/${SNAP_UID}
echo "SNAPSHOT=/mnt/urbackup_snaps/$SNAP_UID"
exit 0

Snapshot removal script at /usr/local/etc/urbackup/remove_filesystem_snapshot:

set -e
if [[ $SNAP_UID == "" ]]; then
        echo "No snapshot uid specified"
        exit 1
if [[ "$SNAP_MOUNTPOINT" == "" ]]; then
        echo "Snapshot mountpoint is empty"
        exit 1
if ! test -e $SNAP_MOUNTPOINT; then
        echo "Snapshot at $SNAP_MOUNTPOINT was already removed"
        exit 0
if ! df -T -P | egrep "${SNAP_MOUNTPOINT}\$" > /dev/null 2>&1; then
        echo "Snapshot is not mounted. Already removed"
        rmdir "${SNAP_MOUNTPOINT}"
        exit 0
if lsblk -r --output "NAME,MOUNTPOINT" --paths > /dev/null 2>&1; then
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" --paths | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
        VOLNAME=`lsblk -r --output "NAME,MOUNTPOINT" | egrep " ${SNAP_MOUNTPOINT}\$" | head -n 1 | tr -s " " | cut -d" " -f1`
if [ "x$VOLNAME" = x ]; then
    echo "Could not find LVM volume for mountpoint ${SNAP_MOUNTPOINT}"
    exit 1
if [ ! -e "$VOLNAME" ]; then
    echo "LVM volume for mountpoint ${SNAP_MOUNTPOINT} does not exist"
    exit 1
echo "Unmounting $VOLNAME at /mnt/urbackup_snaps/${SNAP_UID}..."
if ! umount /mnt/urbackup_snaps/${SNAP_UID}; then
        sleep 10
        umount /mnt/urbackup_snaps/${SNAP_UID}
echo "Destroying LVM snapshot $VOLNAME..."
lvremove -f "$VOLNAME"

The snapshot scripts are specified via the file /usr/local/etc/urbackup/snapshot.cfg:


Then I have a virtual client for each LVM volume that needs to be backed up. I have put those virtual clients in a settings group with the default path to backup “/|root/require_snapshot”.

For restore I need to recreate the LVM volume. Create a file system on it (e.g. with mkfs.ext4) mount it in the hypervisor and then restore via.

urbackupclientctl restore-start --virtual-client VOLUMENAME -b last –map-from / --map-to /mnt/localmountpoint

Windows Backup API support in UrBackup 2.1.x

UrBackup 2.1.x has more completselect_windows_componentse Windows Backup API support. Previously the backup API was only used to create snapshots of the specified paths to backup (volume shadow copy snapshots). Now it does a so called component level backup, if configured to do so. You can select the components to backup via the client user interface and then the selected components automatically communicate to UrBackup which files need to be backed up, and on restore it communicates where the files should be restored and if e.g. services should be restarted during/after restore.select_restore_components

This works with applications which in turn support the Windows Backup API, such as for example Microsoft Exchange, Microsoft SQL Server, Microsoft Hyper-V, Oracle DB on Windows.

Testing backup and resrestore_componentstore with those different applications is now the big item on the to-do list. Every help and pointers to applications where backup or restore is broken will be helpful.

Visual Studio 2015 runtime and MSI installer

If you are using the MSI installer to install either UrBackup Client or Server on Windows there is a potential problem you might run into.

Starting with Visual Studio 2015, with which UrBackup Client/Server 2.x are compiled, Microsoft decided to split the Visual Studio runtime into a operating system level component (Universal Runtime) and another “normal” runtime component, wheras earlier it was only a “normal” runtime component.
The operating system component is installed via Windows Update and cannot be installed by a MSI installer. With Windows 10 it is always installed, but with Windows Vista or 8.1 the system needs to be up to date in order for the system component to be present (KB2999226), otherwise UrBackup will not start.

Another work-around is to use the .EXE (NSIS) Installer which includes the operating system compontent. Installation depends on Windows Update functioning correctly (which it may not).

Let’s just say this change does not make the software developer’s and user’s life easier.

New in UrBackup 2.0.x

Wactivitieseb interface modernization. The web interface was a little bit utilitarian which gave many people the wrong impression. With the help of mombojuice the web interface was improved such that it looks much more modern and professional. Many small improvements were made as well. For example the dates are now formatted according to browser locale, backups can be started via drop-down menu and the live log of a running backup can be directly accessed from the activities screen.

Improved file deduplication. Completely reworked the file deduplication and file backup statistics calculation. This should be much faster, scalable and reliable now.

File lastmodifiedbackup improvements. File meta-data such as last modified time and file permissions are now backed up on all supported client systems (Windows, Linux, Mac OS X). Supporting more exotic file system features such as sparse files UrBackup is now a fully featured file backup solution.

File backup restore. To restore the file meta-data UrBackup has now an integrated file restore. The file restore reuses client-side hashes, if present, and transfers only differences, such that restoring folders with only few changes since the restored backup is fast.

access_backupsDirect backup access. If configured, the backed up file permissions are used to allow clients direct access to their files with only minimal configuration. On Windows there is a shortcut in Explorer which directly opens the relevant/file folder in the browser. There is a new list view which shows a file/folder in all backups. For files, hashes are used to show when the file content changed (versions).

image_backup_settingsImage backup improvements. UrBackup supports GPT formatted disks now and the restore CD boots on UEFI firmware devices (also with secure boot enabled). In combination with btrfs, UrBackup supports an incremental forever style image backup and image backups over 2TB. For VHD/VHDZ UrBackup has now settings to base incremental backups on the last or last full image backup. Full image backups can be configured to be synthetic full backups transferring only changes since the last image backup.

Significant security improvements. Forward secrecy for Internet clients via ECDH and Internet client security improvement by using AES-GCM. Switch from DSA to ECDSA for client update and server identity signatures. Web server/restore CD login now uses PBKDF2.

Mac OS X client. There is nowmac_backup_running a UrBackup Client for Mac OS X. This client is fully featured, excluding image backup (like Linux client). The Mac OS X client can be used as a technically superior backup solution to Time Machine.

Improved command line. Mainlttyy for Linux all command line usage has been significantly improved. This includes the command line client (urbackupclientctl), the server command line (urbackupsrv) and the restore client.

Linux file system snapshotting. Snapshotting now also works on Linux and is fully integrated. A portable Linux client includes snapshot scripts for LVM, dattobd and btrfs which work without changes in most cases.

Lots of other changes. Proper symbolic link backup. Virtual clients allow you to backup different sets of files at different intervals and max/min amounts. Simultaneous image and file backups. Different backup speeds and backup intervals at different times. Improved Internet transfer compression. New hashing method where the server only needs to hash changed parts of a file.


Start of UrBackup 2.0 beta phase

UrBackup Server/Client/Restore 2.0.0 beta was recently released. This marks the beginning of the UrBackup 2.0 beta phase.

See the forums for download links and discussion.

UrBackup 2.0 marks the beginning of UrBackup having no major limitations. If you still find some please start a discussion in the forums. The next beta version will also properly support sparse file backup.

Progress update on the next major UrBackup version

Currently the next major UrBackup version is kind of close to getting finished. There are a few major work in progress areas. Once they are finished and I have done some overall testing I will release a beta version.

The major changes in the new version are:

  • Completely reworked the file deduplication and file backup statistics calculation. This should be much faster, scalable and reliable now.
  • The Copy-on-Write image backups on btrfs mentioned in the last post. Synthetic full backups for the VHD/VHDZ file format and settings for basing image backups on the last full or last incremental backup (differential/incremental).
  • File backups include file metadata including file modification time, ACLs, alternate data streams etc.
  • Backup of streaming data. E.g. the output of “mysqldump”/”pg_dump”. I plan to add basic backup scripts for popular Open Source databases to the client
  • New file restore feature which restores file backups and properly restores the file meta-data
  • The ACLs/file permissions are used to enable users to directly access backups on the web interface from the explorer on the clients (via right click -> Access/Restore backups)
  • Proper backups of symbolic links. Symbolic links which point to folders/files which are backed up are backed up as symbolic links and symbolic links which point outside of the selected backup set are followed/not followed depending on a setting
  • The web interface has been bootstrapified ( – mombojuice did the work) and looks much more modern now
  • Simultaneous image and file backups
  • Backup and restore EFI boot sector and partition on UEFI systems. Restore CD that boots with UEFI firmware
  • Client for Mac OS X
  • Forward secrecy for Internet clients via ECDH and Internet client security improvement by using AES-GCM
  • Switch from DSA to ECDSA for client update and server identity signatures

Still to do:

  • Lot’s of testing and bug fixing
  • Backup and restore of file meta-data on Mac OS X and Linux
  • Symbolic link backup handling on Mac OS X and Linux
  • Automatic client update for Mac OS X like for Windows
  • Restoring files which are in use on Windows (via restarting)
  • Update Documentation

Remaining UrBackup limitations (to be done after with a subsequent version):

  • Recognize hard links and backup the files only once
  • Backup only used areas of sparse files
  • Continuous file backup

Once the “to do”s are done this is a big step forward for UrBackup. Having streaming file backups and incremental, differential, synthetic full and full image backups basically allows you to implement pretty much every backup strategy with UrBackup. The only thing missing is the continuous file backup and I already started work on that.

For example you could use UrBackup instead of TimeMachine on Mac OS X and do a full system restore via the file restore feature (this is not implemented at all – it is just an example what it might be able to do). You probably don’t even need an image of your Windows system partition but can restore it via the file backup restore (albeit inefficiently, because the hard links in C:\windows\winsxs are not handled properly).