Python on Windows issues

Because the file transfer in UrBackup 1.4 was changed, I wanted to test it with a second layer: During the file backup a Python script which goes over all files in all backup paths on the client, selects a random subset and calculates the md5-sums of this random subset. On the server those md5-sums are then verified.

Turns out file tree walking in Python on Windows has a few issues.

First of all, even though the os.walk function has a „followlinks“ parameter, this parameter does not work on Windows with Junctions. In particular it followed the compatibility junctions in the C:\Users\Username folder causing an infinite loop, till the file system complained about the path length (32k). Usually this is not an issue, because users seem to not have the permission to use those junctions, but the UrBackup client backend is running as a SYSTEM-service.

os.path.islink returns False for those junctions.

os.path.realpath does not return the target path of the junction.

To solve this I had to install the Python for Windows extensions and manually call the respective API functions. The Python for Windows extensions does not define the FILE_ATTRIBUTE_REPARSE_POINT file attribute constant. So I had to use the value explicitly. Solution:

def is_reparse_point(path):

    try:
        attrs = win32file.GetFileAttributesW(path)

        return attrs & 1024

    except:
        print("Error with GetFileAttributesW with dir "+path)
        return False

def get_all_files(path):

    output = []

    for root, dirs, files in os.walk(path):
        for name in files:
            output.append(os.path.join(root, name))

        torm = []
        for cdir in dirs:
            if is_reparse_point(os.path.join(root, cdir)):
                torm.append(cdir)

        for cdir in torm:
            dirs.remove(cdir)

    return output

 

New Features in UrBackup Server 1.3

It has been a while and there are now a lot new features in UrBackup Server 1.3.

Users of the web interface can download a Client specific installer directly from the server now. The installer has the UrBackup Server information embedded, such that the client automatically connects to the server once it is installed. I’ve also published a script that connects to a UrBackup server, creates a client named like the local computer and then downloads and executes the client installer. This enables a one click setup experience for Internet clients.

The new live log lets you see what the UrBackup server is currently busy with. You can either see all debug level log messages or client specific log messages.

 

 

You can see which files the UrBackup Server is currently working on and the usual log messages which you can also view afterwards via web interface or on the client.

 

A “hidden” feature is now accessible via web interface: You can disable any type of backup for any client.


More fine grained permissions
for the client allow you to prevent the users from starting full file backups, but still allow incremental file backups.

 

The soft client quota allows you to limit the amount of storage each client can use. During the nightly cleanup UrBackup deletes the client’s backups until the storage usage is within the bounds prescribed by the soft client quota. Other than a percentage value you can also use something like “20G” as soft client quota.


You can now have separate backup windows for incremental/full file/image backups.

 


Client-side file hashes
prevent the re-transfer of files that are already on the server, e.g., because another client has the same file. In some situations this drastically reduces the bandwidth requirements and speeds up file backups over Internet.


If you have performance problems with file backups the new file entry cache may help you. If the file entry cache is enabled file entries (a mapping from file hash to file paths) are cached in a separate database which may speed up backups. The caches are automatically created and destroyed if this setting is changed (and the server restarted), but creation may take a long time. LMDB makes heavy use of memory mapped files and is therefore only advisable on a 64bit operating system. It does also create a very large sparse file on Windows. When in doubt use the SQLite cache.

 

 

Starting a process for all currently logged in users

For silently updating the client I needed to

  1. Kill all the client GUIs (tray icons) of all users
  2. Stop the background service; Update the files and executables; Start the new background service
  3. Start the client GUI for all currently logged in users

The silent update is done by simply calling the normal NSIS installer with a parameter that makes it non-interactive. Information about how step one is handled is in the previous post. Step two was already in the Installer. Step three remained to be solved.

For that I created a NSIS plugin which checks if the current user is the SYSTEM user. If yes, it starts the executable for all currently logged in users. If not it returns zero. It is used like this:

StrCpy $0 "$INSTDIRUrBackupClient.exe"
startplugin::start
Pop $0
${If} $0 == '0'
Exec "$INSTDIRUrBackupClient.exe"
${EndIf}

Source code is here: http://download.urbackup.org/startplugin.zip
Binary is here (for NSIS Unicode): http://download.urbackup.org/startplugin_bin.zip

Killing 64bit processes from a 32bit NSIS installer

There is no 64bit version of the NSIS (Nullsoft Scriptable Install System). 64Bit processes can only be terminated from 64bit processes, so all the available NSIS plugins, such as the KillProc plugin, do not work. Additionally they do not kill the processes of all users – only those of the currently logged in users – which I wanted as well.

The problem is easily solved by packing an executable into the Installer which is then extracted and called. It is used like this in the UrBackup installer:

${If} ${RunningX64}
File "data_x64KillProc.exe"
nsExec::Exec '"$INSTDIRKillProc.exe" UrBackupClient.exe'
${Else}
File "dataKillProc.exe"
nsExec::Exec '"$INSTDIRKillProc.exe" UrBackupClient.exe'
${EndIf}

The source code of that executable is available here: http://download.urbackup.org/KillProc_src.zip
The binary files here: http://download.urbackup.org/KillProc_bin.zip
It expects only one parameter: The name of the running process to kill.

Internet Mode

Currently I’m working on a new internet mode for UrBackup. This means that you will be able to backup clients to a server on the internet with the upcoming new version.
This communication is of course encrypted and authenticated. It uses a shared key encryption with AES256 in CFB mode. It should be easy to configure: You just need to supply the server with its internet name/IP and the ports the clients should connect to. These settings, as well as random keys, are then pushed to the clients via the local (trusted) network. They can be manually entered on the client side as well. Then the key is pushed from the client to the server.

If you are not in the local network the client tries to connect to the internet server, if you entered something (e.g. a dns name or IP address) there. Then both check if they have the same key and if they do have the same shared key a normal connection, like if the client were in the local network, is established and backups can be performed.

I’ll now implement special options for disabling image and full file backups for clients connected via internet. Then I will implement a special, block based file (rsync like) transfer mode which will be used for those clients and which transfers less data in some scenarios.

Then you can look forward to backup archival and more detailed backup retention capabilities, which I’ll be working on next.

Latest multiple volumes image backup feature and why you should not use it

I recently added a setting to configure which volume(s) UrBackup should backup as image. I added this because it was requested in the forums. Feature requests work! Try them as well!

I will now continue on to modifying the restore CD to allow one to select the volume to restore. This will not be much work, so expect it soon with UrBackup Client 0.39, Server 0.25 and Restore CD 0.3.

Nevertheless I felt a little bit uneasy while adding this feature, because it is not really necessary in my opinion.

Incremental file backups are usually really fast. Incremental image backups currently not so much. So you should use file backups whenever possible. As additional benefit restoring only part of the data is much easier with file backups than with image backups, where you have to mount the image. If you want to be able to do a bare metal restore though and you use Windows having an image backup of your system volume is unavoidable. Restoring from files is not possible without reinstalling.

I will talk about why you do not need image backups at all if you use Linux soon.

So because file backups are faster and more convenient and you only need an image backup of your system volume backing up any other volume as image backup is suboptimal.

But now UrBackup does not stop you from doing it. It is your choice now.

UrBackup Client Unstable crashing on Windows XP

I just noticed that the new Unstable UrBackup client crashed on Windows XP. Looking at the logfile did not give any indication where the error came from. I could however say where it did not come from, because only one thread crashed and the other ones continued writing debug info into the logfile.

To get some more info where this error was happening I installed userdump on the XP laptop (http://support.microsoft.com/kb/241215). This gave me a nice image of the process memory at the point of failure. Sadly Visual Studio cannot display that much information about the dump file. For example I did not get it to display a call stack. I went on to install WinDbg which is part of the Windows SDK. This had all the information needed to pinpoint the problem. It showed a call stack including line numbers in the source files. It was however mysterious how it got the location of the source files and the line number of the error, because of course I used a release build on the XP laptop and release versions do not include debug information. Strange.

Even though it could display everything quite fine WinDbg complained about missing debug information. Which is, as explained, only natural. But then why could it show the call stack with function names?

Analyzing the information WinDbg provided did not help: The error was at a position where normally no error should occur. I double checked.

So whatever magic WinDbg does it must be wrong. Right? I continued to point WinDbg at the right debug information, but that did not change the position of the error in the code. I was just in the process of collecting all the right DLLs to get a debug build to run on the XP laptop, when the day saving thing happened: I cleaned the whole project and build everything. The universal advise for every problem computer related: “Have you tried turning it off and on again?”. Of course it worked perfectly after that.

Visual Studio must have done something wrong in the calculation of what it has to rebuild, causing the XP build target to be outdated, running with a more recent host process. This caused a function to be called without a parameter which now has one, which then caused the memory access error.

Once again the solution was easy but finding the solution was hard.