I finally bit the bullet and worked on MSI installers for Windows. As anticipated it was not easy. I used WiX.
They do have some advantages over an installer distributed as “exe”:
- One can add the Microsoft Visual Studio runtime as a “merge module” thus avoiding starting it in the installer manually
- Apparently installing centralized on domain computers is easier
On the negative side:
- No shared 32/64 bit MSIs are possible. That means the user has to select the right one before downloading
- You cannot add custom commands as easily as in NSIS
I think I will only publish 64bit MSIs for now. Most Windows Servers should be 64bit now anyway and I will still publish the “old” installers for users of older and 32bit systems.
I recently added a setting to configure which volume(s) UrBackup should backup as image. I added this because it was requested in the forums. Feature requests work! Try them as well!
I will now continue on to modifying the restore CD to allow one to select the volume to restore. This will not be much work, so expect it soon with UrBackup Client 0.39, Server 0.25 and Restore CD 0.3.
Nevertheless I felt a little bit uneasy while adding this feature, because it is not really necessary in my opinion.
Incremental file backups are usually really fast. Incremental image backups currently not so much. So you should use file backups whenever possible. As additional benefit restoring only part of the data is much easier with file backups than with image backups, where you have to mount the image. If you want to be able to do a bare metal restore though and you use Windows having an image backup of your system volume is unavoidable. Restoring from files is not possible without reinstalling.
I will talk about why you do not need image backups at all if you use Linux soon.
So because file backups are faster and more convenient and you only need an image backup of your system volume backing up any other volume as image backup is suboptimal.
But now UrBackup does not stop you from doing it. It is your choice now.
A few days ago while I was testing the Image capability of UrBackup on XenServer (another story) I noticed that Windows Server 2008 R2 creates a 100MB large Partition called “System Reserved”. On closer look a partition like this also exists on my Windows 7 (x64). Windows 7 apparently keeps some files for booting there so that it can still repair the main C: partition in case something happens to it.
One user already mentioned that such a partition exists and UrBackup should also include it into the image backups, but I couldn’t reproduce the problem till now (I asked him which Windows he was using and he didn’t answer).
In order for the partition to not be created and used you have to manually partition during the Windows setup.
As this is a major bug/limitation of UrBackup, because it causes the restored images of operating systems with this partition not to be bootable I decided to fix this problem in Server 0.24/Client 0.38 (further delaying said versions). If this partition exists it gets downloaded to the server and is associated with the normal image backup of the “C” partition. If you restore an image the “System Reserved” partition will be restored after the “C” partition, if it exists on the server thus making a restored image of the operating systems mentioned above bootable.
UrBackup uses SQLite to save information about your backups. The data ranges from small stuff such as settings and when a file/image backup was done and where it is saved, to a entry for every copy or in some cases linked file in a file backup. If you have a lot of files there will be a lot of entries in the last case, resulting in a very large SQLite file.
So maybe you have heard of SQLite. It is used practically everywhere right now. Firefox saves its bookmarks with it, Android phones and the IPhone their settings.
Here is a list of well-known companies which use SQLite: http://sqlite.org/famous.html
Lately SQLite got a new journalling mode which allows simultaneous reads and writes. This journalling mode is also used in stand-alone databases like PostgreSQL, thus making SQLite competitive in the concurrency area as well. This works by appending the changes to the database to a separate file. After some accumulation of changes they are applied to the database (checkpoint). Read access is blocked only during this application of changes.
Now I thought that appending the changes to that files does not require actual data to be written to disk and that because of that the write performance of SQLite is greatly improved (The appended data can be kept in system buffers until the checkpoint occurs).
There was some speed improvement but not that much. The reason for that is that SQLite is paranoid by default and enforces that every change to the database is actually written to disk after some data was saved. This is reasonable if this data is important, but in this case a backup interrupted by a power outage is worthless anyway (incomplete).
Of course you can change this default behaviour by executing:
On each connection. This will be done in the new version of both UrBackup server and client, greatly speeding up anything write related. Especially the statistics calculations are far faster now.
I just noticed that the new Unstable UrBackup client crashed on Windows XP. Looking at the logfile did not give any indication where the error came from. I could however say where it did not come from, because only one thread crashed and the other ones continued writing debug info into the logfile.
To get some more info where this error was happening I installed userdump on the XP laptop (http://support.microsoft.com/kb/241215). This gave me a nice image of the process memory at the point of failure. Sadly Visual Studio cannot display that much information about the dump file. For example I did not get it to display a call stack. I went on to install WinDbg which is part of the Windows SDK. This had all the information needed to pinpoint the problem. It showed a call stack including line numbers in the source files. It was however mysterious how it got the location of the source files and the line number of the error, because of course I used a release build on the XP laptop and release versions do not include debug information. Strange.
Even though it could display everything quite fine WinDbg complained about missing debug information. Which is, as explained, only natural. But then why could it show the call stack with function names?
Analyzing the information WinDbg provided did not help: The error was at a position where normally no error should occur. I double checked.
So whatever magic WinDbg does it must be wrong. Right? I continued to point WinDbg at the right debug information, but that did not change the position of the error in the code. I was just in the process of collecting all the right DLLs to get a debug build to run on the XP laptop, when the day saving thing happened: I cleaned the whole project and build everything. The universal advise for every problem computer related: “Have you tried turning it off and on again?”. Of course it worked perfectly after that.
Visual Studio must have done something wrong in the calculation of what it has to rebuild, causing the XP build target to be outdated, running with a more recent host process. This caused a function to be called without a parameter which now has one, which then caused the memory access error.
Once again the solution was easy but finding the solution was hard.