A few days ago while I was testing the Image capability of UrBackup on XenServer (another story) I noticed that Windows Server 2008 R2 creates a 100MB large Partition called “System Reserved”. On closer look a partition like this also exists on my Windows 7 (x64). Windows 7 apparently keeps some files for booting there so that it can still repair the main C: partition in case something happens to it.
One user already mentioned that such a partition exists and UrBackup should also include it into the image backups, but I couldn’t reproduce the problem till now (I asked him which Windows he was using and he didn’t answer).
In order for the partition to not be created and used you have to manually partition during the Windows setup.
As this is a major bug/limitation of UrBackup, because it causes the restored images of operating systems with this partition not to be bootable I decided to fix this problem in Server 0.24/Client 0.38 (further delaying said versions). If this partition exists it gets downloaded to the server and is associated with the normal image backup of the “C” partition. If you restore an image the “System Reserved” partition will be restored after the “C” partition, if it exists on the server thus making a restored image of the operating systems mentioned above bootable.
So UrBackup decided yesterday that it’s time for a full backup on my laptop. I have a lot of small files and just going over them and getting their file size and change date usually takes half an hour. This is one reason why the incremental backup in UrBackup is so great. Because it does not have to go over all files.
Anyways after 8 hours it was finally finished and the server was happily sorting the files into the database (much faster now – see previous entry).
Then it basically started the full backup again! It couldn’t copy the list of all transferred files to the right location because all the space on the drive where these lists are saved was used up by temporary files. Now you probably shouldn’t store your temporary files on the same drive as you save e.g. the UrBackup database (as it can fill up often). But this is nevertheless a problem, which I fixed.
Now it’s backuping again. 4 hours and going. Hopefully there will be no other error at the end.
Keep in mind that this is not a serious bug, as the full backup was complete and no data was lost and no successive errors occurred. The bug only causes the backup to be worthless for successive incremental backups (they have to backup more). It also does only occur if you have less space in the temporary folder then the files which are backed up and if your temporary folder is on the same drive as your application data. This is not a good idea anyways.
UrBackup uses SQLite to save information about your backups. The data ranges from small stuff such as settings and when a file/image backup was done and where it is saved, to a entry for every copy or in some cases linked file in a file backup. If you have a lot of files there will be a lot of entries in the last case, resulting in a very large SQLite file.
So maybe you have heard of SQLite. It is used practically everywhere right now. Firefox saves its bookmarks with it, Android phones and the IPhone their settings.
Here is a list of well-known companies which use SQLite: http://sqlite.org/famous.html
Lately SQLite got a new journalling mode which allows simultaneous reads and writes. This journalling mode is also used in stand-alone databases like PostgreSQL, thus making SQLite competitive in the concurrency area as well. This works by appending the changes to the database to a separate file. After some accumulation of changes they are applied to the database (checkpoint). Read access is blocked only during this application of changes.
Now I thought that appending the changes to that files does not require actual data to be written to disk and that because of that the write performance of SQLite is greatly improved (The appended data can be kept in system buffers until the checkpoint occurs).
There was some speed improvement but not that much. The reason for that is that SQLite is paranoid by default and enforces that every change to the database is actually written to disk after some data was saved. This is reasonable if this data is important, but in this case a backup interrupted by a power outage is worthless anyway (incomplete).
Of course you can change this default behaviour by executing:
On each connection. This will be done in the new version of both UrBackup server and client, greatly speeding up anything write related. Especially the statistics calculations are far faster now.