Because the file transfer in UrBackup 1.4 was changed, I wanted to test it with a second layer: During the file backup a Python script which goes over all files in all backup paths on the client, selects a random subset and calculates the md5-sums of this random subset. On the server those md5-sums are then verified.
Turns out file tree walking in Python on Windows has a few issues.
First of all, even though the os.walk function has a „followlinks“ parameter, this parameter does not work on Windows with Junctions. In particular it followed the compatibility junctions in the C:\Users\Username folder causing an infinite loop, till the file system complained about the path length (32k). Usually this is not an issue, because users seem to not have the permission to use those junctions, but the UrBackup client backend is running as a SYSTEM-service.
os.path.islink returns False for those junctions.
os.path.realpath does not return the target path of the junction.
To solve this I had to install the Python for Windows extensions and manually call the respective API functions. The Python for Windows extensions does not define the FILE_ATTRIBUTE_REPARSE_POINT file attribute constant. So I had to use the value explicitly. Solution:
def is_reparse_point(path):
try:
attrs = win32file.GetFileAttributesW(path)
return attrs & 1024
except:
print("Error with GetFileAttributesW with dir "+path)
return False
def get_all_files(path):
output = []
for root, dirs, files in os.walk(path):
for name in files:
output.append(os.path.join(root, name))
torm = []
for cdir in dirs:
if is_reparse_point(os.path.join(root, cdir)):
torm.append(cdir)
for cdir in torm:
dirs.remove(cdir)
return output
