My webhost (Mosso) has a system folder in trhe root the site that can't be removed or read. There's an issue with the installer where it tries to crawl all of the folders in silverstripe installation folder.
I managed to get around this (with some help on the IRC channel) by first installing in a sub-folder, then moving the installation back to root using FTP. I had to modify the .htaccess file to point to the root, but it's working fine now.
Just thought I'd post this in case someone ran into the same issue, or in the hopes that the installer might be adjusted to take this kind of situation into account ;-)