memory_limit = 32M ; Maximum amount of memory a script may consume (32MB)
upload_max_filesize = 32M

--
Then, save this php.ini file in every folder that has a script in it (all the folders if you're not sure which have scripts or not). This means all sub-folders too.
--
Also create a file called, "silverstripe-cache" and include it in the domain/subdomain that you plan on installing SS. The folders you should have are the following, "assets, auth_openid, cms, jsparty, mysite, saphire, silverstripe-cache, themes, tutorial".
--
Then re-install.

The easiest way to do this is to do it on your PC/ Mac using copy and paste before uploading the folder containing the CMS up to your server. I.e. it can be a pain in the ass copy and pasting (and making sure it's the right file you're working with etc) on the server [unless you're good with shell commands and all that stuff].
--
Good Luck - I hope this helps others, because I was going mental for hours trying to find a solution.

If 32M doesn't cut it - keep increasing it until you get the sweet spot...

Shumon!