If you're comfortable editing the PHP source of of SilverStripe, you could try renaming the HTTPResponse class to something like SSHTTPResponse, and searching & replacing the rest of the code for references to HTTPResponse, and updating those too. You would also need to rename HTTPResponse.php to SSHTTPResponse.php.
But if this is your first time using SilverStripe, this might wind up being more hassle than it's worth. You could also ask your host if they can remove the custom HTTP extension to PHP that's discussed here: http://nz.php.net/manual/en/book.http.php
I used notepad++ and opened up all the files in sapphire\core and sapphire\core\control - then used the "find in all files" to replace all instances of HTTPResponse with SSHTTPResponse, I also renamed the HTTPResponse.php found in sapphire\core\control to SSHTTPResopnse.php and everything is now working fine.
This may be related to the mod_rewrite AllowOverride issue that some people seem to be having on install.
2.3 didn't install for me and kept getting stuck on Friendly Urls not working. It turns out that i needed to replace all instances of HTTPResponse and HTTPRequest in the code with SSHTTPResponse, and HTTPRequest (can do with notepad++ automatically) then rename the files in core\control to (SSHTTPResponse.php, and HTTPRequest.php) and it installed nice and clean.
i have this problem too, and i asked the host to switch off the PHP extension "http". If it works afterwards i will report here! I hope this will be fixed soon, and i wonder why this issue isn't a problem for more people around...