Although this bug is apparently slated for fixing in 2.4.0 (see Ticket #2196) the same problem with redirecting after installation seems to occur regardless of browser, not just in Safari (I should say I encountered on Firefox 3 (for Mac OSX), and when I loaded up safari (from which I did not do the install) the redirect worked ok). The ?flush=1 trick did not help. However, force quitting Firefox and then reloading the page with ?flush=1 appended did work (e.g.: http://localhost/silverstripe/?flush=1).
A related and similar issue is that, after following the vanilla install procedure, when I point the browser to http://localhost/silverstripe without the trailing slash, I get a directory index. That can be fixed with a simple change to the .htaccess to prohibit directory indexing--however, that would give an error. Another solution would be by replacing an index.php with a meta-refresh header to http://localhost/silverstripe/ with the trailing slash. A better solution would be to rewrite the .htaccess rewrite engine commands to route attempts without a trailing slash correctly.
Obviously allowing unwanted directory indexing after following your vanilla install instructions is a possible security issue and should be remedied, imho, asap.
For the revised installation procedures, I would suggest having the install files in a subdirectory as done with wordpress or joomla.