This is how I got Subsites working in Plesk 11.5 with SilverStripe 3.2.
1. Set up primary domain
2. Add subsite domains and/or subdomains in same subscription as primary
3. Set 'Document Root' to same folder as primary domain
4. For each subsite add lines similar to the following to the .htaccess for the primary domain
# subsitedomain.tld Subsite
RewriteCond %{REQUEST_URI} ^(.*)$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{HTTP_HOST} ^subsitedomain.tld$
RewriteRule .* framework/main.php?url=%{REQUEST_URI}&SubsiteID=<thissubsiteID> [QSA]
These are the lines at the bottom of my .htaccess for a test with one primary domain and two subsites.
# Process through SilverStripe if no file with the requested name exists.
# Pass through the original path as a query parameter, and retain the existing parameters.
# Subdomain subsite
RewriteCond %{REQUEST_URI} ^(.*)$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{HTTP_HOST} ^subtest.primary.tld$
RewriteRule .* framework/main.php?url=%{REQUEST_URI}&SubsiteID=1 [QSA]
# subsitedomain.tld Subsite
RewriteCond %{REQUEST_URI} ^(.*)$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{HTTP_HOST} ^subsitedomain.tld$
RewriteRule .* framework/main.php?url=%{REQUEST_URI}&SubsiteID=2 [QSA]
# primary.tld Primary Domain
RewriteCond %{REQUEST_URI} ^(.*)$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule .* framework/main.php?url=%1 [QSA]
Thanks to other posters in this thread.
Note: Home pages for Subsites must have the URL Segment 'home'.
See - http://www.silverstripe.org/community/forums/general-questions/show/96795#post412728