Thank you Willr for your helpful response - Sphinx does look very promising. I shall recommend we upgrade my current project to 2.4 when there is a stable version so that we can use it. However, as someone else has already commented in the forum, the documentation will need to be improved if it is to be understandable to any but advanced SilverStripe developers - dataObject decorators is fairly advanced stuff.
"What do you mean about configurable pagetype tables? Could you clarify that. "
What I meant is that we have a lot of pagetypes we have developed ourselves and significant site content goes in fields other than $Content, so it is stored in tables other than SiteTree. Currently these aren't included in the search, although there do seem to be some hooks in the code to allow future development to pass an array of searchable tables. Currently all you can do is remove SiteTree or Files. I would like to be able to define a list of tables(pagetypes) and preferably columns that are included in the search. Perhaps it could be set when the SiteSearchForm is instantiated in a redeveloped version of method classesToSearch.
I have also found that the documentation on the search isn't as comprehensive as it might be - there is no mention of it searching the Files table there or the public method SearchForm->classesToSearch. It would be helpful to all if it is revised.