Hi Aram. Sorry for the delay in getting back.

So you are getting an issue with mysql connection failure?

One thing you could try is setting the query mode to xmlpipes. When you do this, the search daemon doesn't attempt the connection itself, so you eliminate that. Instead, it calls a sapphire controller to get an XML structure that it indexes instead. An advantage of this is that you can run that controller yourself, and see what data is being fed through. To see the command to issue, switch to xmlpipes mode (just set the $sphinx static, setting "mode" to "xmlpipe") and dev/build. If you look in sphinx.conf, it will have different source sections, which contain a command line that sphinx will execute to get the xml. You can run this command yourself. This can help to identify if the right data is being indexed, or if its an indexer configuration problem.

I've used sphinx on a few sites now. The external environment is the fiddly part, getting permissions and paths right. I tend towards using xmlpipes by default, and add in the messagequeue module so offload delta reindexing from the user request.

Mark

Hi Mark, thanks for getting back to me.

I am not entirely sure what the issue is, it looks from the diagnosis like it is connecting to the DB fine and it says the reindex is 'OK' too, but when I click search I get the following error:

[User Error] no enabled local indexes to search

I have set all the files in the tmp dir to 0777 so it shouldn't have any issues with permissions, but to be fair I am not an expert when it comes to root server stuff.

Also perhaps I am not setting up the decorators correctly? in the docs it says to decorate from within the class (using $extensions), however it also says you have to decorate SiteTree and File to make it work, so I have done that via _config.php using:

Object::add_extension('SiteTree', 'SphinxSearchable');
Object::add_extension('File', 'SphinxSearchable');

Then in my dataobject I have:

	static $extensions = array(
         'SphinxSearchable'
    );
	
	static $sphinx = array(
		"search_fields" => array("Title", "Description"),
		"filter_fields" => array("Title", "Description"),
		"sort_fields" => array("Title"),
		"mode" => "xmlpipe",
	);		

I am sure I am missing something, just not sure what as all seems to be reported as OK until I actually search.

It looks like it is a super useful module and would do everything I need so would be great if I could get it working :)

Thanks again

Aram

Hi.

Extensions can be added using either of the two syntaxes; they are equivalent. Setting the $extensions static in the class is sometimes considered better practice, because the class is then self-contained. The other syntax is required to provide a way to decorate classes that you don't control, most particularly built-in classes like SiteTree and File. I think what you've got there should be OK. You don't have to decorate SiteTree or File, only if you want to use sphinx to index and search those classes.

One thing I would recommend changing is taking Title and Description out of filter_fields. Filter fields (called attributes in sphinx documentation) should be integer or integer-equivalent only, which is done for performance; sphinx searchd keeps the document IDs in memory, as well as the attributes (but not the text), which part of why it's so fast). You'll need to dev/build after any such change to the sphinx statics.

If that doesn't fix it, we need to understand the point of failure. Currently the module diagnosis isn't sophisticated enough to determine this automatically. With xmlpipes, you can run the command the indexer uses to get it's data, and see what its being fed, which may provide some indication.

In your sphinx.conf, there should be a whole bunch of sections (e.g. "source PageSrc"). If you find the source for your data object, you should see a line like this:

xmlpipe_command = /Users/mstephens/Sites/sphinx_dev/sapphire/sake sphinxxmlsource/MyDataObjectClass

(Note there will be at least sources for each class, Foo and FooDelta. If the data object is extended with versioning, it will have Live variants as well, giving four indexes. Try the non-Delta, non-Live source.)

Copy this command and run it, and you'll see an XML stream. You should see a representation of your data objects here. Is this providing what you expect?

Mark

Hi Mark, thank for the continued help with this.

I removed the fields from filter_fields and replaced them with 'ID', which didn't make a difference so I then tried removing it all together but again no change.

So I then ran the command you suggested from the SHELL and it seemed to list everything I would expect, each field on product and their values. Is it normal that it would be making tag for every field? It also looked from the sphinx.conf file like every single class was getting xml as they each had sources and indexs set, is that normal even though I have only decorated SiteTree, File and Product? For example BlogHolder had:

source BlogHolderSrc : BaseSrc {
	type = xmlpipe2
	xmlpipe_command = /home/aabweb/public_html/ukc/sapphire/sake sphinxxmlsource/BlogHolder
}
index BlogHolder : BaseIdx {
	source = BlogHolderSrc
	path = /tmp/silverstripe-cache-home-aabweb-public_html-ukc/sphinx/idxs/BlogHolder
}

source BlogHolderLiveSrc : BaseSrc {
	type = xmlpipe2
	xmlpipe_command = /home/aabweb/public_html/ukc/sapphire/sake sphinxxmlsource/BlogHolderLive
}
index BlogHolderLive : BaseIdx {
	source = BlogHolderLiveSrc
	path = /tmp/silverstripe-cache-home-aabweb-public_html-ukc/sphinx/idxs/BlogHolderLive
}

source BlogHolderDeltaSrc : BaseSrc {
	type = xmlpipe2
	xmlpipe_command = /home/aabweb/public_html/ukc/sapphire/sake sphinxxmlsource/BlogHolderDelta
}
index BlogHolderDelta : BaseIdx {
	source = BlogHolderDeltaSrc
	path = /tmp/silverstripe-cache-home-aabweb-public_html-ukc/sphinx/idxs/BlogHolderDelta
}

source BlogHolderLiveDeltaSrc : BaseSrc {
	type = xmlpipe2
	xmlpipe_command = /home/aabweb/public_html/ukc/sapphire/sake sphinxxmlsource/BlogHolderLiveDelta
}
index BlogHolderLiveDelta : BaseIdx {
	source = BlogHolderLiveDeltaSrc
	path = /tmp/silverstripe-cache-home-aabweb-public_html-ukc/sphinx/idxs/BlogHolderLiveDelta
}

Also at first I had some warnings about setting $_FILE_TO_URL_MAPPING in my _ss_environment.php which I then did and the warnings went away, however I am still getting the same error that no searches have been enabled :(

Is there something else I can try?

Thanks again

Aram

Hi Aram.

Yes, it's normal in the XML that every field has a tag, with CDATA being applied to each tag.

Sphinx.conf will have an index for all decorated classes, and all their subclasses. It can, however, merge indexes for classes which have the same "signature". e.g. If I decorate class Page, and I defined a subclass MyPage that has no additional properties, and doesn't provide extra filters or things that affect the index, MyPage objects will be included in the same index as Page. However, if you define class YourPage, and it has additional text properties, then it will automatically be put into it's own index.

If the XML pipe command is producing the results that you expect, then the provision of data for indexing is OK.

Check that the sphinx directory in temp and all it's contents are owned by the same user that runs apache, and run the searchd process as the same user.

Can you also try doing a search from the command line, which uses the 'search' command that comes with sphinx. You need to specify the config file as an option and the search text. This command searches the indexes directly, rather than use searchd. If you can locate content that you know is present, then it would suggest the indexing itself is OK.

Mark

I've been playing with Sphinx but like aram I am experiencing some issues. The idxs folder in temp is never populated and searchd complains about not finding the indexes because the file does not exist. Sphinx/reindex returns ok, but no index files are written to temp. This is on a Windows machine and permissions should not be an issue.

Any ideas?

Hi.

Unfortunately the issues on Windows are going to be different. The sphinx module is completely untested on Windows, and I wouldn't expect it to work out of the box. There are some dependencies such as use of sockets to communicate with searchd that are probably unix-specific, as well as a few places where backtick is used to execute unix commands, etc.

This is not to say sphinx couldn't be used on Windows, but it will require a little work. Sounds like a good candidate feature for 0.3 :-)

Mark

Thanks a ton Mark. I reverted back to UNIX and managed to get sphinx going. Quite a bit simpler than Windows.

Anyways the main issue in Windows was the paths generated in the config file (they were not command friendly :)) as well as the location the module was trying to locate the sphinx binaries in. I could have manually recoded it, but decided not to bother.

Back to nix. After moving the binaries to /usr/local/bin I managed to generate the idxs in tmp. Excellent :P.

I seem to be sorted now.

By the way my $SphinxSearchForm also does not show up by default.