Symphony.

I’ve builded site, with simple search engine. Basically it is Data Source with regexp:{$url-q}. It searches only one field in one data source (name and surname). It works but when searching for name+surname but stops working in case of surname+name. Why there is no search extension (I’m not interested in Google and Yahoo extensions)? There are more things that symphony lacks of eg. no SMTP support (here in Poland there are no providers that permits use of mail() function) or front-end members (I know there is extension, but is not official for now, and need hacks to use it) For me symphony is not yet ready for commercial use!

It works but when searching for name+surname but stops working in case of surname+name

Probably because it’s looking for an exact match on that single string surname name rather than surname OR name. Presently OR filters on a single field are not possible.

Why there is no search extension (I’m not interested in Google and Yahoo extensions)?

Remember extensions are written mostly by the community. If you want one, you can always go ahead and build one! Or write a specification and make a request. The problem with search is that everybody has different needs. Some people want to search their whole site, some people need sorting by relevance, some people just need to search a single field in a single section.

There is a thread entitled Implementing Search which specifies how a search index might work. It’s just nobody has had the need to build it yet. It’s on my to-do list, but it’s a big job and will likely take time and collaboration.

Are you comfortable with PHP and SQL? You can download the Forum ensemble and take a look at how search works there.

If you can wait until 2.0.7 (the next release) you would be able to use my SymQL extension to achieve this “OR” functionality that you need. But again, you’ll need to write a few lines of PHP to do it.

Thanx Nick, problem is i’m not so comfortable with php. Means I got some knowledge about it, but I’m not specialist. It seems i need to wait… probably about year or at least half when next version will be released (sorry, today i’m pessimist, maybe because of winter).

Search is something I’d definitely like as well, even if it was just a simple but easy to implement search.

What do others think of a donation drive? The proceeds go to paying some Symphony developers to create the necessary extension.

This is something that has been on my to-do list for a while now, I haven’t got time to build it right now, but I can start working on it in february..

It would be good if some of the thinking (how should the searching be implemented, what should be possible, etc) could be done in advance, so I can focus on the coding.

icek, have you tried without the regexp:? Looking at the code for the Textarea is already implements FULLTEXT searches. If you don’t specify a regexp: filter then FULLTEXT is used (the SQL uses a MATCH(value) AGAINST ('name surname)). So removing the regex entirely might solve your problem.

creativeditchmen, I have a pretty good idea about how a search index extension would work. I’m putting some ideas together :)

Nick: sounds good! Input in this one is very much appreciated!

@Nick: it’s not textarea, but simple Text Input. I’ll try with textfield and let you know.

Doesn’t work, even for exact match :/

Edit: forgot about curly braces… and it works! Thank You Nick!!!

Edit: forgot about curly braces… and it works! Thank You Nick!!!

Great :-)

creativedutchmen, I’ve got a proof of concept half working, so things are looking good. In the next week or two I’ll put what I’ve done onto Github for others to try out and refine. It won’t solve all search requirements, but it’ll cover most of them…

Anything happening with that search stuff on Github?

Anything happening with that search stuff on Github?

Not enough hours in the day I’m afraid. I’ll probably pick it up again if the need arises for a client project though.

Nick: can you show me what you’ve already got? I won’t have too much time, but it could be a nice getaway from work..

Nick’s proof of concept

Let me explain where I got, and what needs to be done:

Search Index aims to provide the following functionality:

• provides a pre-indexed set of data to query (no SQL joins at runtime)
• index sections based on XPath against an <entry>
• allows search of a single section based on a DS Filter (simple mode, no ordering by relevance)
• allows search with multiple sections on an included custom DS (advanced, allows ordering by relevance)

The UI for configuring the sections to index is missing, since the focus was on the search itself. The driver has a getIndexedSections method which should in time get an array from config.php. This array is indexed by section ID, each section should have two keys:

• fields (comma list of element names to include)
• filters (array of DS filters)

When an entry is created or saved, and the section is to be indexed, the entry is first run against these DS filters. This means you can set up rules to only index “published” entries, or whatever critera you want. If an entry passes DS filtering, its XML is built and the text() values from each included element are concatenated together to form a plain text string which is stored in the search index table.

At runtime, you can perform two types of searches:

Searching within a single section:

Start by attaching the Search Index field to the section you wish you search (and also make sure its configuration is set!). This field will appear in the DS editor so that you can filter on it. For now it accepts a single string to search on (it doesn’t support multiple strings or regex).

When the DS runs the string passed to the filter is used in a MATCH/AGAINST query (using MySQL FULLTEXT search) and matching entries returned.

This method is useful for when you already have filters on a DS and need to add another for searching. Using a standard DS returns <entry> elements as usual.

The downside here is that you can’t order the results by relevance (such is the way Symphony builds queries). But that’s an acceptable limitation.

Searching multiple sections:

If you want more of a site-wide search, then there is a custom DS for querying multiple sections. Attach this to your page and send a form via GET using the following:

• keywords (the search query, required)
• sections (comma list of section handles, required)
• per-page (number of results to page, optional, defaults to 20)
• sort (score, date or id, optional, defaults to score)
• direction (asc or desc, optional,defaults to desc)

So a search URL might look like:

/search/?keywords=woo+yay&sections=articles,comments,images

This DS doesn’t output <entry> elements, instead it creates a <pagination> element and provides the entry IDs as output parameters. For each section you are searching within, create a new DS and filter its System ID by the $ds-search-index output parameter.

This leaves the display of entries entirely to the developer, thereby keeping the search routine generic and lean.

And because it’s a custom DS this supports ordering by relevance.

A word on MySQL’s FULLTEXT search: there are limitations which should be known. For example the default minimum search term is 4 characters. And entries that contain a high percentage of that word (e.g. if more than 50% of the text is the search keyword) the row is ignored.

So to finish this off all that is needed is a UI for configuring the search indexes. I see this as a cut-down DS editor:

• an overview page with a table listing each section that is being indexed
• click through to an editor page with UI to add filters (lifted directly from the DS Editor) and a list of included elements (again from the DS Editor)
• persist this as an array into the config.php (or to a separate file, for easier source control integration)
• the overview page should have a “With Selected” option to manually rebuild the index for that section (e.g. if you change the index Filters), which will need to iterate over all entries in the section and run them through the filter again

By all means take what I’ve done and run with it :-)

Nice!

It will take some time to totally grasp what you’ve done here, but I think it matches my ideas pretty well.

I’ll keep you posted!

I’ll keep you posted!

Please do. It would be great to get this under way and, who knows, even completed!

If you’ve able to work on it, please send pull requests through Github and I’ll pull it together. I’m happy to write lengthy documentation as an implementation guide. When complete I’d also be happy to release an Ensemble as a full working example of single and multi-section searching.

I just had some more time to understand what you’ve done: I’m really impressed!

In an attempt for a client a few months back I just took the lucene class from zend, and fed data into that, but this is way more suited to symphony.

Working on the UI now.

Nick: is it possible to hook into the DS-editor without using any core hacks?

Nope, I don’t think there are delegates for that. What are you trying to do?