The Official vBulletin Modifications Site.

amcd · #**706** 12 Aug 2009, 06:55

I may have to run a cron job to check for any *.new.* files in that directory, and possibly put together a shell script to kill and restart searchd.

I would rather look for the cause of the malfunction. If most people do not get such problems, why should you?

--------------- Added 12 Aug 2009 at 06:57 ---------------

Think I'm safe to change "value" to "1" (and rebuild the datastore cache)?

I have no idea. Maybe someone from Jelsoft can answer. Or you can try reading the AdminCP PHP script which changes search type.

UK Jimbo · #**707** 12 Aug 2009, 09:06

Originally Posted by amcd

I would rather look for the cause of the malfunction. If most people do not get such problems, why should you?

I agree on this one.

RedWingFan do you have a log of what the indexer is doing incuding any messages from the cronjobs?

Shamil. · #**708** 13 Aug 2009, 20:10

Is this verified to work with the latest vB, 3.8.4 ?

RedWingFan · #**709** 16 Aug 2009, 20:12

Originally Posted by amcd

I have no idea. Maybe someone from Jelsoft can answer. Or you can try reading the AdminCP PHP script which changes search type.

Well, I changed the value in the database, regenerated the options datastore, and it's working properly (as though I had chosen "Fulltext"). So that's a success!

Originally Posted by UK Jimbo

RedWingFan do you have a log of what the indexer is doing incuding any messages from the cronjobs?

Memory allocation error during rotate. It happened again last night, and I found the memory allocation error in the log. I changed this value in my conf file: "seamless_rotate=0". Searching will be interrupted a bit when the indexes rotate, but since I do this during off-hours, it won't affect many users at all. (Apparently, Sphinx's indexer will load indexes into RAM so that users can still search while the old indexes are being replaced by the new.) The mem_limit is at 512MB right now, but I don't want to increase that and possibly starve everything else on the server. http://www.sphinxsearch.com/docs/cur...eamless-rotate

One other setting looks a bit deceptive: max_matches is set for 1000 in my conf file, but I'm only pulling in 500 in vB's search. What Sphinx does, for max_matches, is send back the BEST 1000 matches, not necessarily just running the search and returning the FIRST 1000 matches it finds. My point here is that I should be setting the maximum search results in the conf file and in vB to be the same number. Basically I'm searching for 1000 best matches, but throwing away 500 of them for visitors. I will probably bump the forum to display those 1000 matches. Visitors will think they're getting a bonus.

http://www.sphinxsearch.com/docs/cur...nf-max-matches

Here is something else: http://www.sphinxsearch.com/docs/cur...nf-enable-star . You can search Sphinx using the asterisk ("star") as a wildcard. I have thought of enabling this, but I am thinking that the search_sphinx.php file (or vB itself) would strip out the asterisk and make no difference during searches. This would be a neat addition. Has anyone else here tried it?

I came across another option for the indexer. When indexing, there is a --merge option for the indexer, which will merge your delta indexes with your main indexes, rather than generating new indexes once per day. I have it running now where the indexes are regenerated once each night. Would there be any disadvantage to using merge? If it goes correctly, it should work just as well, since you have essentially the same indexes when you're finished. But I can also see a tiny opportunity for the main indexes to get corrupted. Otherwise, --merge takes less time and CPU cycles, which is attractive. http://www.sphinxsearch.com/docs/cur...#index-merging

Finally, I see that Sphinx also has a plugin for MySQL, where you can specify using SphinxSE as an additional engine in MySQL. It will not do us much good here, I know, but I could see a future use for it between vB and Sphinx. http://www.sphinxsearch.com/docs/current.html#sphinxse

IMHO, given how much Sphinx's popularity is growing, and after having pored over the documentation this afternoon, it is disappointing that there apparently will not be any built-in support for Sphinx in vB 4.0. Sphinx can do a lot, and is a lot more flexible, than the built-in vB search, as well as MySQL's own fulltext indexing. A shame we'll probably still have to patch these hacks together to use Sphinx...

amcd · #**710** 17 Aug 2009, 06:42

I came across another option for the indexer. When indexing, there is a --merge option for the indexer, which will merge your delta indexes with your main indexes, rather than generating new indexes once per day. I have it running now where the indexes are regenerated once each night. Would there be any disadvantage to using merge? If it goes correctly, it should work just as well, since you have essentially the same indexes when you're finished. But I can also see a tiny opportunity for the main indexes to get corrupted. Otherwise, --merge takes less time and CPU cycles, which is attractive. http://www.sphinxsearch.com/docs/cur...#index-merging

moved/merged/edited/deleted posts are a huge reason to regenerate the whole index periodically.

RedWingFan · #**711** 17 Aug 2009, 17:35

Originally Posted by amcd

moved/merged/edited/deleted posts are a huge reason to regenerate the whole index periodically.

Oh jeez...yeah, you're right! I didn't even think of that!

RedWingFan · #**712** 24 Aug 2009, 19:48

Originally Posted by RedWingFan

Here is something else: http://www.sphinxsearch.com/docs/cur...nf-enable-star . You can search Sphinx using the asterisk ("star") as a wildcard. I have thought of enabling this, but I am thinking that the search_sphinx.php file (or vB itself) would strip out the asterisk and make no difference during searches. This would be a neat addition. Has anyone else here tried it?

Just curious if anyone has tried this at all. (This was buried in my last lengthy post.) I believe Sphinx also has phrase searching, where you enter search terms in quotes, but I have a feeling that vB strips out anything that's not alphanumeric...

Both of these would be a welcome addition to vB search.

columbusgeek · #**713** 09 Sep 2009, 21:47

RedWingFan are you using this on your music forum? Was trying to find examples of people using it, but having a hard time trying to find out if they are or not. Its a lot of work to get this installed on the forum, I want to make sure its worth it.

RedWingFan · #**714** 09 Sep 2009, 22:27

Yes, we're using it, and aside from a few minor quirks (which I can't even remember right now), it is working great! I don't know if it helped server loads or not (we had to do several changes at once to get us out of our overloads), but it returns searches much faster than using vB's own built-in search.

I still with vB would develop something official for this. The built-in searching (using vB or fulltext) is just too hard on the server resources...

amcd · #**715** 10 Sep 2009, 08:24

Originally Posted by columbusgeek

RedWingFan are you using this on your music forum? Was trying to find examples of people using it, but having a hard time trying to find out if they are or not. Its a lot of work to get this installed on the forum, I want to make sure its worth it.

Believe me, it is worth the trouble.

cobaku · #**716** 29 Sep 2009, 03:24

i somehow managed to set it up was difficult for me cause it was with latest stable version of sphinx

I have 1 problem i can only search latest data
is the problem this line
$cl->SetLimits(0, $vbulletin->config['Sphinx']['limit'], 1000);

indexer --config /etc/sphinx/sphinx.conf --all

did not work for me

do you see anything strange below ?

Block Disabled: (Update License Status)
Suspended or Unlicensed Members Cannot View Code.

is it normal nothing returns from postdelta and threaddelta

--------------- Added 29 Sep 2009 at 21:21 ---------------

if i can ever overcome this problem i will make a post sphinx search for dummies this post is so messed up.

Raun · #**717** 21 Oct 2009, 07:06

If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB

DaiTengu · #**718** 21 Oct 2009, 07:35

Originally Posted by Raun

If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB

slave DB server would be your best bet.

UK Jimbo · #**719** 21 Oct 2009, 09:22

Originally Posted by Raun

If I am installing this on a vbulletin installation that is setup as follows what box would it go on?

1load balancer
4 web servers
1 master DB
1 slave DB

It depends on the load on the two db servers. My guess would be that the least loaded in this configuration will be the master db server. Both master and slave are handling the same number of write (INSERT/UPDATE/DELETE) queries but the slave will be handling a load of reads too.

With a bit of scripting wrapping index copying (rsync probably) and restarting searchd you could generate the indexes on one machine and then copy them out to another one. This might be handy if you want to build some kind of redundancy into your setup or be able to balance sphinx traffic.

Raun · #**720** 21 Oct 2009, 16:32

If I were to install it on both DBs would that work the same as copying the indexes?

New To Site?	Need Help?
About vBulletin.org You will need to register to Participate Review Our Site Rules	Mark Forums Read Do Your Research Before Hiring! Site Map Meet Our Staff Do Not Sell My Personal Information

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

The Official vBulletin Modifications Site.

Sphinx Search