vBulletin Mods

The Official vBulletin Modifications Site
https://www.vbulletin.org/forum/showthread.php?t=127868

eoc_Jason 19 Jan 2010 16:24

Quote:

Originally Posted by mute (Post 1958667)
We have 2 post indexes, one or our live post table, and one for our archived post table. They each have 30 million posts each. I don't see a point in sharding the post indexes aside from being able to take advantage of multiple CPUs when indexing.

The way I see it, if I can keep the old indexes online while I do a full reindex, I don't really care how long the full reindex takes since (at least in our case), the search server is just a slave database server and not our primary.

Splitting up the posts index into several sources has more advantages that just re-indexing. As kmike eluded to, you can setup "agents" on your server so that when a person does a search it will search all the sources in parallel utilizing one source per CPU.

For most people though, even large indexes we are all probably just using 1 CPU without realizing it and still maintaining less than one second search times...

This week I'm going to be swamped working on some new AMD processor reviews, so work on the new sphinx code will have to be set aside until Tuesday next week.

I'm super pumped though, after diving into the documentation and code, I realized it's really not that bad at all...

kris 19 Jan 2010 17:45

Quote:

Splitting up the posts index into several sources has more advantages that just re-indexing. As kmike eluded to, you can setup "agents" on your server so that when a person does a search it will search all the sources in parallel utilizing one source per CPU.
eoc_Jason, can you share how to split post table index into several sources ?

kmike 26 Jan 2010 10:52

FYI, realtime index updates in Sphinx are near:
http://twitter.com/sphinxsearch/statuses/8179531738
Looks like so far the realtime indexes are only accessible via SphinxQL.

weeno 25 Feb 2010 00:30

hey guys,

Can someone post their sphinx.conf file for sphinx version 0.9.9? I'm still running into a few problems so wanted to compare.

arn

boggseric 26 Feb 2010 02:43

Was post number 1 supposed to contain some XML mods? Or am I missing something here.

edit: Never mind, there are no xml files, you have to create the plugins manually and copy and paste the code into them.

ivanp 01 Mar 2010 21:42

Any update for Sphinx for vB4?

Lea Verou 01 Mar 2010 22:12

Has anyone used Sphinx with languages != english?
Does it work correctly?
FULLTEXT had issues with greek...

boggseric 13 Mar 2010 17:18

Quote:

Originally Posted by kerplunknet (Post 1650862)
Has anyone figured out the "Find More Posts by UsernameHere" issue? It results in these error messages, but the results seem to work properly:


Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

I really don't want to put Sphinx live until this is fixed... any advice would be welcome.

I am wondering if it has something to do with using an older 3.6 version of vBulletin.

This is a pretty old post but it is still relevant. The from looking at the line it's refereing to it looks like he problem is with the function that it is trying to find the search text in the post result to highlight it. Since there is no search text the value is null and strpos does't like that.


edit: I think I got it fixed. The problem is with the process_quote_removal function.
Keep in mind I am running 3.8.4pl2
I edited includes/functions_search.php and made the following changes

Search for this block of code:

Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

This line is the culprit for me:
Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

it appears that $word ends up being null which causes strpos to puke the errors.

so I changed the function to this:

Block Disabled:      (Update License Status)  
Suspended or Unlicensed Members Cannot View Code.

You can see I just added an if statement to check if $word is null. If $word is null it skips over the code since there is obviously nothing to highlight.

I have tested the normal search and the search term is still highlighted and I am no longer getting errors during search for more posts by this user.. YMMV.

WoodiE 14 Mar 2010 00:11

Is this real or just an info about Sphinx? Someone told me that it costs $2,000 to have it setup on vBulletin - no?

TechGuy 14 Mar 2010 00:57

Unfortunately, no... you can install Sphinx yourself for free, but it is no small task and there are some issues to deal with.

The $2k is likely referring to the "Axivo Searchlight" product, though I hesitate to use the word product as it's been "coming soon" for more than a year. They finally released a closed beta, so I suspect it's still being developed, but that was over three months ago and still no product.

boggseric 14 Mar 2010 00:59

Quote:

Originally Posted by WoodiE (Post 2003132)
Is this real or just an info about Sphinx? Someone told me that it costs $2,000 to have it setup on vBulletin - no?

I just set it up using the info in post 1 and the files from post 582

I used the info from this post to make sure it stayed running and this post to setup the crons for delta and nightly.

Other than that the only changes I made were from my previous post to fix the "find more posts by this user" issue.

It is now up and running on my 3.8.4pl2 forums with:
Threads: 799,861, Posts: 11,892,282, Members: 24,815

I set the max amount of of results to 20,000 and it's working great. Searches no longer take my forum to it's knees.

The things I haven't done yet are to remove the indexes from post and convert my tables to innodb.


Total cost? My time reading through this never ending thread and compiling the data above.

WoodiE 14 Mar 2010 14:10

TechGuy - I didn't realize there was two different options for using Sphinx on vBulletin. Though by the sounds of it neither are very easy to setup. One either very pricey, two according to boggseric a lot of files and editing to do - both still have issues and nothing everything working.

Does anyone know of any other solutions to making vBulletin search easier on the server and sill maintain all of vB's features?

adnoid 14 Mar 2010 19:24

Quote:

Originally Posted by boggseric (Post 2003154)
I just set it up using the info in post 1 and the files from post 582

I used the info from this post to make sure it stayed running and this post to setup the crons for delta and nightly.

Other than that the only changes I made were from my previous post to fix the "find more posts by this user" issue.

It is now up and running on my 3.8.4pl2 forums with:
Threads: 799,861, Posts: 11,892,282, Members: 24,815

I set the max amount of of results to 20,000 and it's working great. Searches no longer take my forum to it's knees.

The things I haven't done yet are to remove the indexes from post and convert my tables to innodb.


Total cost? My time reading through this never ending thread and compiling the data above.

Thank you for putting this all together. I've been looking at the alternatives for our board, now approaching 5,000,000 posts. I'm not a heavyweight coder, but I'll be giving your method a shot.

boggseric 15 Mar 2010 12:39

Quote:

Originally Posted by WoodiE (Post 2003387)
TechGuy - I didn't realize there was two different options for using Sphinx on vBulletin. Though by the sounds of it neither are very easy to setup. One either very pricey, two according to boggseric a lot of files and editing to do - both still have issues and nothing everything working.

Does anyone know of any other solutions to making vBulletin search easier on the server and sill maintain all of vB's features?

You'd probably be best off if you did it yourself, that way if you need to make changes in the future you know what you are doing. It's not overly complicated.

I don't know squat about linux or vB but I know computers and some programming (other than PHP). I managed to fumble my way through it.

If you can afford the 2k solution, good. If not maybe you'd be better off paying someone to set sphinx up for you.

TechGuy 15 Mar 2010 12:42

My point is that even if you can afford the $2k solution, it doesn't matter -- that solution doesn't exist.

Without reading through the entire thread, what's the status of using advanced search fields using the solution you mentioned earlier? (It was a long time ago when we last tried, but I remember we had to drop it because we couldn't sort by most of the fields.)


All times are GMT. The time now is 02:54.

Powered by vBulletin® Version 3.8.14
Copyright © 2021, MH Sub I, LLC dba vBulletin. All Rights Reserved. vBulletin® is a registered trademark of MH Sub I, LLC
Copyright ©2001 - , vbulletin.org. All rights reserved.