Forums Register Login Forgot your login/password? Search
Common forum | 1 | 2 | 3 | 4 | 5 | ... | 451 | 452 | 453 | 454 | next »» | Create new thread
|
garyis
Name: garyis |
2006-10-18 00:29:01
| reply! Is it possible to enter no query and to just browse results based on group/date? If not could it be a possible new feature. I would use mysql but I think doing it with sphinx could be faster =) Also looking forward to the new version. This is a great thing you're doing. |
|
shodan
Name: Andrew Aksyonoff |
to: garyis, 2006-10-18 01:45:41
| reply! > Is it possible to enter no query and to just browse results based on group/date? No - Sphinx is definitely not intended to be used as a replacement for SQL server. |
|
garyis
Name: garyis |
to: shodan, 2006-10-18 14:01:07
| reply! > > Is it possible to enter no query and to just browse results based on group/date? > > No - Sphinx is definitely not intended to be used as a replacement for SQL server. ok. I just wanted to see if it's faster. Sorting huge tables with mysql can be quite slow. If anyones interested I found a way to do this but it isn't very elegant. Include a word with each record used in the index that is unlikely to ever be searched and search for that. I used a three letter string with non-standard characters. |
|
ilance
Name: Len |
to: garyis, 2006-10-19 16:06:02
| reply! > ok. I just wanted to see if it's faster. Sorting huge tables with mysql can be quite slow. And... Was it faster ? |
|
garyis
Name: garyis |
to: ilance, 2006-10-20 01:34:33
| reply! > > ok. I just wanted to see if it's faster. Sorting huge tables with mysql can be quite > slow. > > And... Was it faster ? Yes alot faster. Most of all it freed up mysql to do other things such updating, serving content etc. Bare in mind that my mysql index size way exceeds my ram so doing anything takes longer than usual. If you want to do exactly what I did. Add '___' AS b somewhere in your index sql query. When you want to browse just search for ___ in boolean mode. I used '___' as it is rarely searched for and treated as a word by sphinx. You'll have to extend it if your min size is bigger than 3. |
|
shodan
Name: Andrew Aksyonoff |
to: garyis, 2006-10-23 05:59:44
| reply! > ok. I just wanted to see if it's faster. Sorting huge tables with mysql can be quite slow. Well, my own preference would be to optimize MySQL side in this case. Creating indexes and/or special pre-calculated tables to accelerate selects should theoretically provide more or less the same results. |
|
dennisj
Name: Dennis Jacobfeuerborn |
to: shodan, 2006-10-27 15:19:11
| reply! > > ok. I just wanted to see if it's faster. Sorting huge tables with mysql can be quite > slow. > > Well, my own preference would be to optimize MySQL side in this case. Creating indexes > and/or special pre-calculated tables to accelerate selects should theoretically provide > more or less the same results. Good point. The problem however is that people have to implement two completely different search strategies depending only on whether the query is empty or not. Yes, you could create special pre-calculated tables but in my case I already do this for sphinx so having to do this twice makes this very maintenance heavy. Given that Sphinx basically first searches for matches in the index and then filters and sorts the result wouldn't it be relatively simple to make the search stage simply return the entire index as matches? It looks like a small modification that could make the life of Sphinx users a lot less painful. |
|
shodan
Name: Andrew Aksyonoff |
to: dennisj, 2006-10-28 16:10:45
| reply! > wouldn't it be relatively simple to make the search stage simply return the entire index > as matches? Yes, it'd be rather easy. The idea still seems somewhat strange to me, because I can't immediately think of any usage scenario which would be significantly faster with selecting the necessary records through Sphinx rather than MySQL. My guess is that it should be a matter of creating a few indexes to optimize those slower SELECTs. On the other hand, if there's so much demand for that feature, I could definitely add it even if I don't quite understand the need :) |
|
dennisj
Name: Dennis Jacobfeuerborn |
to: shodan, 2006-10-30 10:45:26
| reply! > The idea still seems somewhat strange to me, because I can't immediately think of any > usage scenario which would be significantly faster with selecting the necessary records > through Sphinx rather than MySQL. My guess is that it should be a matter of creating a > few indexes to optimize those slower SELECTs. I agree that the direct database approach is most likely faster but in many cases the difference in performance is not big enough to force users down that road which is effectively what sphinx does at the moment. When I ask sphinx the question "return the documents that match 'xyz' and have a userid of 5 and sort them by date" then it replies "there you go" and delivers the requested information. However when I ask "return the documents that have a userid of 5 and sort them by date" which is a perfectly valid inquiry sphinx basically replies with "Well, I could give you the answer but I refuse to because I think there is a more optimal way to do this". I'd prefer it to answer this question with "Well, I don't think this is the best way to do this but since it's a valid request that I can answer here you go". Simply put I believe Sphinx should respond to all valid inquiries with a positive response regardless of how optimal it thinks the inquiry is. |
|
shodan
Name: Andrew Aksyonoff |
to: dennisj, 2006-10-30 16:32:00
| reply! > "return the documents that have a userid of 5 and sort them by date" You're omitting that "match" part here, and Sphinx is a full-text search system, after all, so there's implicit empty full-text query. On an unrelated note, the question "what's the right answer to empty query" in itself seems something like a Zen koan to me :) > which is a perfectly valid inquiry sphinx basically replies with "Well, I could give you > the answer but I refuse No, this "Well I could" is what you just made up, it's far from being that simple. It's possible that Sphinx doesn't keep a complete list of all document IDs at all. Pre-0.9.7 versions never kept that list at all. 0.9.7 only keeps it if a) there actually are additional attributes attached to each document b) external docinfo storage is specified in config file. So this means that if we decide to change the behaviour, from now on (ie. in 0.9.7-release and higher) we should explicitly store a complete list of document IDs and attributes, even if it's not needed (!) at all for full-text searches. OK, let's imagine for a moment that we decided to that. Now, the "empty" query should always match the whole database, right? What if the query was not empty before we applied stopwords? What if the query was all whitespace? What if the query was all whitespace but the whitespace is configured as a valid in-word character - it's possible, and some installation are doing this to search through tags? To summarise, my strong belief is that the behaviour which returns all records in N cases and no records in M cases (depending on a number of factors from software version to the query layout) is inconsistent and counterintuitive, and as such, a Bad Thing To Do. > Simply put I believe Sphinx should respond to all valid inquiries with a positive > response regardless of how optimal it thinks the inquiry is. Now, in *some* cases Sphinx could technically return the answer to non-full-text query ("I know you have userd and date anyway, so give me all the IDs that have userid=5 sorted by date"). My point is that this would no longer be full-text query; it's rather a simplified SELECT. It seems more or less OK to have another mechanism to do such simplified SELECTs against Sphinx's internal dataset. However it's definitely not OK to interbreed full-text querying and simplified SQL - too much issues, too much inconsistency. |
|
garyis
Name: garyis |
to: shodan, 2006-10-31 11:08:12
| reply! > What if the query was not empty before we applied stopwords? > > What if the query was all whitespace? > > What if the query was all whitespace but the whitespace is configured as a valid in-word > character - it's possible, and some installation are doing this to search through tags? It doesn't have to be done by leaving the query empty. How about an extra query mode, or by not passing a query string at all. |
|
shodan
Name: Andrew Aksyonoff |
to: garyis, 2006-10-31 12:58:54
| reply! > How about an extra query mode, or by not passing a query string at all. I'm thinking of a new API call to run simple SELECT-like queries against internal attributes database. |
|
robert
Name: Robert Freund |
to: shodan, 2006-12-01 06:00:05
| reply! This feature (sphinx returning search results on empty query string) is quite important for us. |
|
dennisj
Name: Dennis Jacobfeuerborn |
to: shodan, 2006-12-02 19:09:56
| reply! > > How about an extra query mode, or by not passing a query string at all. > > I'm thinking of a new API call to run simple SELECT-like queries against internal > attributes database. Maybe that's not necessary. Wouldn't it be enough to allow users to specify the query as "null"? Like calling "$sphinxclient->Query(null)" or maybe "$sphinxclient->NoQuery()". That way you could get around the query ambiguities mentioned above, define a behavior for this specific case and wouldn't require to create a whole new API. |
|
shodan
Name: Andrew Aksyonoff |
to: dennisj, 2006-12-02 19:32:22
| reply! > Maybe that's not necessary. Wouldn't it be enough to allow users to specify the query as > "null"? Like calling "$sphinxclient->Query(null)" or maybe "$sphinxclient->NoQuery()". Well, these are completely different to me, and that's what I was talking about :) Modifying existing $cl->Query ( $search_string, "idx1 idx2" ) call raises a lot of questions - but adding another call seems OK to me. So the code would look like $cl->SelectFiltered ( "idx1 idx2" ) and all the other calls would stay just the same. |
|
tibo
Name: Thibault Hanin |
to: shodan, 2007-01-25 08:06:06
| reply! Was this feature (Sphinx returning search results on empty query string) implemented in 0.9.7-RC2 ? If not, is it planned for another version ? |
|
sid
Name: Sid |
to: shodan, 2007-01-25 08:18:41
| reply! Would love this feature. |
|
shodan
Name: Andrew Aksyonoff |
to: tibo, 2007-01-28 17:39:59
| reply! > Was this feature (Sphinx returning search results on empty query string) implemented in > 0.9.7-RC2 ? Not in RC2, but planned. |
|
ecaron
Name: Eric Caron |
to: shodan, 2007-08-01 14:23:56
| reply! I too would find this useful. It simplifies the coding on our end and keeps the datasets unified (currently people browsing have access to newer data than searchers). |
|
eweaver
Name: Evan |
to: shodan, 2007-08-20 15:55:36
| reply! > > Was this feature (Sphinx returning search results on empty query string) implemented in > 0.9.7-RC2 ? > > Not in RC2, but planned. Is this implemented in 0.98 yet? If not, when do you expect it to be ready? Thanks |
|
shodan
Name: Andrew Aksyonoff |
to: eweaver, 2007-08-21 03:49:01
| reply! > Is this implemented in 0.98 yet? If not, when do you expect it to be ready? Not in the current snapshot. I'm hoping to add it until 098-release though. |
|
markmcspadden
Name: Mark McSpadden |
to: shodan, 2007-11-07 17:26:33
| reply! > > Is this implemented in 0.98 yet? If not, when do you expect it to be ready? > > Not in the current snapshot. I'm hoping to add it until 098-release though. Any update on this feature? |
|
dennisj
Name: Dennis Jacobfeuerborn |
to: markmcspadden, 2007-11-07 17:32:29
| reply! > Any update on this feature? From the news page: ... Full scan, or in other words the long awaited “empty query” feature, means that Sphinx will now process and return all document IDs if there's no query string specified. ... |
|
scottbu24
Name: Scott Bullock |
to: dennisj, 2011-08-10 01:45:32
| reply! Any update on how this works? I assume its implemented now 4 years later after this thread discussion. Whenever, I supply empty search text I get back no records. I need away to get them all and filter on all records without having to supply search text. Thanks in advance, Scott |
|
barryhunter
Name: Barry Hunter |
to: scottbu24, 2011-08-10 01:48:02
| reply! > > Any update on how this works? Yes it works. The index needs at least one attribute, and needs 'extern' storage. Entering a empty query should just work, but can use SPH_MATCH_FULLSCAN to be explicit. |
|
scottbu24
Name: Scott Bullock |
to: barryhunter, 2011-08-10 04:08:30
| reply! Barry, thanks for the confirmation. Looks like I am using a client DLL that is preventing the "" being set as the search text so Sphinx is not the guilty party. Good to know they now support this. This is an important part of the app I am porting from SQL Server based searches. Thanks again. |
Common forum | 1 | 2 | 3 | 4 | 5 | ... | 451 | 452 | 453 | 454 | next »» | Create new thread