Distinction of AND and OR in search
Closed this issue · 23 comments
The and/or within a search criteria isn't working. Examples follow:
Diagnosis Type (if search for bald eagles and bacteria AND fungus, get same results as for bald eagles and bacteria OR fungus)
Event Diagnosis (if search for bald eagles and Toxicosis (lead) AND Anemia, get same results as for bald eagles and Toxicosis (lead) OR Anemia)
Species (if search for bald eagles AND American coot and bacteria and fungus, get same results as for bald eagles OR American coot and bacteria and fungus)
State (if search for bald eagles and Washington AND Alaska, get same results as for bald eagles and Washington OR Alaska)
County (if search for bald eagles in Alaska for Juneau Borough AND Kodiak Island Borough, get same results as for bald eagles in
Alaska for Juneau Borough OR Kodiak Island Borough)
@aaronstephenson I checked these and can confirm - the services seem to be ignoring the and_params query param in the request. The client is indeed sending it, but results do not respect it.
I suspect this has something to do with the change to using the django-filters package instead of custom-written filters.
Yep, that's exactly what it was. The query params were no longer in the kwargs dict (where the code was looking for them), but in the request dict. Working on the fix now.
This should now be fixed in 1dd1b24, and I have pushed the changes to the Test server only. @JChipault care to try your queries again to confirm this is fixed?
@aaronstephenson The AND search still seems wonky.
For example, if search for Event Diagnosis = Toxicosis (lead) OR Peritonitis and species = bald eagle, get 9 results. Note event 13319, which has both lead and peritonitis. Yet when I search for Event Diagnosis = Toxicosis (lead) AND Peritonitis and species = bald eagle, get NO results.
@JChipault I just ran those same searches in the test web client and got 9 results using OR and 1 result (event 13319) using AND. As confirmation, I ran those same searches using the API:
https://whisperstest.wim.usgs.gov/api/eventsummaries/?diagnosis=321,337&species=181
returns 9 results, the same as what you got.
and
https://whisperstest.wim.usgs.gov/api/eventsummaries/?and_params=diagnosis&diagnosis=321,337&species=181
returns 1 result for me, Event 13319.
Huh, it gets more strange. I was logged in as abilenetest. When I logged out, I get that 13319 as expected during an AND search. Log back in as abilenetest and get no search results. That event is not private. Not sure why abilenetest would not see it????
abilenetest password is testtesttest2! if you want to try it out
When logged in as abilenetest, I can do a search for event=13319 and it shows up. But when I do a search for event=13319 and event diagnosis = Toxicosis (lead) AND Peritonitis, get no results. If I do the OR version of the event diagnosis search, 13319 shows up again.
same issue in both Firefox and Chrome
Yeah, the API works the same regardless of client. JS clients sometimes behave differently in different browsers, though.
OK, I figured out why a partner user gets an empty response for this query. It has to do with how django combines two lists. Apparently, when using the recommended pipe operator (|), if one of the two lists is empty, the result is also empty, which is not at all what I expected. I'm testing a workaround now.
Fixed in 542419d and pushed to test server. @JChipault care to give it another try? It looks to be working from what I see.
And thanks again for your thorough testing using multiple roles. I tend to just test as the anonymous (public) user and as Admin, and often don't think to try the other roles. I've gotta change that habit! 😃
Hi @JChipault sorry to bother you, but are you able to test this fix this morning? I have another bug fix (for issue #468) that I would like to deploy to the test server but I don't want to mingle two fixes, to keep things tidy.
Thanks for the prompt. Will test now. Have been spending most of the morning on the 475 ticket.
Still some strangeness.... mostly seems related to accounts/permissions. For example, if I'm logged out (so just unregistered public user) and do search for Bacteria OR Fungus and Bald Eagle, I get 12 results. If I log in as abilenetest, I suddenly get only 1 result (which is what I should get if the search were Bacteria AND Fungus and Bald Eagle; but I haven't changed the search). Log out, and I still see only 1 result.
If I hit the "search" button again (either while logged in as abilenetest or while logged out), it'll reset to 12 results.
I get 12 results for Bacteria OR Fungus and Bald Eagle, both when not logged in and when logged in as abilenetest. I tried the search on two different browsers. I asked a coworker to try it out and she got the same results as me.
Are you maybe using the prod site when doing this search?
definitely in the test site. just tried it again and it happened
are you keeping the same browser tab open and just logging out and logging in? looks like it's maybe only an issue in Chrome for me
Chrome Version 90.0.4430.212 (Official Build) (64-bit)
We log in and out in one browser tab. Have you tried clearing your cache? Can you maybe try a different computer or ask a colleague to give it a try?
ok, working now with cache cleared. also works on my personal laptop
Thanks for checking!