Impala shell request not returning same results as impala shell through ssh
Closed this issue · 5 comments
Hi,
I have been attempting to use the opensky.request() function to retrieve data for a latitude, longitude, altitude range for January 1st, 2019 00:00-01:00 (just one hour). I have start time at the 00:00 epoch time (UTC) and the end time as 01:00 epoch time (UTC) minus 1 second. I have the request including hour>={before_hour} and hour<={after_hour}.
Utilizing opensky.request(request_patter, start, stop, columns), 136,181 results are returned. However, I noticed some of these results are for after 01:00. I then checked with pandas how many records were above the stop time, and 65,350 results wee above the stop time. Therefore, I checked the same query using ssh with the OpenSky Impala Shell, and only 78,091 results are returned.
Therefore, the request using opensky.request() is returning results in the next hour when it has not been specified to. I have looked in the code, and I cannot tell how you compute the before and after times and hours. In this case, the stop - start is actually less than the default time delta, which is 1hr, so it should be somehow created from some variable "iter", which I am not sure where that is defined. I have also tried changing the date_delta to timedelta(minutes=59), thinking maybe it was the specified date delta of 1 hr that was causing results to be returned in the next hour. This did not fix the issue.
Thank you for any insight you have regarding this!
Thank you @scorrado10 for the feedback. I am sorry, I need time that I don't have to investigate your issue but will come back to you after September 1st.
Hello, I have to look more in depth into what you mention, but there may be an issue of open/closed intervals along start <= time < stop which is sometimes (by mistake) start <= time <= stop. You may want to try to correct this manually before I do that if you are in a hurry...
Hello @scorrado10 do you have any update on this issue? or should we close it?
I am not sure I get what should be fixed without a specific query showing wrong results.
Hi @xoolive, the results are not necessary wrong, just for a larger interval than specified. My work-around is to just do a check after the request and remove any state vectors with timestamp < start time and > stop time in my code. It is also probably possible to do the same thing in the package code, but I chose to just keep in it my code for simplicity. Thank you!
Ok I'll just close the issue until I can reproduce then. Thank you