The screener result list is not complete nor stable

[garlicbulb-puzhuo]

Run the following example from README.md. The len(stock_list) is always consistent with finviz UI but the actual stocks from the for loop is not incomplete and always changing.

from finviz.screener import Screener

filters = [
    'sh_avgvol_o100', # average daily volume > 100k shares
    'sh_curvol_o100', # current daily volume > 100k shares
    'sh_float_u50', # floating stocks < 50m shares
    'sh_price_u3' # current price is under $3
]
stock_list = Screener(filters=filters, table='Performance')  # Get the ownership table

# Print the table into the console
print(len(stock_list))

i = 1
for stock in stock_list:
    ticker = stock['Ticker']
    # print(f'{i} {ticker}')
    i = i + 1
print(i)

[d3an]

@garlicbulb-puzhuo Quick correction in your code: i = 0 instead of i = 1. I'm currently addressing this issue and another, and have written a test verifying that len(stock_list) == i. If you could post the result of the code above, it'd be much appreciated.

[garlicbulb-puzhuo]

Thanks for pointing the change. It is minor b/c the discrepancy is more than 1. I did some tracing myself and found it was caused by the rate limiting by finviz. There are paged requests and some requests failed (although with HTTP status 200) due to Too Many Requests. So I suggest that either we add some rate limiting on client side or retry upon this type of response (not an HTTP error though).

[d3an]

Thanks for pointing the change. It is minor b/c the discrepancy is more than 1. I did some tracing myself and found it was caused by the rate limiting by finviz. There are paged requests and some requests failed (although with HTTP status 200) due to Too Many Requests. So I suggest that either we add some rate limiting on client side or retry upon this type of response (not an HTTP error though).

I thought about the bug earlier, and added a basic retry to the request function I built. It's currently sitting in the #78 PR. I added a test to cover your code and it works on my end. Could you download my branch https://github.com/d3an/finviz-1/tree/SD-76/fix_download_chart_image and see if it fixes your issue?

EDIT: Also, not sure if you're aware, but the pypi finviz package that you install with pip install finviz hasn't been updated since April. To get the most recent version you should be downloading direct from this repo, i.e. pip install -U git+https://github.com/mariostoev/finviz.

[garlicbulb-puzhuo]

Thanks that is it! I tried the latest version and it works. Closing this.

[d3an]

Happy to help!