ahivert/tgtg-python

get_item() doesnt return anything after hours of continuous use

Closed this issue · 12 comments

nickhir commented

First of all, thanks for this brilliant and straight forward package.
I have been playing around with it for some time now and seem to run into a strange situation and was wondering if anyone might have experienced something similar.

I am running a script which checks a bunch of stores every 20 seconds if any item becomes available and if it does, it sends me an email. In between each API call, I add a delay of 2 seconds to make sure not too send to many requests. This works great for the first ~15 hours. However, at some point client.get_item() simply doesnt return anything. Not a proper response nor an error. Just nothing and the "code gets stuck". I was unable to reliably reproduce the issue, but it occurs every time after running it for a long time.

Did anyone experience something similar yet?

Any help is much appreciated!

ahivert commented
nickhir commented

Exactly, once it is blocked I kill my script with ctrl+c and when I restart it everything works perfectly fine for the next 15h.

I am not entirely sure what you mean with log in with email, but this is how I initialize the Client:

with open("/home/pi/projects/tgtg_access/credentials.json", "r") as f:
    credentials = json.load(f)

client = TgtgClient(access_token=credentials["access_token"],
                    refresh_token=credentials["refresh_token"],
                    user_id=credentials["user_id"])

item_metadata = client.get_item(item_id=594474)
ahivert commented
nickhir commented

Thank you for your quick answer! Could you elaborate how I can print out the token use?

ahivert commented

add this line inside your loop (in your script), it should print the access token used and the refresh token:

 print(client.access_token, client.refresh_token)
nickhir commented

Okay, I reran my script and the same "issue" occurred again. This time it ran for ~7.5h.
From what I can tell access_token and the refresh_token do not change at any point during the script.

Do you think this might cause issues and if so, how can I generate new tokens during the run?

ahivert commented

I tested it locally and looks like it should works.

This is the script I used to test it:

import time
from tgtg import TgtgClient
ACCESS_TOKEN = "..."
REFRESH_TOKEN = "..."


if __name__ == "__main__":
    client = TgtgClient(access_token=ACCESS_TOKEN, refresh_token=REFRESH_TOKEN, access_token_lifetime=60, user_id="5729945")
    while True:
        print("access_token", client.access_token)
        print("refresh", client.refresh_token)
        print(len(client.get_items()))
        time.sleep(30)

On the third iteration, the access token was updated.

Can you try to set the token lifetime when you create the client something like 1 hour (default is 4) to see if it fix your issue:

TgtgClient(..., access_token_lifetime=3600)
svove commented

Hey,

i have the same issue. After some hours of continous use the get_items() threw an error. I got the stacktrace here:
(most recent call last): File "watch_script.py", line 202, in refresh toogoodtogo() File "watch_script.py", line 125, in toogoodtogo api_response = tgtg_client.get_items( File "/usr/local/lib/python3.8/dist-packages/tgtg/__init__.py", line 256, in get_items raise TgtgAPIError(response.status_code, response.content) tgtg.exceptions.TgtgAPIError: (403, b'<html><head><title>apptoogoodtogo.com</title><style>#cmsg{animation: A 1.5s;}@keyframes A{0%{opacity:0;}99%{opacity:0;}100%{opacity:1;}}</style></head><body style="margin:0"><p id="cmsg">Please enable JS and disable any ad blocker</p><script>var dd={\'cid\':\'AHrlqAAAAAMA58IS9zhH2CoBKgNAAAA1CARkr1j__p-j6w==\',\'hsh\':\'1D42C2CA6131C526E09F294FE96F94\',\'t\':\'fe\',\'r\':\'b\',\'s\':35558,\'e\':\'81ff72d7c0ffba785a14aa2c40cf1bb50b63107950d1e97eb5faea0ed2f840eb\',\'host\':\'geo.captcha-delivery.com\'}</script><script src="https://ct.captcha-delivery.com/c.js"></script></body></html>\n')

My guess is that the refresh token expires after some time, so that you cannot generate new access tokens.
For me it happend at 8:30pm (GMT+2) after about 4 hours of continous running.

ahivert commented

@svove have you tried to reduce the access_token_lifetime like I said here => #176 (comment) ?

How many seconds do you wait between each get_items calls ?
Is it working if you restart your script ?

svove commented

@ahivert intially no. now i have restartet with access_token_lifetime = 60.
Between each get_items call is also 60 seconds.

I cant say for restarting the script now. I have restarted and its been up and running for 2 hours.
I guess i'll have to wait and see what happens in a few hours.

dijkstrar commented

@ahivert intially no. now i have restartet with access_token_lifetime = 60. Between each get_items call is also 60 seconds.

I cant say for restarting the script now. I have restarted and its been up and running for 2 hours. I guess i'll have to wait and see what happens in a few hours.

Do you have any update?

nickhir commented

What I am doing right now is the following. I have another python script which calls the "real" script, i.e. the script which interacts with tgtg. It looks something like this and works like a charm. Its been running for weeks without any problem.

import subprocess
import time

while True:
    with open("/home/pi/projects/tgtg_access/log.txt", "w") as log:
        print("starting subprocess")
        script_process =  subprocess.Popen(["/usr/bin/python", "/home/pi/projects/tgtg_access/tgtg_work.py"],
                                           stdout=log, bufsize=1)
    time.sleep(60*60) # restart script every hour
    script_process.kill()