Queue worker error: The file reference has expired and is no longer valid
Closed this issue · 14 comments
Hello, if I starts a download job about 20 and more files, a few will download correctly but the most will not start downloading. There are only a many empty *.tdd files.
Installed on a Synology, using docker.
89.07 % (1858600960 / 2086666240), 90.3 % (1884291072 / 2086666240), 89.57 % (1869086720 / 2086666240), 90.82 % (1895301120 / 2086666240), 91.58 % (1911029760 / 2086666240), 92.16 % (1923088384 / 2086666240), 92.91 % (1938817024 / 2086666240), 93.39 % (1948778496 / 2086666240), 94.14 % (1964507136 / 2086666240), Queue worker error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest), Downloading file Hyperspin AIO Windows Promo Disk.zip.085 (2086666240 bytes), Queue worker error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest), Downloading file Hyperspin AIO Windows Promo Disk.zip.094 (2086666240 bytes), Queue worker error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest), Downloading file Hyperspin AIO Windows Promo Disk.zip.100 (2086666240 bytes), Queue worker error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest),
By the way, I didn't send the files with timer on the channel 😉
Well, that message comes from Telegram. If Telegram says that file does not exists, I can not do anything.
Hmm, but the file is there. And this message: [WARNING/2021-05-14 12:50:23,307]telethon.client.users:Telegram is having internal issues HistoryGetFailedError: Fetching of history failed (caused by GetChannelDifferenceRequest)
What happens if you try to download only one of those 'dissapeared' files again? (without reuploading it, obviously)
If I add the same files without uploading, then it worked for some files. Yesterday it worked the very good in the evening. A added only 20 files at once.
Maybe is it a problem that I use the API hash for 2 bots? There is another one (smarttvtelegram) but with different IP (macvlan) running on the Synology.
Or maybe sometimes the CPU is by 100 %, because of indexing a lot of pictures?
For now it works good 🙂
Yes, it seems related to your installation. Pause picture indexing and retry dowloading.
Yes, it worked. It was the CPU overload.
I came across the same issue today. In my case, there are dozens of files of 1~2GiB size each. The first dozen downloads were successful, while the latter ones encountered the following errors:
Downloading file ... (2010007000 bytes)
Error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest)
Queue worker error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest)
I think the problem is that the file reference in event.message gets expired after some time (several hours?). Maybe we can try to retrieve the message every time by its id before downloading it or when the specific FileReferenceExpiredError so that the file reference wouldn't be stale.
Have you tried to resend the expired file to the daemon? Just to confirm it is still in Telegram servers.
Yes. Resending fixes everything. Now I just forward every five files to the channel at one time, wait for finishing, and forward another five again. So it shows that they are still in Telegram servers. But just "file reference has expired" after some time since they were sent the daemon, for unknown reasons.
Yes, i got also this error.
Downloading file xxx.rar (226545363 bytes)
Error: The file reference has expired and is no longer valid or it belongs to self-destructing media and cannot be resent (caused by GetFileRequest)
telethon.client.users:Telegram is having internal issues TimeoutError: A timeout occurred while fetching data from the worker (caused by GetFileRequest)
I think that event object has expiration timeout (unknown for me). Resending is fixing this error. Error occure if queue more than 5 or 10 files (depends of network speed and size of file more than 1GB).
Please set status issue to Open.
I experimented with push to queue not event, but message linked with event. I did forward 15 big (around 4gb) files to channel. I'm waiting for download around 2 hours now, first 10 files did download and filelinks in queue are valid yet.
I'm planning to publish code in my repo and do pull request
Ha-ha, my waiting for end of download is not sucessed. Sometime after start downloading links are dead. I think it needs to read updated message to get updated info about file. Expe
Also if queue is full and large files fill memory out, we got OOM.