BrotliDecompress failed on some pages
Closed this issue · 3 comments
jbd7 commented
Describe the bug
Some pages monitor fail with Exception: BrotliDecompress failed
Version
Exact version in the top right area: 47.04
(but has been happening with previous versions too)
To Reproduce
- URL to watch: https://docs.n8n.io/reference/release-notes/
- Processor: Webpage Text/HTML, JSON and PDF changes
- Request: Basic fast Plaintext/HTTP Client (advanced options: none)
Steps to reproduce the behavior:
- Wait for changedetection to fetch the monitored page, or click on "Recheck"
- Refresh changedetection homepage
- See error
Exception: BrotliDecompress failed
Logs
Using docker logs -f <container id>
, I see the following:
2024-11-05 18:39:59.804 | DEBUG | changedetectionio.processors.text_json_diff.processor:run_changedetection:350 - Watch UUID f97ab8da-5d0c-4e3e-8b25-c58842da78fb content check - Previous MD5: 209cc19efa0f71bddc65bb9a6faa46d5, Fetched MD5 f01d5f462bc688efd381e9b7436399c6
2024-11-05 18:39:59.812 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:39:59.813 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:01.153 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
... [about 50 similar lines removed]
2024-11-05 18:40:05.882 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:05.906 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:05.929 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:05.953 | DEBUG | changedetectionio.model.Watch:history:184 - Reading watch history index for f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:05.954 | ERROR | changedetectionio.update_worker:run:479 - Exception reached processing watch UUID: f97ab8da-5d0c-4e3e-8b25-c58842da78fb
2024-11-05 18:40:05.955 | ERROR | changedetectionio.update_worker:run:480 - BrotliDecompress failed
2024-11-05 18:40:05.957 | DEBUG | changedetectionio.update_worker:run:580 - Watch f97ab8da-5d0c-4e3e-8b25-c58842da78fb done in 6.86s
2024-11-05 18:40:25.211 | INFO | changedetectionio.store:sync_to_json:384 - Saving JSON..
Expected behavior
- Page is successfully parsed and monitored
Installation
- Ubuntu 24..04 on a VPS
- Docker v 24.0.7
- ChangeDetection version 47.04
- Docker-compose
- image: ghcr.io/dgtlmoon/changedetection.io
- image: selenium/standalone-chrome:4
Additional context
- Happens on webpages that were successful in the past (incl. example)
- Tried adding dns 8.8.8.8 and 1.1.1.1 in the docker compose, no change in behavior
- I tried on another Brotli supported page,
https://apnews.com/projects/election-results-2024/
, for which https://tools.keycdn.com/brotli-test says "Brotli compression is supported.", like https://docs.n8n.io/reference/release-notes/- The page check is successful, and the docker logs show:
2024-11-05 18:59:53.670 | DEBUG | changedetectionio.processors.text_json_diff.processor:run_changedetection:350 - Watch UUID 781ca46a-28ff-4f18-bc71-695f481489a5 content check - Previous MD5: 09f737081a21384cf4d52a45df4e69a9, Fetched MD5 09f737081a21384cf4d52a45df4e69a9
2024-11-05 18:59:53.670 | DEBUG | changedetectionio.update_worker:run:580 - Watch 781ca46a-28ff-4f18-bc71-695f481489a5 done in 0.15s
- I tried adding
https://docs.n8n.io/
as a Watch, from the same domain as the page generating BrotliDecompress errors, and it is fetched successfully. - I tried checking for Brotli in the changedetection container:
sudo docker exec -it 2f61860d3526 /bin/bash
root@changedetection:/app# brotli --version
bash: brotli: command not found
root@changedetection:/app# apt list --installed | grep brotli
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
libbrotli1/now 1.0.9-2+b6 amd64 [installed,local]
root@changedetection:/app# pip3 list | grep brotli
[notice] A new release of pip is available: 24.0 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
dgtlmoon commented
hello, I cant reproduce this, is your disk full or something?
also the package name is Brotli
not brotli
root@c3920c7f594a:/app# pip3 list|grep -i brot
Brotli 1.1.0
dgtlmoon commented
hmm I think the snapshots in your datastore are corrupt somehow, clear the history for that watch and it will probably work again
jbd7 commented
Thanks and good tip, clearing the history did the job!