Unable to capture retweets?
Closed this issue · 3 comments
erikcas commented
Hi, first of all, thank you for this nice piece of work.
I noticed repeatedly, that the capturing fails on retweets. Is this a known issue?
I see this appearing in logging:
31-12-2021 00:28:03 DEBUG Starting new HTTP connection (1): localhost:48777
31-12-2021 00:28:03 DEBUG http://localhost:48777 "POST /session HTTP/1.1" 200 750
31-12-2021 00:28:03 DEBUG Finished Request
31-12-2021 00:28:03 DEBUG POST http://localhost:48777/session/70629e524c65383da3765a60645b9bbb/url {"url": "https://twitter.com/dancalegria/status/1476696589444038658"}
31-12-2021 00:28:04 DEBUG http://localhost:48777 "POST /session/70629e524c65383da3765a60645b9bbb/url HTTP/1.1" 200 14
31-12-2021 00:28:04 DEBUG Finished Request
31-12-2021 00:28:04 DEBUG POST http://localhost:48777/session/70629e524c65383da3765a60645b9bbb/cookie {"cookie": {"name": "night_mode", "value": "0"}}
31-12-2021 00:28:04 DEBUG http://localhost:48777 "POST /session/70629e524c65383da3765a60645b9bbb/cookie HTTP/1.1" 200 14
31-12-2021 00:28:04 DEBUG Finished Request
31-12-2021 00:28:04 DEBUG POST http://localhost:48777/session/70629e524c65383da3765a60645b9bbb/url {"url": "https://twitter.com/dancalegria/status/1476696589444038658"}
31-12-2021 00:28:04 DEBUG http://localhost:48777 "POST /session/70629e524c65383da3765a60645b9bbb/url HTTP/1.1" 200 14
31-12-2021 00:28:04 DEBUG Finished Request
31-12-2021 00:28:11 DEBUG POST http://localhost:48777/session/70629e524c65383da3765a60645b9bbb/element {"using": "xpath", "value": "//a[translate(@href,'ABCDEFGHIJKLMNOPQRSTUVWXYZ','abcdefghijklmnopqrstuvwxyz')='/dancalegria/status/1476696589444038658']/ancestor::article/.."}
31-12-2021 00:28:12 DEBUG http://localhost:48777 "POST /session/70629e524c65383da3765a60645b9bbb/element HTTP/1.1" 404 995
31-12-2021 00:28:12 DEBUG Finished Request
31-12-2021 00:28:12 DEBUG no screenshot made. Error Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[translate(@href,'ABCDEFGHIJKLMNOPQRSTUVWXYZ','abcdefghijklmnopqrstuvwxyz')='/dancalegria/status/1476696589444038658']/ancestor::article/.."}
(Session info: headless chrome=96.0.4664.110)
Stacktrace:
#0 0x55fe67012ee3 <unknown>
#1 0x55fe66ae0608 <unknown>
#2 0x55fe66b16aa1 <unknown>
#3 0x55fe66b16c61 <unknown>
#4 0x55fe66b49714 <unknown>
#5 0x55fe66b3429d <unknown>
#6 0x55fe66b473bc <unknown>
#7 0x55fe66b34163 <unknown>
#8 0x55fe66b09bfc <unknown>
#9 0x55fe66b0ac05 <unknown>
#10 0x55fe67044baa <unknown>
#11 0x55fe6705a651 <unknown>
#12 0x55fe67045b05 <unknown>
#13 0x55fe6705ba68 <unknown>
#14 0x55fe6703a05f <unknown>
#15 0x55fe67076818 <unknown>
#16 0x55fe67076998 <unknown>
#17 0x55fe67091eed <unknown>
#18 0x7f15a4ca9609 <unknown>
I am using ubuntu20.04, python3.10.
This is the script that is used to capture: https://github.com/erikcas/scriptjes/blob/stream/capture.py
I am not very familiar with the usage of xpath stuff, maybe it is time to do some reading ;)
xacnio commented
Thanks for your report. It's fixed. Please update the module to 0.0.11.
erikcas commented
going to test and report back
erikcas commented
All good now! Thanks!