Newsletter store
niqdev opened this issue · 8 comments
niqdev commented
@juzim The ebook downloaded from the newsletter can't be uploaded to drive or stored on firebase and break the script, I would suggest at least to change the newsletter feature to optional i.e. add --newsletter or -n as parameter
[*] getting free ebook from newsletter
[*] fetching url... 200 | https://www.packtpub.com/packt/free-ebook/javascript-high-performance
[*] fetching url... 200 | https://www.packtpub.com/account/my-ebooks
[+] book successfully claimed
[-] downloading file from url: https://www.packtpub.com/ebook_download/20590/pdf
[################################] 12594/12594 - 00:00:00
[+] new download: XXX/packtpub-crawler/ebooks/Mastering_Javascript_High_Performance.pdf
[+] new file upload on Drive:
[+] uploading file...
-[+] updating file permissions...
- [path] XXX/packtpub-crawler/ebooks/Mastering_Javascript_High_Performance.pdf
[download_url] https://drive.google.com/uc?id=XXX&export=download
[name] Mastering_Javascript_High_Performance.pdf
[mime_type] application/pdf
[id] XXX
[-] skip store info: missing upload info
[-] <type 'exceptions.TypeError'> can't multiply sequence by non-int of type 'str' | spider.py@119
Traceback (most recent call last):
File "script/spider.py", line 119, in main
handleClaim(packpub, args, config, dir_path)
File "script/spider.py", line 55, in handleClaim
Notify(config, packpub.info, upload_info, args.notify).run()
File "XXX/github/packtpub-crawler/script/notify.py", line 30, in run
self.service.send()
File "XXX/packtpub-crawler/script/notification/gmail.py", line 98, in send
message = self.__prepare_message()
File "XXX/packtpub-crawler/script/notification/gmail.py", line 41, in __prepare_message
html *= "</ul>"
TypeError: can't multiply sequence by non-int of type 'str'
juzim commented
Isn't this about the Gmail notification? Looks like the file was uploaded.
It also gets the file information from the claim page, so nothing was
changed there.
juzim commented
Ah, it's a typo *= instead of +=. Sorry, I can fix it, bit it might take to
next email.
niqdev commented
yep, sorry, the problem is not the upload but with firebase.
Just checked now and the ebook is on drive.
By the way Firebase option depends on Drive option
niqdev commented
Just few questions about the newsletter
- is the spreadsheet updated every day now and the newsletter url added only if there is a "new" newsletter? if yes, can you update the code in the gist?
- there is a reason for why line 104 to line 132 are inside the main while you externalize handleClaim?
- can you just share me the link of the old newsletter, I need to test the script with firebase and I can't download the newsletter now
- do you agree that the newsletter should be another option? like
-wor--newsletter
juzim commented
1. The script runs daily at 6 and checks if there is a mail with the label.
If there is no new newsletter, it does nothing. If there is one and it has
a free book URL, the spreadsheet is updated, of there is no URL (like now)
the spreadsheet is cleared. The gist is up to date.
2. HandleClaim is only called after a book was successfully claimed. There
is no reason to call packtpub.py or handleClaim when there is no new
newsletter.
3. I really don't see why. There is no difference between those one the
daily books, just how there are distributed. Who would want the daily books
but not the additional ones?
juzim commented
Link to online version of newsletter:
http://us11.campaign-archive2.com/?u=693897ba2220b83ddb807103a&id=744b167654&e=3cbd391791
raw: https://gist.github.com/juzim/7ecab774e2435a86c5948ce99e456581
niqdev commented
Ok for all the points and thanks for the link, probably I didn't explain well point 2, I meant just refactor the method inside the main.
Anyway I will leave the issue open, if you agree, until we can't test again with the firebase option
niqdev commented
Already fixed. Obsolete