Speed.cd not downloading
aldariz opened this issue · 34 comments
Hi,
I've been using this fork for a while and its great.
I recently noticed that the speed.cd provider is no longer downloading episodes and was wondering if the cause could be investigated.
My Log:
2016-04-15 09:56:46 INFO SEARCHQUEUE-BACKLOG-261690 :: Searching for stuff we need from The Americans (2013) season 4
2016-04-15 09:56:46 DEBUG SEARCHQUEUE-BACKLOG-261690 :: sickbeard.db: SELECT * FROM tv_episodes WHERE showid = ? AND season = ? with args [261690, 4]
2016-04-15 09:56:46 DEBUG SEARCHQUEUE-BACKLOG-261690 :: cache.db: SELECT DISTINCT show_name FROM scene_exceptions WHERE tvdb_id = ? with args [261690]
2016-04-15 09:56:46 INFO SEARCHQUEUE-BACKLOG-261690 :: [Speed] Performing Search: The.Americans.2013 S04E05
2016-04-15 09:56:46 INFO SEARCHQUEUE-BACKLOG-261690 :: [Speed] parseResults() No results found.
2016-04-15 09:56:46 INFO SEARCHQUEUE-BACKLOG-261690 :: [Speed] Performing Search: The.Americans S04E05
Thanks in advance for the help.
@aldariz , I don't have a valid account atm to check this, i've requested one via their website... unless you want to email an invite or some account credentials...
@aldariz , cheers mate, have fixed and tested , commit id e4df2ed, have to wait on @junalmeida to merge into code, but you can cherry pick using that id for fix right away...
@junalmeida closed.
@VeNoMouS Hope you are well. I was wondering if you have time, if could have a look at this as it does not seem to be working again. Many Thanks
LOL... man, i just finished the torrentday issue :P , yea I'll try look at it this weekend, i cant remember if i have a login or not, I may need to get a login from you for testing is all.
Comitted f0b94dc, it'll action under PR #296 which has a couple PR's waiting for @junalmeida to accept.
@VeNoMouS I was wondering if you could have another look as it does not seem to be working again. Thanks in advance.
@aldariz ok.. with in 2 seconds i already see a problem
Login form on there site has changed..
<form method="post" action="takeElogin.php" id="loginform">
vs my code... in _doLogin()
response = self.session.post(self.url + "takelogin.php", data=login_params,
timeout=30, verify=False)
I'm unsure if I still have a login to this site to check the rest of the scraping. I'll have to have a hunt for one, If I'm unable to find one, am I able to borrow yours?
@VeNoMouS , It say you are already a member so should an access with your email address :)
Thanks again.
@aldariz ok sorry about the delay, its 11:45 here atm, I'm at work, I've looked , there is an issue with the title regex, they have put html tags around the season and episode number, but that is easy to bypass, I've already written a fix on the fly
>>> for torrent in re.compile("<td class=\"lft\"><div><a href=\"\/t\/.*?\" class=\"torrent\" id=\"(?P<id>.*?)\"><b>(?P<title>.*?)</b></a>", re.MULTILINE|re.DOTALL).finditer(res.content):
... print torrent.group('title')
...
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E05</span> Istanbul 480p x264-mSD
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E04</span> Burgundy 480p x264-mSD
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E02</span> Sicily 480p x264-mSD
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E03</span> Borderlands 480p x264-mSD
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E06</span> Orlando XviD-AFG
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E07</span> Taiwan XviD-AFG
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E08</span> China XviD-AFG
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E05</span> Istanbul XviD-AFG
Huangs World 2016 <span class='hO'>S01</span><span class='hG'>E02</span> Sicily XviD-AFG
Defining a pre-compiled regex..
TAG_RE = re.compile(r'<[^>]+>')
then simply stripping
TAG_RE.sub('',torrent.group('title'))
>>> for torrent in re.compile("<td class=\"lft\"><div><a href=\"\/t\/.*?\" class=\"torrent\" id=\"(?P<id>.*?)\"><b>(?P<title>.*?)</b></a>", re.MULTILINE|re.DOTALL).finditer(res.content):
... print TAG_RE.sub('',torrent.group('title'))
...
Huangs World 2016 S01E05 Istanbul 480p x264-mSD
Huangs World 2016 S01E04 Burgundy 480p x264-mSD
Huangs World 2016 S01E02 Sicily 480p x264-mSD
Huangs World 2016 S01E03 Borderlands 480p x264-mSD
Huangs World 2016 S01E06 Orlando XviD-AFG
Huangs World 2016 S01E07 Taiwan XviD-AFG
Huangs World 2016 S01E08 China XviD-AFG
Huangs World 2016 S01E05 Istanbul XviD-AFG
Huangs World 2016 S01E02 Sicily XviD-AFG
Huangs World 2016 S01E03 Borderlands XviD-AFG
I'll do a PR shortly.
Actually, I didnt realise... (cause i wasn't paying attention), I already allowed for this in the original list construction...
torrent.group('title').replace("<span class='hO'>","").replace("<span class='hG'>","").replace("</span>","").replace('.',' ')
So technically all that needs to be changed is the login URI, But I think the regex strip is a more cleaner solution
@aldariz I've committed 496d32f , how ever my branch is much ahead of @junalmeida as I haven't done a push to him in a some time , he's had alot going on in his life, I'll PR my branch to him, I can ask him to merge, but I won't be able to give you an ETA.
@junalmeida , @aldariz PR #304
@aldariz I also must state, I didn't get a chance to test that, so please let me know if you have any issues...
@VeNoMouS I tried the updates but get the following:
2016-08-09 08:33:22 SEARCHQUEUE-RSS-SEARCH :: Error while searching Speed, skipping: cannot concatenate 'str' and 'NoneType' objects
2016-08-09 08:32:49 SEARCHQUEUE-MANUAL-SEARCH :: [Speed] _getPassKey() Failed to scrape authentication parameters for rss.
2016-08-09 08:32:49 SEARCHQUEUE-MANUAL-SEARCH :: Error while searching Speed, skipping: [Speed] _doLogin() Could not extract rssHash info... aborting.
Thanks for trying so quickly thou
Hrrrm I'm not getting that issue.... @aldariz
are you able to change the except and log in _getPassKey() on 216,217 to
except Exception as e:
logger.log("[" + self.name + "] " + self.funcName() + " Failed to scrape authentication parameters for rss. Error: " + str(e),logger.ERROR)
And restart sickbeard... its caused an exception... but I'm unsure as to why.
ok @aldariz i managed to replicate the issue, im getting a site error wanting me to verify my account etc..
<td>
<h3 id="username">XXXXXXXX</h3>
</td>
</tr>
</table>
</div>
<div class="txtWarp">
<div class="tx">
<img src="/V3/img/.gif" class="txtIcon critical" title="Note">
<div class="txt">We could not recognize your account properly. Mind if we double check that? It's for your own security.</div>
</div>
</div>
<div>
<form action="checkpass.php" method="post" autocomplete="off">
<div class="infoHead infoHover togNext"><b>Security Question</b></div>
<div class="infoBody">
<dl>
<dt class="label">Question:</dt>
<dd><h3>XXXXXXX</h3></dd>
<dt class="label">Secret Answer:</dt>
<dd><input type="password" name="answer" maxlength="100" value="" /></dd>
<dt class="label"> </dt>
<dt class="label">Password:</dt>
<dd><input type="password" name="passcheck" /></dd>
<dt class="label"> </dt>
<dd>
<label class="uiButton uiButtoncolor"><button type="submit" class="button">Continue</button></label>
<input type="hidden" name="a" value="1" />
</dd>
</dl>
</div>
</form>
<form action="checkpass.php" method="post" autocomplete="off">
<div class="infoHead infoHover togNext"><b>Request a login token by Email</b></div>
<div class="infoBody Hidden">
<dl>
<dt class="label"><b>Email me a link to reset my IP lock settings.</b></dt>
<dd><input type="radio" name="a" value="2" checked="1" />XXXXXXXX</dd>
<dt class="label">Type the full email Address:</dt>
<dd><input type="text" name="femail" maxlength="17" value="" /></dd>
<dt class="label"> </dt>
<dd>
<label class="uiButton uiButtoncolor"><button type="submit" class="button">Continue</button></label>
</dd>
</dl>
</div>
</form>
@aldariz ok.... it seems to 'whitelist' ip's that you access from... I only got that error on my colo as i hadn't logged in there before... i used lynx and logged in and answered the security questions... after that my test code worked fine... so if your using a colo, login once from there or if its your home ... login via browser one time to get it to whitelist... the only way i can bypass that is on the providers config page, and add a request box that the user put in their security answer , and have it automatically post if prompted for that error... , What I should do is actually put a trap into the _doLogin() and at least log that issue to alert the user.
@aldariz I also discovered this.... under the profile -> security -> Geo Locking section , unsure if its related... but it basicly matches the issue that's going on... well for me , i don't know if you are using a seed box.
if this is the case, I will be forced to add the "security answer" under search providers config section for speed.cd
@VeNoMouS, thanks for trying. I don't use a seedbox I made the change on line 216&217, but I get the following:
AA
AATypeError: '_sre.SRE_Pattern' object is not callable
AA item = (self.remove_tags(torrent.group('title')), self.url + "download.php?torrent=" + torrent.group('id'))
AA File "/Applications/Sick-Beard-torrentProviders/sickbeard/providers/speed.py", line 150, in parseResults
AA return self.parseResults(searchUrl)
AA File "/Applications/Sick-Beard-torrentProviders/sickbeard/providers/speed.py", line 141, in _doSearch
AA itemList += self._doSearch(cur_string)
AA File "/Applications/Sick-Beard-torrentProviders/sickbeard/providers/generic.py", line 283, in findSeasonResults
AA curResults = curProvider.findSeasonResults(show, season)
AA File "/Applications/Sick-Beard-torrentProviders/sickbeard/search.py", line 391, in findSeason
2016-08-09 20:55:19 ERROR SEARCHQUEUE-BACKLOG-289590 :: Error while searching Speed, skipping: '_sre.SRE_Pattern' object is not callable
2016-08-09 20:55:19 INFO SEARCHQUEUE-BACKLOG-289590 :: [Speed] _getPassKey() Attempting to acquire RSS info
2016-08-09 20:55:19 INFO SEARCHQUEUE-BACKLOG-289590 :: [Speed] Attempting to Login
@VeNoMouS don't know if I missed a cherry pick request but I'm getting this error when I restart
File "SickBeard.py", line 46, in
import sickbeard
File "/Applications/Sick-Beard-torrentProviders/sickbeard/init.py", line 41, in
from providers import speed, revolutiontt
File "/Applications/Sick-Beard-torrentProviders/sickbeard/providers/speed.py", line 199
+ sys.tracebacklimit = 0 # raise exception to sickbeard but hide the stack trace
@aldariz just grab the raw :P and replace it...https://raw.githubusercontent.com/VeNoMouS/Sick-Beard/torrentProviders/sickbeard/providers/speed.py
hrrm actually its not fixed.. theres still a bug..
2016-08-10 10:21:25 SEARCHQUEUE-BACKLOG-270701 :: Error while searching Speed, skipping: '_sre.SRE_Pattern' object is not callable
2016-08-10 10:21:10 SEARCHQUEUE-BACKLOG-282994 :: Error while searching Speed, skipping: '_sre.SRE_Pattern' object is not callable
2016-08-10 10:20:49 SEARCHQUEUE-BACKLOG-269640 :: Error while searching Speed, skipping: '_sre.SRE_Pattern' object is not callable
Yea, I found the issue... god damn it, typo , missing the .sub() call on the regex call..