How to import the library to create the RequestProxy() object
vnjk opened this issue · 2 comments
Hi, as above:
I have tried importing RequestProxy but with no success.
from project.http.requests.proxy.requestProxy import RequestProxy
Following the style of:
from project.http.requests.parsers.UrlParser import UrlParser
The error returned is:
Traceback (most recent call last):
File "crawler_copy.py", line 18, in
requests_proxy = RequestProxy()
File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/proxy/requestProxy.py", line 21, in init
self.userAgent = UserAgentManager()
File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/useragent/userAgent.py", line 8, in init
self.useragents = self.load_user_agents(self.agent_file)
File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/useragent/userAgent.py", line 16, in load_user_agents
with open(useragentsfile, 'rb') as uaf:
IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3->py2.7.egg/project/http/requests/useragent/../data/user_agents.txt'
How can I import the library correctly? Please advise. Thank you
nevermind, figured out the problem. cheers
@vnjk I fixed an issure in the setup.py file and now the easiest way would be to create a new project and import the pypi library (using requirements.txt for example) http-request-randomizer==0.0.5
After create a python file and test the code below:
import time
from http.requests.proxy.requestProxy import RequestProxy
if __name__ == '__main__':
print "Hello"
start = time.time()
req_proxy = RequestProxy()
print "Initialization took: {0} sec".format((time.time() - start))
print "Size : ", len(req_proxy.get_proxy_list())
print " ALL = ", req_proxy.get_proxy_list()
test_url = 'http://icanhazip.com'
while True:
start = time.time()
request = req_proxy.generate_proxied_request(test_url)
print "Proxied Request Took: {0} sec => Status: {1}".format((time.time() - start), request.__str__())
if request is not None:
print "\t Response: ip={0}".format(request.text)
print "Proxy List Size: ", len(req_proxy.get_proxy_list())
print"-> Going to sleep.."
time.sleep(10)
Thanks for reporting and sorry for the delay!