Google Appengine SDK problems getting started on SDK
katzlbt opened this issue · 9 comments
From the documentation and examples I conclude that the library should work, or has worked on Appengine prior to integration of requests. I got it working in production (with lots of pain by installing urllib3 and investigating tons of outdated links and including the ssl loader). Instructions would help greatly (one requirement is a paid Appengine account for SSL support, etc.) However, I fail to get it working locally (localhost) finally with ConnectionError: ('Connection aborted.', error(13, 'Permission denied')). Who says that? Is that a local or remote message?
The first issue on GAE_SDK is the exception: "TypeError: must be _socket.socket, not socket"
Patching the sandbox on the SDK results in a permission denied error:
"Accept": "application/json", "User-Agent": self.user_agent
File "distlib/paypalrestsdk/api.py", line 161, in http_call
response = requests.request(method, url, proxies=self.proxies, *_kwargs)
File "distlib/requests/api.py", line 48, in request
return session.request(method=method, url=url, *_kwargs)
File "distlib/requests/sessions.py", line 451, in request
resp = self.send(prep, *_send_kwargs)
File "distlib/requests/sessions.py", line 557, in send
r = adapter.send(request, *_kwargs)
File "distlib/requests/adapters.py", line 407, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))
This traces to the following exception in requests:
except (ProtocolError, socket.error) as err:
INFO 2014-09-29 10:58:26,837 api.py:158] Request[POST]: https://api.sandbox.paypal.com/v1/oauth2/token
INFO 2014-09-29 10:58:26,849 connectionpool.py:695] Starting new HTTPS connection (1): api.sandbox.paypal.com
File "/Users/cat/repositories/web/parts/google_appengine/google/appengine/api/remote_socket/_remote_socket.py", line 1112, in getsockopt
raise _SystemExceptionFromAppError(e)
ProtocolError: ('Connection aborted.', error(13, 'Permission denied'))
Is this an error from the remote site or from the local system?
The connection is from a server running on localhost.
I found the following links, all of which add some insights and
http://stackoverflow.com/questions/23093201/permission-denied-error-when-opening-socket-to-apns-on-google-appengine-developm
http://stackoverflow.com/questions/16192916/importerror-no-module-named-ssl-with-dev-appserver-py-from-google-app-engine
http://stackoverflow.com/questions/23024801/send-ios-push-notifications-from-google-app-engine-python
https://code.google.com/p/googleappengine/issues/detail?id=9246
stripe/stripe-python#73 (payment solution stripe.com)
@katzlbt From an initial look, this seems to be more related to deploying an app on appengine that uses requests (or the sessions library of requests) as the http library. Have you raised this on the requests github repo? It would be interesting to see if other apps built on top of requests ran into similar issues with requests or the issue is unique to this library.
It is known that the authors of "requests" do not support google appengine and will not do so in future. requests works now on the live environment (only for billing enabled appengine apps), but not on the local SDK, which makes implementing a solution based on your library a nightmare on appengine, and basically untestable offline. Only uploading to the live server works.
If we could monkeypatch your library by replacing calls to requests with urlfetch calls this would solve our problems. A REST interface should not require socket() calls, this is too low level.
I see only one line in the Api using requests it should be easy to make this call or method use urlfetch on appengine api.py#L161
I succeeded in monkeypatching the library to work with Google Appengine even on the SDK by replacing requests with the designated solution on appengine for such things. Writing this patch took 1h time, significantly less than the day I spent making requests work in production.
import datetime
from google.appengine.api import urlfetch
def monkeypatched_http_call(self, url, method, **kwargs):
logging.info('Request[%s]: %s' % (method, url))
start_time = datetime.datetime.now()
payload = kwargs.get("data")
headers = kwargs.get("headers")
uf_response = urlfetch.fetch(url, method=method, payload=payload, headers=headers)
# requests.request(method, url, proxies=self.proxies, **kwargs)
class _Response(object):
def __init__(self, dict):
self.__dict__ = dict
# construct a request-like response
response = _Response({
'status_code' : uf_response.status_code,
'reason' : None,
'content' : uf_response.content,
'headers' : uf_response.headers,
})
duration = datetime.datetime.now() - start_time
logging.info('Response[%d]: %s, Duration: %s.%ss.' % (response.status_code, response.reason, duration.seconds, duration.microseconds))
debug_id = response.headers.get('PayPal-Debug-Id')
if debug_id:
logging.debug('debug_id: %s' % debug_id)
return self.handle_response(response, response.content.decode('utf-8'))
paypalrestsdk.Api.http_call = monkeypatched_http_call
Hey @katzlbt thanks for this solution! I am going to refer to this issue in the official README as potential solution for developers trying to deploy on google appengine.
@katzlbt Thanks so much for your input. Googled this discussion on another question of integrating Google App Engine and Google Cloud Storage via JSON API. Faced the same problem that 'requests' doesn't work with GCS. I've tried urfetch and it works fine.
Spent the last few hours banging my head on the wall about this issue...
@katzlbt and @obrizan - Thanks for sharing your experience in using urlfetch to get around the issue, but does anyone know another way to address the issue if swapping requests with urlfetch is NOT an option?
As others have stated, running the app in production works, but it does not work in sdk/development mode...
Use an old version of requests that works.
Can Python Requests library be used on Google App Engine?http://stackoverflow.com/questions/9604799/can-python-requests-library-be-used-on-google-app-engine
With requests toolbelt, requests now works in both production and development:
from requests_toolbelt.adapters import appengine
appengine.monkeypatch()
For more information, see this article.