the platform is not working on google app_engine
thinksource opened this issue · 12 comments
I try to build a google app_engine include your code as additional libs.
currently google app_engine have only libs:
https://cloud.google.com/appengine/docs/python/tools/libraries27
I add:
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'lib'))
basedir = os.path.abspath(os.path.dirname(__file__))
to make the lib directory as my external libs.
I added some libs oauth2client, dateutil, simplejson, and bigquery
File "C:\Users\sheng\PycharmProjects\helloboard\main.py", line 104, in get
client=bigquery.get_client(project_id, service_account=service_email, private_key_file=key, readonly=True)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\bigquery\client.py", line 85, in get_client
readonly=readonly)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\bigquery\client.py", line 99, in _get_bq_service
credentials = _credentials()(service_account, private_key, scope=scope)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\oauth2client\util.py", line 135, in positional_wrapper
return wrapped(*args, **kwargs)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\oauth2client\client.py", line 1454, in __init__
_RequireCryptoOrDie()
File "C:\Users\sheng\PycharmProjects\helloboard\lib\oauth2client\client.py", line 1408, in _RequireCryptoOrDie
raise CryptoUnavailableError('No crypto library available')
CryptoUnavailableError: No crypto library available
the error is no crypto libary, I try to add pycrypto, but the pycrypto contain lots of c++ or code libraries.
I do not think it will run on could when do you know the library do not contain the other language library for crypto?
or you can use other way to invoke oauth2client
You can't add pycrypto as an external lib, you need to enable it on your appspot via app.yaml: https://cloud.google.com/appengine/docs/python/tools/libraries27
Note that you may need to convert the .p12 key file to a .pem to work on app engine. See this issue for more information.
That said, the recommended way to use this on App Engine is to not use a private key at all but instead to use AppAssertionCredentials
.
from bigquery import get_client
from bigquery import BIGQUERY_SCOPE
from oauth2client.appengine import AppAssertionCredentials
credentials = AppAssertionCredentials(scope=BIGQUERY_SCOPE)
client = get_client('my-project', credentials=credentials)
can you tell me the import password for convert my key.p12 to key.pem.
when google generate the key.p12 it do not give me password
The password defaults to 'notasecret'
Still have error :
Traceback (most recent call last):
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "C:\Users\sheng\PycharmProjects\helloboard\main.py", line 107, in get
results=client.get_query_rows(job_id)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\bigquery\client.py", line 252, in get_query_rows
limit=limit)
File "C:\Users\sheng\PycharmProjects\helloboard\lib\bigquery\client.py", line 962, in _get_query_results
timeoutMs=0).execute()
File "C:\Users\sheng\PycharmProjects\helloboard\lib\apiclient\http.py", line 292, in execute
raise HttpError(resp, content, self.uri)
HttpError: <HttpError 400 when requesting https://www.googleapis.com/bigquery/v2/projects/foretribebigquery/queries/job__xvopSV8jCsxehtLRG6lfOfe2Y0?startIndex=None&alt=json&maxResults=0&timeoutMs=0 returned "Invalid unsigned long value: 'None'.">
Can you post the code you're using which gives this error? Looks like a bad request is happening.
Specifically, the problem is startIndex=None
.
The error is 0 ms second, do you know how to set the second?
I think that is the timeoutMs=0 problem. do you know where to set this time?
INFO 2015-01-29 02:11:32,084 client.py:529] Attempting refresh to obtain initial access_token
INFO 2015-01-29 02:11:32,279 client.py:771] Refreshing access_token
INFO 2015-01-29 02:11:39,181 discovery.py:522] URL being requested: https://www.googleapis.com/bigquery/v2/projects/foretribebigquery/queries?alt=json
WARNING 2015-01-29 02:11:39,250 util.py:132] new_request() takes at most 1 positional argument (2 given)
INFO 2015-01-29 02:11:43,282 discovery.py:522] URL being requested: https://www.googleapis.com/bigquery/v2/projects/foretribebigquery/queries/job_QqtZ34Vlmz3tI3iuICFgQqeNduE?maxResults=0&alt=json&startIndex=0&timeoutMs=0
WARNING 2015-01-29 02:11:43,282 util.py:132] new_request() takes at most 1 positional argument (2 given)
INFO 2015-01-29 02:11:45,499 discovery.py:522] URL being requested: https://www.googleapis.com/bigquery/v2/projects/foretribebigquery/queries/job_QqtZ34Vlmz3tI3iuICFgQqeNduE?maxResults=0&alt=json&startIndex=None&timeoutMs=0
WARNING 2015-01-29 02:11:45,499 util.py:132] new_request() takes at most 1 positional argument (2 given)
ERROR 2015-01-29 02:11:46,950 webapp2.py:1552] <HttpError 400 when requesting https://www.googleapis.com/bigquery/v2/projects/foretribebigquery/queries/job_QqtZ34Vlmz3tI3iuICFgQqeNduE?maxResults=0&alt=json&startIndex=None&timeoutMs=0 returned "Invalid unsigned long value: 'None'.">
I use the basic solution of your example:
QUERY = "SELECT * FROM [testdata.zip_code] LIMIT 1000"
class JsonHandler(webapp2.RequestHandler):
def get(self):
client=bigquery.get_client(project_id, service_account=service_email, private_key_file=key, readonly=True)
job_id, _results = client.query(QUERY)
complete,row_count=client.check_job(job_id)
results=client.get_query_rows(job_id)
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(json.dump(results))
What happens if you try:
client.get_query_rows(job_id, offset=0)
it works, but as the maxResult=0, it just give me 0 result, can you tell me how to set the maxResult variable?
The limit
kwarg should control how many results are returned.
On Wed, Jan 28, 2015, 9:07 PM thinksource notifications@github.com wrote:
it works, but as the maxResult=0, it just give me 0 result, can you tell
me how to set the maxResult variable?—
Reply to this email directly or view it on GitHub
#43 (comment)
.
Thank you it works