In this tutorial, we are going to save a log to Datastore without deploying a server. A server-less log system! Think about how we can apply it to other services, we can pretty much do anything server-less!
Cloud Function is a light-weight solution to deploy your service, especially when your need is simple. Datastore can handle large amount of non-relational data with relatively low price. Most of all, both of them have an user-friendly web console.
Since Google Cloud Platform hasn't release the Cloud function's implementation of datastore, I decided to write a tutorial how to do it.
- Create Cloud Datastore
- Create a Google Cloud Platform Project
- Setup Cloud Datastore
- Obtain Datastore Credential
- Download Example Code
- Deploy Cloud Function
- Enable Cloud Function API
- Install Cloud SDK
- Install Cloud Function SDK
- Deploy
- Save a Log
- Take a Look at the Function
- Send a Request to Cloud Function
- Check the Log on Datastore
- Summary
- OS & APIs versions
-
Go to https://console.cloud.google.com/projectcreate.
If you don't have an account yet, you need to sign up. You might need your credit card info. But no worry, we are far from getting pass the free-tier limit.
-
Enter a project name, such as
cloud-function-datastore-tutorial
, create. -
Take down your project id.
-
Wait until the project is created. It would take about a minutes.
-
Go to the datasotre web console.
-
Create entity.
-
Select a region. For example:
asia-northeast-1
. -
Keep
Namespace
,Key identifier
as default.Kind
enteruser-log
. -
Add property.
Name: uid Type: Integer Value: 0 Check the "Index this property".
Click Done.
-
Add property.
Name: log Type: Text Value: "My very first log on datastore!"
Click Done.
-
Add property.
Name: time_create Type: Integer Value: 0 Check the "Index this property".
Click Done. Click Create.
This credential can let your code access datastore.
<img src="https://imgur.com/LGOSar2.png" width="600px"/>
* Click the bar on the top left Navigation menu on the [GCP console](https://console.cloud.google.com/datastore)
* Click `Credentials` in the `APIs & Services` section.
* Click `Create credentials`, `Service account key`
* Set a `Service account`. `Service account name` enter `datastoreuser`. `Role` select `Datastore`->`Cloud Datastore User`.
* Select JSON. Create. Download to your desktop. **Do not share it on the internet**
The credential looks like
```
{
"type": "service_account",
"project_id": "cloud-function-datastore-tutorial",
"private_key_id": "xxxxx",
"private_key": "xxxxx",
"client_email": "xxxxx",
"client_id": "xxxxx",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "xxxxx"
}
```
-
Download cloud-function-datastore-tutorial, you will see.
index.js
, which is the function we will later on deploy on cloud function.datastore-credential.json
, replace this file with the credential you obtained.README.md
package.json
, within it, the cloud datastore nodeJS library, which is installed in the file already.- Others are the cloud datastore nodeJS library.
-
Replace
datastore-credential.json
with the credential you obtained. -
Change the variable in index.js. If you forget your project id, click the project name on the top off the navigation bar. You will see a list of Name and ID.
const datastore = new Datastore({ projectId: 'YOUR_PROJECT_ID', keyFilename: 'YOUR_DATASTORE_CREDENTIAL_FILE_NAME.json' }); // change it to something like const datastore = new Datastore({ projectId: 'cloud-function-datastore-tutorial-123', keyFilename: 'cloud-function-datastore-tutorial-44b44d44b4bb.json' });
- Check out the Datastore official guide.
- If you need to query by single names (column), for example, uid, you can simply index the uid.
- But if you need to query or order by multiple names (column), you will need to setup
composite indexes
. Its a bit complecated, I will leave it to the next tutorial. Or check out the official doc. - If you want to delete the whole KIND (table), you need to remove all entities (rows).
- Click the bar on the top left Navigation menu on the GCP console
- Find
Cloud Functions
in theCOMPUTE
section. - Enable billing acount. You might need to setup the payment information. Don't worry, we are far from getting pass the free-tier limit.
- Enable API.
- Next, instead of creating cloud function on the web console, our code is a bit complicated, so we need to deploy our code by Cloud SDK.
Check out the cloud SDK guide if needed.
$ python -V
# Python 2.7.10
Download the sdk file on the offical guide.
Place it on the directory you want to install. For example: home/directory/
$ cd google-cloud-sdk
$ bash install.sh
# enter n
# enter y
# enter ENTER
$ sudo bin/gcloud init
The browser will show a login window. Login. Allow.
You are logged in as: [your.gmail@gmail.com].
Choose the project.
By the way, if you need to change the project
$ gcloud config set project <project-id>
# for example
$ gcloud config set project rpg-log-test
First, make sure the cloud SDK is installed.
$ gcloud components update && gcloud components install beta
Check out the offical cloud function sdk guide if needed.
# go to the example code directory
$ cd cloud-function-datastore-tutorial
$ gcloud beta functions deploy savelog --trigger-http --region asia-northeast1
You should see
Deploying function (may take a while - up to 2 minutes)...done.
availableMemoryMb: 256
entryPoint: rpglog
httpsTrigger:
url: https://asia-northeast1-cloud-function-datastore-tutorial.cloudfunctions.net/savelog
labels:
deployment-tool: cli-gcloud
name: projects/cloud-function-datastore-tutorial/locations/asia-northeast1/functions/savelog
runtime: nodejs6
serviceAccountEmail: cloud-function-datastore-tutorial@appspot.gserviceaccount.com
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-asia-northeast1-7158586f-ae4b-4f57-8e10-f1627de6d371/67772a3d-9745-439d-8b29-76f18a3252ff.zip?GoogleAccessId=service-858392701215@gcf-admin-robot.iam.gserviceaccount.com&Expires=1531637449&Signature=tIpithyM7%2BNzFYDGZ%2Bey8Awntg3%2BPyoitpVtfA%2Fszm4FZg49sfBCCkBHtjKVWNI7XZP2VLOGSp3wBfY4p82MhY7mwZ%2FgA4rPpAy2KTIC1KqHikI%2BYe3aPkXhZC9oUjtFDFaROC2pJnRCNTHafRNN%2F7lKZ1Sj8d4mvgEqSTBPeyUCK%2Fu%2FvgH4h3P39SYuoFUO%2BCTdUtlg5fonrZ3A83oyWgFcvOLriTjIM7W0LKdSmBtII5jnSKepmI84lJMb82q0CFLYpAfe04cHFtdS7%2BwnVslX%2FmRS%2B0HFZEJX%2FcJLKjcN8jbyADud64TGDeXUVpXu59pH4L5ihR5UXfosiXgR6g%3D%3D
status: ACTIVE
timeout: 60s
updateTime: '2018-07-15T06:21:40Z'
versionId: '1'
You can deploy again if you change something. For more info about deploy.
Go to the cloud function web console, you should see the function savelog
with a green check.
If you plan to use cloud function on your product, definitely worth a looks at its best practice.
The Cloud function will look for index.js then execute the function savelog
.
const Datastore = require('@google-cloud/datastore');
const datastore = new Datastore({
projectId: 'YOUR_PROJECT_ID',
keyFilename: 'YOUR_DATASTORE_CREDENTIAL_FILE_NAME.json'
});
const kindName = 'user-log';
exports.savelog = (req, res) => {
let uid = req.query.uid || req.body.uid || 0;
let log = req.query.log || req.body.log || '';
datastore
.save({
key: datastore.key(kindName),
data: {
log: log,
uid: datastore.int(uid),
time_create: datastore.int(Math.floor(new Date().getTime()/1000))
}
})
.catch(err => {
console.error('ERROR:', err);
res.status(200).send(err);
return;
});
res.status(200).send(log);
};
This function took two parameters. uid
and log
.
Send an HTTP request to from browser. This request will go to the index.js and send back an response.
https://asia-northeast1-cloud-function-datastore-tutorial.cloudfunctions.net/savelog?log=i_funcking_did_it&uid=1
https://asia-northeast1-cloud-function-datastore-tutorial.cloudfunctions.net/savelog
is the url endpoint of the cloud function. You can find the URL when you click the function, then Trigger.
-
Use
?
to seperate the endpoint URL and parameters. Use=
to asign values to parameters. And use&
to seperate each parameters and value pairs. -
For example, if you want to save
log
:i_funcking_did_it
with anuid
of1
First, you need the endpoint. https://asia-northeast1-cloud-function-datastore-tutorial.cloudfunctions.net/savelog Use ? to seperate parameters and URL endpoint. Asign i_funcking_did_it to log, log=i_funcking_did_it Use & to seperate parameters and value pairs. Asign 1 to uid, uid=1 Then you get https://asia-northeast1-cloud-function-datastore-tutorial.cloudfunctions.net/savelog?log=i_funcking_did_it&uid=1
We just saved a log to Datastore without deploying a server. Its exciting to think about what we can do next with Cloud Function and its server-less implementation. Anything!
Cloud Function: v1 (still in Beta)
Cloud Datasotre: v1
Cloud Datasotre NodeJS Library: 1.4.1
MacOS High Sierra 10.13.3
Google Cloud SDK 208.0.2
Google Cloud SDK beta 2018.06.22