This repository contains the core runtime used inside Scaleway Functions
's platform, built with Golang
, as well as language specific sub-runtimes.
This runtime is the Gateway to functions running in Scaleway's Cloud. Its goal is to provide a set of features optimized for Function As A Service
technologies, and make it easy for users to run their applications in the cloud.
A runtime
on Scaleway Serverless Platform is made of 2 components:
- Core Runtime written in Golang: Starts a HTTP server, handles incoming request, transforms data into event objects, handles authentication (users can deploy private functions), starts a sub-runtime and proxy request to the sub-runtime
- Sub-Runtime (Language specific code), Starts a HTTP server, receives requests from Core-Runtime containing "event", "context" and user's handler informations (handler file and exported function to execute). The sub-runtime is a bridge between the core-runtime and the user's code. We can not dynamically import and execute node.js code in Golang, so we have to create a bridge in node.js to handle this logic.
We decided to split the logic in 2 components because we wanted to handle the common logic (transforming HTTP requests in event/context, handling authentication...) in one place, instead of maintaining 5 different repositories (at this time), all doing the same logic. It makes it easier for us to maintain our runtimes, as sub-runtimes have a minimal task to fulfill: import and invoke user's code.
- Simple HTTP Webserver
- Manages Authentication (On Scaleway Serverless Platform, users can choose to deploy their functions
privately
and they will have to provide a valid JWT generated by our APIs to trigger the function, see the official documentation - Handles incoming requests, transforms data into events (and execution context of the function)
- Dynamically import and execute functions handlers, provided by end-users.
- Manages Handler's responses based on trigger-type (for example, HTTP requires function handlers to return an HTTP response while CRON jobs do not require any of it).
At the moment, we only support the following runtimes (see the corresponsing sub-runtimes):
- Node.js version 8, version 10 and version 14
- Python version 2.7 and version 3.7
- Golang version 1.11+
You can use this runtime on your own infrastructure, or on your local environment if you wish to contribute to its development.
In order to create a new runtime extending Scaleway's Core runtime, for example ruby
or dotnet
, you will have to respect certain criterias:
- Your runtime should create an HTTP server, listening on given
$SCW_UPSTREAM_PORT
environment variable, on interface127.0.0.1
(As both core-runtime and your sub-runtime will run in the same docker container, core-runtime will proxy requests to local network interface). - Listen for traffic on endpoint
POST /
(e.g.POST 127.0.0.1:$SCW_UPSTREAM_PORT
) - Manage incoming requests, with the following structure:
event
: Data for the event that triggered the function execution (in case of an HTTP request, it contains the request body, incoming headers, query parameters...)context
: Execution context of your functionhandlerPath
: Path to your Handler file (e.g. for handler located in/home/app/function/handler.js
:/home/app/function/handler
), you will have to handle dynamic import of your handler file (if dynamic language)handlerName
: Name of the exported function to use as a handler (e.g./home/app/function/handler.js
: module.exports.handle = ...,handlerName
ishandle
). Full Example of request body for an function invoked via HTTP Trigger:{ "event": { "body": {...request body}, "headers": {...request headers}, "httpMethod": "POST", "path": "/test", "pathParameters": {}, "queryStringParameters": { "query": "value" }, "multiValueHeaders": null, "multiValueQueryStringParameters": null, "stageVariables": null, "isBase64Encoded": false, "requestContext": { "eventType": "http", "httpMethod": "POST", "path": "/test", "stage": "dev", }, "resource": "http" }, "context": { "functionName": "myFunction", "memoryInMb": 128 }, "handlerPath": "/home/app/function/handler", "handlerName": "handle" }
- Send an HTTP response with the following structure (for HTTP Triggers):
body
: Response bodystatusCode
: Status Code for HTTP Response to the client invoking the functionheaders
: Map of headers (key: value) to send in HTTP Response Example of Response from custom runtimes (invoked via HTTP Trigger):{ "statusCode": 200, "body": { "key": "value" }, "headers": { "key": "value" } }
To start using Scaleway's core-runtime
, you'll need to go through multiple steps:
- Install Golang (version >= 1.11 as we are using
go mod
for our dependencies) - Download this repository (outside of your
$GOPATH
as we are using go mod). - Pre-requisites related to your custom runtime (for example, to run node.js runtime, you need to install node.js)
In order to create a custom runtime extending this core-runtime
with a dynamic language (such as Python, Node.js or Ruby for example), you will have to develop a sub-runtime script, in charge of:
- Starting an HTTP Server on port
$SCW_UPSTREAM_PORT
(provided via environment variable) on host$SCW_UPSTREAM_HOST
(e.g.127.0.0.1:8081
is the default for core-runtime's sub runtimes), handlingPOST
requests on/
. - Dynamically import function handler from given
handlerPath
andhandlerName
(see partHow to use this runtime
above). - Handle potential errors (while importing handler), and during handler execution, to format the error response properly (this may be useful, to provide good feedbacks on errors to your users).
- Send Back an HTTP response with handler's return result.
You may find examples of Node and Python custom runtimes (developped and maintained by Scaleway, used on Scaleway Functions platform).
In order to configure core-runtime
to use your "Language Specific Runtime", you will have to set different environment variables:
variable name | description |
---|---|
SCW_HANDLER_NAME | Exported function to use as a handler (e.g. handle for a node.js function with a module.exports.handle handler) |
SCW_HANDLER_PATH | Absolute path to your handler file (e.g. /home/app/function/handler or /home/app/function/handler.js ) |
SCW_RUNTIME_BINARY | Absolute path to the binary of the language you wish to use to execute your runtime (e.g. /usr/local/bin/node or /usr/local/bin/python ) |
SCW_RUNTIME_BRIDGE | Absolute Path to your custom-runtime entrypoint (e.g. /home/app/myruntime.js ) |
This Core-runtime will take care of executing $SCW_RUNTIME_BINARY $SCW_RUNTIME_BRIDGE
(e.g. /usr/local/bin/node /home/app/myruntime.js
) to start the sub-runtime HTTP server.
When the Core-runtime receives an HTTP request, it will transform it into a usable event structure
, create the context object
, and execute your sub-runtime code by sending an HTTP request to its HTTP server.
- I have a Node.js handler (see this example) (located in
/home/app/function/handler.js
) with our custom runtime defined here (located in/home/app/index.js
). - I configure my environment properly:
export SCW_RUNTIME_BINARY=/usr/local/bin/node export SCW_RUNTIME_BRIDGE=/home/app/index.js export SCW_HANDLER_PATH=/home/app/function/handler export SCW_HANDLER_NAME=handle export SCW_UPSTREAM_HOST=http://127.0.0.1 export SCW_UPSTREAM_PORT=8081 # PORT used by core runtime to serve HTTP server export PORT=8080 # For this example, we're setting function's authentication privacy to public export SCW_PUBLIC=true
- I can run my core-runtime:
sudo -s go run main.go
- I can trigger my function:
curl localhost:8080
```bash
export SCW_RUNTIME_BINARY=/bin/sh
export SCW_RUNTIME_BRIDGE=/runtimes/python3/bootup.sh
export SCW_HANDLER_PATH=/home/app/function/handler
export SCW_HANDLER_NAME=handle
export SCW_UPSTREAM_HOST=http://127.0.0.1
export SCW_UPSTREAM_PORT=8081
export PORT=8080
export SCW_PUBLIC=true
export SCW_HANDLER_IS_BINARY=false
```
In order to create a custom runtime extending this core-runtime
with a compiled language (such as Golang, C# for example), you will most likely have to develop a custom Library
that will wrap your function handler's code with the necessary logic described above.
As the Function Handler's code is compiled as a binary, you will have to include the runtime logic (creating server, handling incoming traffic and outgoing responses...) alonside your handler code.
You may find an example of a Golang custom runtime here (developed and maintained by Scaleway and running on Scaleway Serverless platform).
In order to configure core-runtime
to use your handler as a Binary (compiled code), you will have to set different environment variables:
SCW_HANDLER_IS_BINARY=true
this is important, as the core-runtime will initialize your runtime by running a command (for example/home/app/function/handler
if you compiled your dotnet program into ahandler
binary).SCW_HANDLER_PATH=absolute/path/to/binary
(for example/home/app/function/handler
if you compiled your program into ahandler
binary)
In this example, we are using the official Golang sub-runtime for Serverless Scaleway.
- I have a Golang Handler in a file
main.go
using my custom runtime described in this sample - I build a
handler
binary and output it to/home/app/function
directory:go build -o /home/app/function/handler main.go
- I configure my environment properly:
export SCW_HANDLER_PATH=/home/app/function/handler export SCW_HANDLER_IS_BINARY=true export SCW_UPSTREAM_PORT=8081 # PORT used by core runtime to serve HTTP server export PORT=8080 # For this example, we're setting function's authentication privacy to public export SCW_PUBLIC=true
- I can run my core-runtime binary:
go run main.go
- I can trigger my function:
curl localhost:8080
Everyone is free to contribute to this project by sending PRs or opening issues.
If you wish to integrate a new language to this runtime, feel free to either add the code and documentation in the runtimes directory. If you do not wish to code the runtime yourself, we would be happy to hear from you on either via Issues on this project, on by messages on our Community Slack Platform (channel #srvless-private-beta).
Refer to Issue templates.