Harbinger Serverless Price Feed Signer

About

harbinger-signer is a signer for the Harbinger oracle system. harbinger-signer is a Serverless Framework application written in Typescript and deployed to Amazon Web Services. To get started with Harbinger, visit the main documentation.

Cryptographic signatures are generated by Amazon Key Management Service, a highly service that uses hardware security modules (HSMs). The app is extensible to produce feeds from other APIs and to use other signing solutions besides Amazon KMS. See 'Customizing Functionality' below.

This library provides functionality for signing a Harbinger price feed. Users interested in posting prices might also be interested in Harbinger CLI and Harbinger Poster which provide both command line and Serverless methods of posting signed price data on-chain. Developers of new Harbinger components may be interested in harbinger-lib.

Introduction

The Harbinger Tezos price oracle consists of two main components:

A signer that retrieves price candles from an exchange and cryptographically signs them.
A poster that retrieves the signed price feed from a signer and sends the signed data to a smart contract on the Tezos network.

This project is a reference price feed signer that demonstrates how to retrieve prices from multiple exchanges including Coinbase Pro, Binance, Gemini, and OKEx, and sign them with a private key that is securely stored inside Amazon Key Management Service. It is a Serverless Framework application. Once deployed, there is no need to spend any time thinking about or managing servers.

Setup Instructions

In order to setup the Serverless application, you'll need to perform the following setup tasks first:

Install the AWS CLI on your system.
Create an AWS access key and configure the AWS CLI by running the aws configure command.
Login to the AWS console with an account that has the ability to create KMS keys and SSM parameters, and grant permissions to use them. An admin role will work best.
Be sure to select the correct region that you would like to deploy to. The serverless.yml file in this repository is set to use the eu-west-1 (Ireland) region, but you can easily edit this file and select a different region if you like. The important thing is to ensure that the region you select in the console is the same region that is specified in the Serverless configuration file.
In the AWS console, select the KMS service:

Click the "Create key" button:

Select Asymmetric under Key type, then select Sign and verify under Key usage, then select ECC_SECG_P256K1 under Key spec, and click the Next button:

On the next page, input an Alias for the key and an optional Description, then click the Next button:

The next page is where you can define key administrators. There is no need to change any settings on this page, unless you would like to give additional IAM users or roles administrative permissions to the key. Click the Next button to continue:

This page is where you can define additional IAM users or roles that have key usage permissions. There is no need to change any settings on this page, unless you would like to give additional IAM users or roles usage permissions for the key. Click the Next button to continue:

Click Next to accept the default key policy, which only grants access to the root user. We'll edit this policy later to give the Serverless application rights to sign with the key.

Finally, click Finish and you should see a Success message similar to the following:

Copy the KMS key ID to your clipboard, or save it somewhere, then, launch Systems Manager from the Services section of the console (top left):

Select Parameter Store on the left navigation bar:

Click the Create Parameter button in the top left:

Name the parameter /tezos/signer-kms-key-id and give it an optional description. Leave the other settings at default (Standard tier, String type, Data type text), and paste or enter the KMS key ID you saved in step 13 as the value, without quotes or any surrounding characters, then click the Create parameter button:

Coinbase Pro API Key Setup

The following steps are only required if you are planning on using the Coinbase Pro API as a price data provider. Other exchanges don't currently require creating an API key to view their price data. If you are using another exchange besides Coinbase Pro, skip to step 25.

Access your Coinbase Pro API key settings either with the link or by accessing your profile menu in the top right:

Click the New API Key button in the top right:

Give the API key a nickname, check the box for View permissions only, and either save the passphrase somewhere secure or replace the default random passphrase with a strong passphrase of your choice, then click the Create API Key button:

If you have 2-factor authentication enabled, you'll need to go through the 2-step verification, then click the Add an API Key button:

Store the API Secret in a secure place, then click the Done button:

Now you should see the View key that you just created, and you'll need to copy the API key itself and store it somewhere for the next steps. You can click the API key itself to copy it to your clipboard:

Create another parameter named /tezos/coinbase-pro-api-key and give it an optional description. This parameter should be of type SecureString, but you can leave the rest of the settings at their defaults, and input your Coinbase Pro API key (with view permissions) as the value, then click the Create parameter button:

Create two more parameters, one named /tezos/coinbase-pro-api-passphrase and the second one named /tezos/coinbase-pro-api-secret with the values that you saved previously in steps 19 and 21. These should both be of type SecureString as well.

Deploying the Serverless Application

Clone this repository to your local system, install all NPM dependencies by typing npm i inside the repository directory, then type sls deploy --stage {{ stage }} (where stage is coinbase, binance, gemini, or okex) to deploy the application. If all goes well, you should see output similar to this. You'll want to save the two endpoints for use later.

Now, navigate back to KMS in the AWS console, and click on the Customer Managed Key you created earlier to modify the key policy. Click the button that says Switch to Policy View:

Now, click the button that says Edit:

Now we'll need to modify the key policy in order to enable the IAM role that the Serverless application will execute with to use the key for signing operations. You'll need to insert an additional JSON element or section into the Statement array. The section you'll need to insert is highlighted in the screenshot below. Don't forget to separate each statement with a comma. Here is the code you'll need to insert:

   	,
       {
           "Sid": "Allow use of the key for digital signing",
           "Effect": "Allow",
           "Principal": {
               "AWS": "arn:aws:iam::{{ AWS Account ID }}:role/harbinger-signer-{{ Exchange }}-{{ AWS Region }}-lambdaRole"
           },
           "Action": [
               "kms:Sign",
               "kms:Verify",
               "kms:GetPublicKey"
           ],
           "Resource": "*"
       }

Important Note: You must replace the 3 sections of the JSON in each statement that have {{ }} (double curly braces) surrounding them with the appropriate information. This string should also have no spaces in it.

AWS Account ID - This is your 12-digit numeric AWS account ID. If you're not sure what it is, look at the root policy above the one we are adding, and you should be able to copy it from that ARN
Exchange - This is the string coinbase, binance, gemini, or okex (all lower case) depending on which signer you are deploying
Region - This is the AWS region you are deploying to, such as eu-west-1

Determining the Tezos address of the KMS key used to generate signatures

Congratulations, you've just deployed a Serverless application that will automatically sign Coinbase Pro or Binance prices for the markets/order books you choose. Here is how to determine the Tezos public key (sppk...) for the private key that is used to sign the prices:

Curl the info endpoint that is displayed when you ran the last step (sls deploy) and it should output the sppk... address. You will need to include an x-api-key header that is set to the API key that was output by the previous sls deploy command. Here is the full command:

curl --silent -H 'x-api-key: {{ your API key }}' https://{{ your API gateway }}.execute-api.eu-west-1.amazonaws.com/binance/info

If you get a {"message": "Internal server error"} instead, you should check your Lambda logs inside the AWS console to see what went wrong. Most likely you have either not created all of the Systems Manager parameters correctly or the KMS key policy is not 100% correct. You should see output like this:

Customizing Functionality

OracleService is a pluggable service that can handle all serverless requests for the signer. It is initialized with an object conforming to the Signer interface, an object conforming to the CandleProvider interface and a list of assets to sign.

End users can customize this library with custom signers and candle providers.

Custom Assets

An assets list is configured in serverless.yml. This list can be customized to any set of assets.

Custom Candle Providers

An object conforming to the CandleProvider interface can retrieve Candle objects from an external feed. Candle Providers are injected into the OracleService via constructor.

Harbinger-Signer has several CandleProviders built in:

BinanceCandleProvider: Provides candles from the Binance API.
CoinbaseCandleProvider: Provides candles from the Coinbase Pro API.
GeminiCandleProvider: Provides candles from the Gemini API.
OkexCandleProvider: Provides candles from the OKEx API.

Custom Signers

An object conforming to the Signer interface can sign bytes and provide a public key. Signers are injected into OracleService via constructor.

Harbinger-Signer has one signer built in, AwsSigner which wraps calls to an AWS KMS Service.

Credits

Harbinger is written and maintained by Luke Youngblood and Keefer Taylor.

tacoinfra/harbinger-signer