liberty-askew/luminesce-examples

luminesce-examples (WIP)

This repo contains reference examples for Luminence.

This project is a WIP.

branch	status
master
master

Run using Docker

You can run with docker as follows.

Build the image:

docker build . -t luminesce-examples-runner

Run a container by passing auth credentials as environment variables:

docker run \
-e FBN_LUSID_API_URL=<API_URL> \
-e FBN_PASSWORD=<PASSWORD> \
-e FBN_APP_NAME=<APP_NAME> \
-e FBN_USERNAME=<USERNAME> \
-e FBN_CLIENT_SECRET=<CLIENT_SECRET> \
-e FBN_CLIENT_ID=<CLIENT_ID> \
-e FBN_TOKEN_URL=<TOKEN_URL> \
-e FBN_DRIVE_API_URL=<DRIVE_URL> \
-e FBN_LUMI_API_URL=<LUMI_URL> \
luminesce-examples-runner -start_dir=examples/drive

Running locally

You can run one or more of these examples locally by using the --secrets and --start_dir parameters.

For example, to run the Drive examples only:

python runner/run.py --secrets=secrets/secrets.json --start_dir=examples/drive

You can also choose to keep the sample Drive files created by the runner:

python runner/run.py --secrets=secrets/secrets.json --start_dir=examples/drive --keepfiles

List of examples

💡 The files in a directory are numbered if they need to be run in order 💡

System

• Append inline properties to system configuration
• Error handling details to file
• Iif and case when statements
• Load one cell of data to table by delimiters
• Pivot data
• To utc and iso
• Using datetimes

For loops with cross apply

• 1 create instrument upsert view
• 2 loop over csv file

Reconcile instruments

• 1 create instrument properties
• 2 setup instrument with properties
• 3 load instruments into source a
• 4 load instruments into source b
• 5 run reconciliation

Check for missing instrument fields

• 1 create instrument properties
• 2 load instruments into lusid
• 3 check for missing instrument fields

Check for price outliers

• 0 create instrument property definitions
• 1 setup instruments with properties
• 2 upload quotes
• 3 create iqr checker view
• 4 create price outlier view
• 5 run price outlier view

View management

• 1 create view with no params
• 2 fetch sql used to create view
• 3 current content of view
• 4 show view history
• 5 show view dependencies
• 6 show dependencies on view
• 7 delete a view
• 8 run table of query strings

File orchestration

• 1 files for testing
• 2 create error file
• 3 create logger view
• 4 create file orchestration view
• 5 run file load

Transactions

• Upload transactions from csv
• Upload transactions from excel
• Upload transactions from txt
• Upload transactions from xml

Corporate actions source

• Create corporate action source
• Read corporate action source

Instruments

• Query instruments
• Upload bond instruments
• Upload equity instruments
• Upload future instruments
• Upload fx forward
• Upload simple instruments
• Upload term deposit instruments

Reference portfolios

• 1 upload instruments
• 2 create reference portfolio
• 3 upload constituents

Run a reconciliation

• 1 create transaction portfolio
• 2 create instruments
• 3 create holdings
• 4 create reconciliation view
• 5 run recon with notifications

Quotes

• Upload fx quotes

Portfolios

• Create transaction portfolio

Properties

• Create instrument properties
• Create properties from csv

Returns

• 1 create transaction portfolio
• 2 upload portfolio returns
• 3 calculate aggregate returns

Relationships

• 1 create properties
• 2 upsert instrument properties
• 3 create legal entities
• 4 create portfolios
• 5 assign lei to portfolio
• 6 create relationship definition
• 7 call back custodians

Run valuation

• 1 create instruments
• 2 create portfolio
• 3 upload quotes
• 4 upload transactions
• 5 run simple valuation
• 6 upload external valuations
• 7 run valuation with srs
• Readme.md

Holdings

• 1 create transaction portfolio
• 2 create instruments
• 3 set holdings
• 4 adjust holdings
• 5 cancel holdings

Run a recon holdings in different scopes

• 1 create transaction portfolios in two scopes
• 2 create instruments
• 3 upload abor transactions
• 4 upload ibor holdings
• 5 create reconciliation view
• 6 run recon workflow
• 7 run recon with generic reconciliation provider

Statistical functions

• Fuzzy search two files

Drive

• Create and move file in drive
• Create log file on error
• Read a file from drive
• Read an excel file from drive
• Save data into drive
• Unformatted string to table

Pdf generation

• 1 load data from drive
• 2 generate pdf

Horizon

• Query company data from gleif

Insights

• Count of requests per lusid method

Automated testing

We run automated tests on the SQL files in this project via GitHub Actions. The configurtion for these tests live in the .github/workflows directory.

Many of our tests require setup data. To create this data, there is a process where the testing runner will search for a _data directory wherever it finds .sql files. Then, two things happen:

1. If there are data files in the _data directory, the runner will upload these to a luminesce-examples folder in LUSID Drive
2. If there is a setup.py file in this directory, the runner will run the setup.py file. We use this setup.py file to configure recipes and other configurations we don't want to setup via Luminesce.

Sample structure below:

upload_equity_instruments.sql
upload_bond_instruments.sql
_data
    eq_instruments.csv
    instruments.txt
    setup.py

⚠️ This file is generated, any direct edits will be lost. For instructions on how to generate the file, see docgen.