neardb: A TypeScript repository from rimenes

NearDB is a simple database that leverages cloud infrastructure like document storage and CDN to deliver an inexpensive unbelievably scalable document database optimized for reads and perfect for edge applications.

Motivation

While working on building edge applications for higher performance and lower latency there is a need store persistent data also on edge.

There are multiple distributed database solutions but they are very involved and costly while having a much lower global footprint than a CDN.

The idea came up to leverage ubiquitous and mature infrastructure like cloud storage and CDNs to deliver a persistent data solution from the edge.

Use with Edge Apps/Functions

Zeit Now - Global Serverless Deployments
Cloudflare Workers - Serverless applications on Cloudflare's global network
Lambda@Edge - Run Lambda functions on CloudFront
Fly.io - Javascript at the edge

Works with the following for database storage

AWS S3
Google Cloud Storage
Digital Ocean Spaces
Minio
any S3 API compatible storage service

Who is this for

This is perfect for persistent data that is read frequently and needs to be avaialble on the edge application to deliver dynamic data while keeping the costs low. Some examples of the best uses are:

Key-value
Configuration
Cached data

Probably not for you if

You plan on using this as your primary database for an app that has complex data needs.
You need transactions. (I have some ideas on how to accomplish this, but its currently not implemented.)
Do many writes/sec in the same document. Reads are incredibly efficient, fast and inexpensive; however, writes are always at the origin.

Installing

npm install neardb
# or
yarn add neardb

Usage

import Neardb from 'neardb'

const config = {
    database: 'bucketName'
    //...
    // These options will change by the final release to provide more control

    // database: string
    // indices?: boolean // experimental feature
    // cdn?: {
    //     url: string
    //     headers?: {
    //     [key: string]: string
    //     }
    // }
    // cacheExpiration?: number
    // storage?: {
    //     endpoint: string
    //     useSSL?: boolean
    //     accessKeyId?: string
    //     secretAccessKey?: string
    //     signatureVersion?: string
    //     s3ForcePathStyle?: boolean
    // }
}

const neardb = NearDB.database(config);

References

You are able to store the reference of a collection or document, and use the reference when interacting with them.

const statesRef = nearDB.collection('states');

const nyRef = nearDB.collection('states').doc('ny')

Add a Document

By using set for document creation allows you to set the document id

nearDB.collection('states').doc('ny').set({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City'
})

By calling add on the collection a document id is auto-generated

nearDB.collection('states').add({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City'
})

Update a Document

By using set if the document does not exist it will create it. If it does exist you can use set to overwrite the whole document.

nearDB.collection('states').doc('ny').set({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City',
    eastCoast: true
})

If you wish to update fields within a document without overwriting all the data you should use update

nearDB.collection('states').doc('ny').update({
    eastCoast: true
})

To delete a value without overwriting the whole document you the following helper constant

nearDB.collection('states').doc('ny').update({
    eastCoast: NearDB.field.deleteValue
})

Delete a Document

By using delete the whole document will be deleted from the bucket

nearDB.collection('states').doc('ny').delete()

Get a Document

You can get the content of a single document by using get

nearDB.collection('states').doc('ny').get()

There a few options that you are able to pass on get depending where you want to get the data from.

By default get will try to retrieve the document the following way.

Get local data if it exists and has not expired
If CDN is configured will get from there
If there is no local cache and CDN is not configured, it will get from the origin.

There are a few options you are able to pass to force where you get the data from.

const options = {
    // Gets data from origin even if 
    // there is local cache and a cdn configured
    source: 'origin' 

    // Gets data from edge even if 
    // there is local cache and a cdn configured
    // source: 'edge' 
}
nearDB.collection('states').doc('ny').get(options)

Running the tests

npm run test

Dependencies

aws sdk - AWS SDK for JavaScript in the browser and node.js
axios - Promise based HTTP client for the browser and node.js

Inspiration

The design of NearDBs API is heavily inspired by Firestore.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

rimenes/neardb