/dynamo-types

Typescript AWS DynamoDB ORM

Primary LanguageTypeScript

Build Status npm version Semantic Release enabled Renovate enabled

DynamoTypes

Typescript ORM of DynamoDB, written from scratch to fully support DynamoDB. Powering Vingle

Features

  1. Serialize / Deserialize DynamoDB record -> TS class object based on annotations.
  2. Table Configurations
    • CreateTable
      • Create secondary indexes (Both local / global)
      • Configure TTL
    • DropTable
  3. PrimaryKey
    • FullPrimaryKey (Hash, Range)
    • HashPrimaryKey (Hash)
  4. Indexes
    • Local, both hash and range key
    • Global, both hash and range key
  5. Attribute
    • Type Support (Number / String / Boolean / Array / Object / Buffer)
    • TimeToLive
  6. DAX Support
    • You can specify this by setting the connection of table.
  7. Optimized aws-sdk usage
    • aws-sdk has a serious problem of not reusing HTTP connection towards DynamoDB by default. check this issue
    • this could cause unbearable latency sometimes with showing > 100ms. it's more of an issue of NodeJS HTTP module but nevertheless, it has been optimized here by keep-alive Code
  8. AWS X-Ray support
    • XRay is serverless distributed tracing service. In order to log DynamoDB transaction into it, you also need to some sort of risk monkey-patching. Here you can turn it on by setting process.env.ENABLE_XRAY = "true"
  9. Testing
    • You can change the endpoint of DynamoDB by setting the environment variable or setting new connection, So you can install local-dynamo locally at setup endpoint to local. refer package.json for the detailed how-to

Also, dynamo-types let you overcome several limits that DynamoDB or the aws-sdk has.

  1. BatchWrite (batchDelete / batchPut) has a limit of a maximum of 25 items per request.
    • dynamo-types automatically splits given items into chunks of 25 and sends requests in parallel
  2. BatchGet has a limit of a maximum of 100 items per requests
    • dynamo-types automatically splits given keys to chunks of 100 and sends requests in parallel
  3. BatchGet doesn't keep the order of items as it is in input keys,
    • dynamo-types sort return items based on input keys
  4. BatchGet doesn't handle "missing items".
    • dynamo-types has "BatchGet" / "BatchGetFull"
      • BatchGet
        order items follow to keys, missing items are just missing. return type Promise<Array>
        so keys.legnth !== items.keys in this case
      • BatchGetFull
        order items follow to keys, fill missing items with "null". return type Promise<Array<Item | null>>
        so keys.length === items.keys always true

And most importantly, all of those queries regardless of whether it's from index or primary key, strongly typed. I mean what's the point of using typescript if not anyway?

Usage

  @Decorator.Table({ name: "prod-Card" })
  class Card extends Table {
    @Decorator.Attribute()
    public id: number;

    @Decorator.Attribute()
    public title: string;

    @Decorator.Attribute({ timeToLive: true })
    public expiresAt: number;

    @Decorator.FullPrimaryKey('id', 'title')
    static readonly primaryKey: Query.FullPrimaryKey<Card, number, string>;

    @Decorator.Writer()
    static readonly writer: Query.Writer<Card>;
  }

  // Create Table At DynamoDB
  await Card.createTable();

  // Drop Table At DynamoDB
  await Card.dropTable();


  // Creating Record
  const card = new Card();
  card.id = 100;
  card.title = "Title";
  //
  await Card.writer.put(card);
  // OR just
  await card.save();

  // Batch Put
  await Card.writer.batchPut([
    new Card(),
    new Card()
  ]);

  // Get Record
  await Card.primaryKey.get(100, "Title");

  // BatchGet
  // This array is strongly typed such as Array<[number, string]> so don't worry.
  await Card.primaryKey.batchGet([
    [100, "Title"],
    [200, "Title2"]
  ])

  // Query
  // Range key opreations are stringly typed. ([">=", T] | ["=", T] ...)
  await Card.primaryKey.query({
    hash: 100,
    range: [">=", "Title"]
  })

  // Delete record
  await card.delete()

  // Delete record only when it meets condition.
  // with this, you can avoid race condition such as somebody updating the record while you're trying to delete it
  await card.delete({
    condition: { title: Equal("Title") }
  });
  // when mismatch occurs, it raises "ConditionalCheckFailedException" error.

  // Likewise, update record only when it meets condition
  card.title = "New Title"
  await card.save({ condition: { title: "Title" } });
  // when mismatch occurs, it raises "ConditionalCheckFailedException" error.
import {
  Config,
  Decorator,
  Query,
  Table,
} from "dynamo-types";

@Decorator.Table({ name: `table_name` })
export class CardStat extends Table {
  @Decorator.HashPrimaryKey("card_id")
  public static readonly primaryKey: Query.HashPrimaryKey<CardStat, number>;

  @Decorator.Writer()
  public static readonly writer: Query.Writer<CardStat>;

  @Decorator.Attribute({ name: "card_id" })
  public cardId: number;

  @Decorator.Attribute({ name: "impressions_count" })
  public impressionsCount: number = 0;

  @Decorator.Attribute({ name: "shares" })
  public shares: number = 0;
}

TS Compiler Setting

DynamoTypes utilize reflect-metadata to read metadata (usually type of variables) from Typescript code. to do so, you must enable those options.

{
    "compilerOptions": {
        // other options..
        //
        "experimentalDecorators": true, // required
        "emitDecoratorMetadata": true // required
    }
}

Connection

DynamoDB supports 2 different kinds of connections. Plain connections to DynamoDB through HTTP, or through DAX. dynamo-types supports this by letting you create a separate connection for each table.

@Decorator.Table({ name: "prod-Card1", connection: new DAXConnection({ endpoints: ["dax-domain:8892"] }) })
class Card extends Table {
  @AttributeDecorator()
  public id: number;

  @AttributeDecorator()
  public title: string;

  @AttributeDecorator({ name: "complicated_field"})
  public complicatedField: string;

  @FullPrimaryKeyDecorator('id', 'title')
  static readonly primaryKey: Query.FullPrimaryKey<Card, number, string>;

  @WriterDecorator()
  static readonly writer: Query.Writer<Card>;
}

Then any query that is sent to the Card table will be sent through DAXConnection.

If you don't specify any connection, it automatically uses default connection, which is DynamoDBConnection.