dynamodb-toolbox/dynamodb-toolbox

Type inferencing issues when using globally scoped Entity and Table

Closed this issue · 8 comments

NeoMopp commented

Hi, firstly thank you for all the hard work in developing this library, it is much appreciated.

I am working on upgrading some existing code to use v0.8.5 from v0.3.x, and have encountered issue with type inferencing (that was introduced in v0.4.x) when trying to define entities with a global scope.

Using version v0.3.x

export type entitySechema = {
  pk: string;
  sk: string;
  firstName: string;
  lastName: string;
};

export class myClass {
  myTable: Table;
  myEntity: Entity<{ [key in keyof entitySechema]: SchemaType }>;
  documentClient: DocumentClient;

  constructor() {
    const agent = new https.Agent({ keepAlive: true });
    const dynamodb = new DynamoDB({
      //Some other config here
      httpOptions: { agent },
    });
    this.documentClient = new DynamoDB.DocumentClient({ service: dynamodb });

    this.myTable = new Table({
      name: 'MyTable',
      partitionKey: 'pk',
      sortKey: 'sk',
      DocumentClient: this.documentClient,
    });

    this.myEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true, required: true },
        sk: { type: 'string', sortKey: true, hidden: true, required: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    });
  }

  async getItem(item: Partial<{ [key in keyof entitySechema] }>): Promise<entitySechema> {
    const { Item } = await this.myEntity.get(item);
    return Item;
  }

  async putItem(item: Partial<{ [key in keyof entitySechema] }>): Promise<void> {
    await this.myEntity.put(item);
  }
}

This works fine and there appears to be no issues with typing here for me, however trying to follow a similar pattern in v0.8.5 is challenging.

Using version v0.8.x

export type entitySechema = {
  pk: string;
  sk: string;
  firstName: string;
  lastName: string;
};

export class myClass {
  myTable: Table<string, string, string>;
  myEntity: Entity;
  documentClient: DynamoDBDocumentClient;

  constructor() {
    const marshallOptions = {
      convertEmptyValues: false,
      removeUndefinedValues: false,
      convertClassInstanceToMap: false,
    };

    const unmarshallOptions = {
      wrapNumbers: false,
    };

    const translateConfig = { marshallOptions, unmarshallOptions };

    const dynamodb = new DynamoDBClient({
      // some config
    });
    this.documentClient = DynamoDBDocumentClient.from(dynamodb, translateConfig);

    this.myTable = new Table({
      name: 'MyTable',
      partitionKey: 'pk',
      sortKey: 'sk',
      DocumentClient: this.documentClient,
    });

    this.myEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true },
        sk: { type: 'string', sortKey: true, hidden: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    } as const);
  }

  async getItem(item: Partial<{ [key in keyof entitySechema] }>): Promise<entitySechema> {
    const { Item } = await this.myEntity.get(item);
    return Item;
  }

  async putItem(item: Partial<{ [key in keyof entitySechema] }>): Promise<void> {
    await this.myEntity.put(item);
  }
}

This however does not function as I expect, I get the following error from this.myEntity.get(item) of TS2339: Property 'Item' does not exist on type 'GetCommandInput'. I assume this is down to type inferencing functioning differently here and if I was able to type myEntity then it might work. According to the documentation I should be able to use an overlay, so something like:

  async getItem(item: Partial<{ [key in keyof entitySechema] }>): Promise<entitySechema> {
    const { Item } = await this.myEntity.get<entitySechema>(item);
    return Item;
  }

But that gives me the same error. I am wondering if I am missing something here, if the Entity is scoped locally like so this will work:

  async localGetItem(item: entitySechema): Promise<entitySechema> {
    const localEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true },
        sk: { type: 'string', sortKey: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    } as const);
    const { Item } = await localEntity.get(item);
    return Item;
  }

Some guidance on this matter would be great if you have anything to share, forcing users to redefine an Entity in every function doesn't seem to be desirable in anyway and the documentation doesn't seem to have anything useful in this regard.

naorpeled commented

Hey @NeoMopp ,

EntityItem<typeof entitySchena> should do it, no need to define an inferface/a type in addition to the schema :)

Let me know if that doesn't work

NeoMopp commented

Hey @naorpeled, not 100% sure I understand the intended usage, but are you suggesting something like:

  async getItemA(item: entitySechema): Promise<entitySechema> {
    const { Item } = await this.myEntity.get<EntityItem<typeof item>>(item);
    return Item;
  }

That gives me some long error like: TS2344: Type 'entitySechema' does not satisfy the constraint 'Entity<string, { [x: string]: any; [x: number]: any; [x: symbol]: any; }, { [x: string]: any; [x: number]: any; [x: symbol]: any; }, TableDef, boolean, boolean, boolean, string, string, ... 7 more ..., { ...; }>'. Type 'entitySechema' is missing the following properties from type 'Entity<string, { [x: string]: any; [x: number]: any; [x: symbol]: any; }, { [x: string]: any; [x: number]: any; [x: symbol]: any; }, TableDef, boolean, boolean, boolean, string, string, ... 7 more ..., { ...; }>': _typesOnly, name, schema, _etAlias, and 36 more.

Whereas if I try something like:

  async getItemB(item: entitySechema): Promise<entitySechema> {
    const { Item } = await this.myEntity.get<EntityItem<typeof entitySechema>>(item);
    return Item;
  }

I get TS2693: 'entitySechema' only refers to a type, but is being used as a value here which I guess is accurate. If I try to define myEntity like this:

myEntity: Entity<EntityItem<typeof entitySechema>>;

I get: TS2589: Type instantiation is excessively deep and possibly infinite., as well as the previous error. Might be missing what you're suggesting though.

naorpeled commented

Hey @NeoMopp,
EntityItem is a utility type that exposes the entity item's type,
but because your SK is hidden, it's a bit problematic to use it as is atm.

I'd recommend doing the following as a temp solution, will probably expose a util type that includes hidden fields in the next release.

Let me know if you have any questions/thoughts and sorry for the delayed response :)

export class myClass {
  // I'd recommend not to explicitly set the type here, it collides with the type inference
  myTable;
  myEntity;
  documentClient: DynamoDBDocumentClient;

  constructor() {
    const marshallOptions = {
      convertEmptyValues: false,
      removeUndefinedValues: false,
      convertClassInstanceToMap: false,
    };

    const unmarshallOptions = {
      wrapNumbers: false,
    };

    const translateConfig = { marshallOptions, unmarshallOptions };

    const dynamodb = new DynamoDBClient({
      // some config
    });
    this.documentClient = DynamoDBDocumentClient.from(dynamodb, translateConfig);

    this.myTable = new Table({
      name: 'MyTable',
      partitionKey: 'pk',
      sortKey: 'sk',
      DocumentClient: this.documentClient,
    });

    this.myEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true },
        sk: { type: 'string', sortKey: true, hidden: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    } as const);
  }

  async getItem(item: Pick<typeof this.myEntity['_typesOnly']['_item'], 'pk' | 'sk'>): Promise<EntityItem<typeof this.myEntity> | undefined> {

    const { Item } = await this.myEntity.get(item);
    return Item;
  }

  async putItem(item: typeof this.myEntity['_typesOnly']['_item']): Promise<void> {
    await this.myEntity.put(item);
  }
}
NeoMopp commented

Hi @naorpeled, thanks for your response.

I've tried your suggestion but I still continue to get: TS2589: Type instantiation is excessively deep and possibly infinite on Promise<EntityItem<typeof this.myEntity> | undefined>

I've played around myself with this and have gotten the following that seems to work:

export type entitySchema = {
  pk: string;
  sk: string;
  firstName: string;
  lastName: string;
};

export class myClass {
  myTable: Table<string, string, string>;
  myEntity: Entity<'MyEntityName', entitySchema, unknown, Table<string, string, string>>;
  documentClient: DynamoDBDocumentClient;

  constructor() {
    const marshallOptions = {
      convertEmptyValues: false,
      removeUndefinedValues: false,
      convertClassInstanceToMap: false,
    };

    const unmarshallOptions = {
      wrapNumbers: false,
    };

    const translateConfig = { marshallOptions, unmarshallOptions };

    const dynamodb = new DynamoDBClient({
      // some config
    });
    this.documentClient = DynamoDBDocumentClient.from(dynamodb, translateConfig);

    this.myTable = new Table({
      name: 'MyTable',
      partitionKey: 'pk',
      sortKey: 'sk',
      DocumentClient: this.documentClient,
    });

    this.myEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true },
        sk: { type: 'string', sortKey: true, hidden: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    } as const);
  }

  async getItem(item: Partial<{ [key in keyof entitySchema] }>): Promise<entitySchema> {
    const { Item } = await this.myEntity.get<entitySchema>(item);
    return Item;
  }

  async getItemAlt(item: Partial<{ [key in keyof entitySchema] }>): Promise<entitySchema> {
    const { Item } = await this.myEntity.get(item);
    return Item;
  }

  async putItem(item: entitySchema): Promise<void> {
    await this.myEntity.put(item);
  }
}

Ideally I'd like to remain in keeping with my pre-existing implementation (similar to the code in my initial post) and return something with a more "concrete" type as this makes other areas of the code somewhat simpler. But if being so explicit about the type is in direct conflict with the type inferencing moving forward I guess I'll need an alternative solution.

naorpeled commented

Hi @naorpeled, thanks for your response.

I've tried your suggestion but I still continue to get: TS2589: Type instantiation is excessively deep and possibly infinite on Promise<EntityItem<typeof this.myEntity> | undefined>

I've played around myself with this and have gotten the following that seems to work:

export type entitySchema = {
  pk: string;
  sk: string;
  firstName: string;
  lastName: string;
};

export class myClass {
  myTable: Table<string, string, string>;
  myEntity: Entity<'MyEntityName', entitySchema, unknown, Table<string, string, string>>;
  documentClient: DynamoDBDocumentClient;

  constructor() {
    const marshallOptions = {
      convertEmptyValues: false,
      removeUndefinedValues: false,
      convertClassInstanceToMap: false,
    };

    const unmarshallOptions = {
      wrapNumbers: false,
    };

    const translateConfig = { marshallOptions, unmarshallOptions };

    const dynamodb = new DynamoDBClient({
      // some config
    });
    this.documentClient = DynamoDBDocumentClient.from(dynamodb, translateConfig);

    this.myTable = new Table({
      name: 'MyTable',
      partitionKey: 'pk',
      sortKey: 'sk',
      DocumentClient: this.documentClient,
    });

    this.myEntity = new Entity({
      name: 'MyEntity',
      attributes: {
        pk: { type: 'string', partitionKey: true },
        sk: { type: 'string', sortKey: true, hidden: true },
        firstName: { type: 'string', required: true },
        lastName: { type: 'string', required: true },
      },
      table: this.myTable,
    } as const);
  }

  async getItem(item: Partial<{ [key in keyof entitySchema] }>): Promise<entitySchema> {
    const { Item } = await this.myEntity.get<entitySchema>(item);
    return Item;
  }

  async getItemAlt(item: Partial<{ [key in keyof entitySchema] }>): Promise<entitySchema> {
    const { Item } = await this.myEntity.get(item);
    return Item;
  }

  async putItem(item: entitySchema): Promise<void> {
    await this.myEntity.put(item);
  }
}

Ideally I'd like to remain in keeping with my pre-existing implementation (similar to the code in my initial post) and return something with a more "concrete" type as this makes other areas of the code somewhat simpler. But if being so explicit about the type is in direct conflict with the type inferencing moving forward I guess I'll need an alternative solution.

I see,
sorry for the misdirections, not sure why I didn't get the same error when I tried it, not sure I'd be able to further investigate until the weekend 😢

I'd recommend trying out our v1 beta, Thomas wrote a blog post about it here.
It doesn't give you the same syntax as mentioned above but it has a great DX imho.

NeoMopp commented

No worries @naorpeled, thanks for responding, I'll continue to use the existing solution we have and have a look to see if we can utilise v1 when I get a chance later in the week 🙂

  • ❤️1
vespertilian commented

For anyone else who ends up here:

See also #631

Summary:

To use EntityItem you need to not use the "overlay feature (Don't set types on the entity directly, let them be infered)" and set your tsconfig.json to strict. That's what I needed to do anyway.

  • ❤️1
naorpeled commented

Marking this as resolved for now,
feel free to ping me if you need anything.