dynamodb-toolbox/dynamodb-toolbox

Version 0.9.x starting to give ESM related errors

Closed this issue · 6 comments

kr1p70n1c commented

When our project's nyc mocha executes the error appears:

Unknown file extension ".ts"

AFAIK this is related to using CJS and ESM in the same environment.

Indeed I notice that with the latest version there are CJS and ESM related changes.

I can not use this import statement:

import { Entity, Table } from 'dynamodb-toolbox/dist/cjs';

It results in the error:

Package subpath './dist/cjs' is not defined by "exports"

The documentation of the project makes no mention of anyting related to:

  • CJS, or
  • ESM

This begs the question: How to include the CJS artifacts from the dist directory if they are not exported.

Does anybody else have this issue?

For now I will downgrade to 0.8.x again to enable our project's test cases to execute.

naorpeled commented

@antstanley could you please take a look?
(sorry for bothering you again, unfortunately I don't have enough time to take a look at it atm 😞)

antstanley commented

@naorpeled no problem!

@kr1p70n1c dynamodb-toolbox has been updated to use conditional exports to detect whether to serve the ES module version of the code vs the CommonJS module version of the code.

in package.json line 7 is the exports property.

"exports": {
    ".": {
      "import": {
        "types": "./dist/esm/index.d.ts",
        "default": "./dist/esm/index.js"
      },
      "require": {
        "types": "./dist/cjs/index.d.ts",
        "default": "./dist/cjs/index.js"
      }
    }
  },

It adds a root export with two conditions, one for require and another for import each with default and types properties. If a your code uses CommonJS or transpiles to CommonJS (including if you're using Import statements in your TypeScript) it will use the values under require. If your code uses ES modules, then it'll use the values under import.

How it decides will be based on whether you have type property in your package.json set to module for ESM or commonjs for CJS. Alternatively by file extension with .mjs or .mts (in TS) for ESM, and .cjs and .cts (in TS) for CJS.

One issue with switching to exports is that your can't just ad-hoc use a subpath. It has to be explicitly defined under exports. So for you to use the line below

import { Entity, Table } from 'dynamodb-toolbox/dist/cjs';

then the exports property in package.json would need to be updated to below...

"exports": {
    ".": {
      "import": {
        "types": "./dist/esm/index.d.ts",
        "default": "./dist/esm/index.js"
      },
      "require": {
        "types": "./dist/cjs/index.d.ts",
        "default": "./dist/cjs/index.js"
      },
    },
    "./dist/cjs": {
      "types": "./dist/cjs/index.d.ts",
      "default": "./dist/cjs/index.js"
    },
  },

This adds an explicit export for ./dist/cjs that points to ./dist/cjs/index.js.

Some detail on how Conditional Exports work here https://nodejs.org/api/packages.html#conditional-exports

Saying that, your existing CommonJS code should just work. You shouldn't need to import from the sub-path. I'm assuming you're writing TS? Can you update the type field in your package.json to commonjs, and make sure your tsconfig.json has the following properties to ensure your TS config knows it's transpiling to commonjs.

"compilerOptions": {
    "module": "CommonJs",
    "moduleResolution": "Node",
    "esModuleInterop": true,
  }

From looking at your error, it looks like its actually a problem with ts-node which is what mocha uses under the hood for TypeScript, and that error message is a common one in ts-node, as ts-node is confused at to which module system to use, so doesn't transpile and attempts to run the .ts file and not the transpiled .js.

It's a very common error, with a ton of posts how to solve it. The tldr is remove "type": "module" from your package.json. My recommendation is be more explicit, and set it to "type": "commonjs". Also make sure you're running the latest versions of TypeScript and ts-node.

Let me know if you come right. Happy to help.

kr1p70n1c commented

Hi there,

Thank you so much for the detailed response.

I have however tried numerous options and angles already before logging the issue, eg:

  • install latest ts-node
  • update package.json accordingly
  • added esm settings for ts-node to package.json
  • changes in tsconfig.json

to no avail. Only fix I have is to downgrade dynamodb-toolbox to the previous minor version again.

Here is my tsconfig.json FWIW:

{
    "compilerOptions": {
        "experimentalDecorators": true,
        "lib": ["ES2022"],
        "moduleResolution": "node",
        "noImplicitAny": true,
        "strict": true,
        "strictNullChecks": true,
        "removeComments": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "noImplicitReturns": true,
        "noFallthroughCasesInSwitch": true,
        "sourceMap": true,
        "esModuleInterop": true,
        "resolveJsonModule": true,
        "target": "ES2022",
        "module": "commonjs",
        "outDir": "dist",
        "typeRoots": [ "./custom-types", "./node_modules/@types"]
    },
    "exclude": [
        "node_modules"
    ]
}

I'm not sure if the lib and target options are fine like that or not.

I appreciate you helping with this 🙏

djghokie commented

Not sure if this is the same issue, but I'm having ESM related errors too:

const { Table } = require('dynamodb-toolbox');

causes
"Error [ERR_REQUIRE_ESM]: require() of ES Module dynamodb-toolbox/dist/cjs/index.js ... not supported"

Node says

index.js is treated as an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which declares all .js files in that package scope as ES modules. Instead rename index.js to end in .cjs, change the requiring code to use dynamic import() which is available in all CommonJS modules, or change "type": "module" to "type": "commonjs"

antstanley commented

@kr1p70n1c @djghokie I think you might have a similar problem, and it could be something missed when setting up dual module support.

Basically Node has two ways of telling if a file is CommonJS or ESM. First is by the "type": "module" property in package.json and second is by the file extension (.mjs, .cjs).

TypeScript doesn't allow changing file extensions when transpiling. It always outputs .js files. Additional to this, if you want to correctly transpile TS to ESM JS the project needs to be an ESM project (ie have "type": "module" in package.json), which is what we've done here. This means by default all the code shipped is interpreted as ESM. We need to override this for the CommonJS files.

One way would be to make all the CommonJS files to have a .cjs extension. The other is to place a package.json file at the base of the dist/cjs with "type": "commonjs" which effectively tells Node that all files in that folder and sub-folders are CommonJS files.

As TypeScript doesn't transpile to custom extensions, I went the package.json route. After looking at the shipped code it looks like that sub-package.json file wasn't shipped, due to not including it in the files property of the root package.json.

I've made the change by adding the sub-package.json files with correct "type" settings to the files property anD have submitted a PR (#619) to resolve it. I've tested, and made it available on my latest test package at @antstanley/dynamodb-toolbox if you want to validate for yourself before waiting for the PR to be merge.

I was able to reproduce @djghokie's issue, and this resolves it.

@kr1p70n1c I haven't been able to reproduce yours, but that's more down to you having a slightly more complex config. Saying that, as ts-node (and hence mocha) were always going to assume the dynamodb-toolbox files were ESM, when they should have been assuming CommonJS. This should resolve that. Please test and let me know if there are any other issues. Also happy to jump on a call to debug quickly.

Sorry about all of this!

kr1p70n1c commented

@antstanley v0.9.2 fixes my issue! Thank you so much 🙏 🍾