cannot resolve esm-only modules with `import` condition and yarn pnp

Question

cannot resolve esm-only modules with `import` condition and yarn pnp

Closed this issue 6 months ago · 8 comments

Environment

Node 21.5.0, yarn 4.0.2, macOS 14.2.1 (23C71)

Reproduction

See the steps detailed here nksaraf/vinxi#43, vinxi fixed yarn berry compat by disabling jiti when not using a typescript vite.config.ts file, but it's still broken due jiti calling require.resolve with the wrong import conditions in the default setup (having a .ts config file)

Describe the bug

See also the linked issue above and run the reproduction steps

daniel@mmmmmmmmmm my-app % yarn dev
Error: Package subpath './lib/chunks' is not defined by "exports" in /Users/daniel/.yarn/berry/cache/vinxi-npm-0.0.54-94f4086925-10c0.zip/node_modules/vinxi/package.json imported from /private/tmp/solid-start-x/my-app/.yarn/__virtual__/@vinxi-server-functions-virtual-824b5314c0/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.0.51-9542aa8426-10c0.zip/node_modules/@vinxi/server-functions/
Require stack:
- /private/tmp/solid-start-x/my-app/.yarn/__virtual__/@vinxi-server-functions-virtual-824b5314c0/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.0.51-9542aa8426-10c0.zip/node_modules/@vinxi/server-functions/client.js
    at require$$0.Module._resolveFilename (/private/tmp/solid-start-x/my-app/.pnp.cjs:12657:13)
    at Function.resolve (node:internal/modules/helpers:187:19)
    at _resolve (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:251148)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:253746)
    at /private/tmp/solid-start-x/my-app/.yarn/__virtual__/@vinxi-server-functions-virtual-824b5314c0/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.0.51-9542aa8426-10c0.zip/node_modules/@vinxi/server-functions/client.js:3:15
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)
    at /private/tmp/solid-start-x/my-app/.yarn/__virtual__/@vinxi-server-functions-virtual-824b5314c0/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.0.51-9542aa8426-10c0.zip/node_modules/@vinxi/server-functions/plugin.js:3:15
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)

Additional context

No response

Logs

No response

Answer 1 · 2024-01-16T16:59:09.000Z

Investigating such issues with higher-order meta frameworks is time-consuming. I appreciate it if you can make a minimal reproduction (with jiti only). closing in the meantime.

Answer 2 · 2024-01-16T17:00:17.000Z

@pi0 will creating a minimal reproduction (with jiti only) lead to the issue being fixed in a timely manner?

Answer 3 · 2024-01-16T17:03:31.000Z

minimal reproduction it is a requirement to make valid issues.

priority is based on how common a feature/bug is.

Answer 4 · 2024-01-16T17:57:48.000Z

Here's a minimal reproduction using only jiti.

Steps:

Unzip jiti-reproduction.zip
Go into the new directory
yarn install
yarn start

Also, read this comment solidjs/solid-start#1177 (comment), it might save you time

You should see this:

daniel@mmmmmmmmmm jiti-reproduction % yarn start
/private/tmp/jiti/jiti-reproduction/.pnp.cjs:13182
      Error.captureStackTrace(firstError);
            ^

Error: Package subpath './lib/chunks' is not defined by "exports" in /Users/daniel/.yarn/berry/cache/vinxi-npm-0.1.4-662cb08e69-10c0.zip/node_modules/vinxi/package.json imported from /private/tmp/jiti/jiti-reproduction/.yarn/__virtual__/@vinxi-server-functions-virtual-848940c2eb/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.1.2-ce18cd26d1-10c0.zip/node_modules/@vinxi/server-functions/
Require stack:
- /private/tmp/jiti/jiti-reproduction/.yarn/__virtual__/@vinxi-server-functions-virtual-848940c2eb/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.1.2-ce18cd26d1-10c0.zip/node_modules/@vinxi/server-functions/client.js
    at require$$0.Module._resolveFilename (/private/tmp/jiti/jiti-reproduction/.pnp.cjs:13182:13)
    at Function.resolve (node:internal/modules/helpers:187:19)
    at _resolve (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:251148)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:253746)
    at /private/tmp/jiti/jiti-reproduction/.yarn/__virtual__/@vinxi-server-functions-virtual-848940c2eb/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.1.2-ce18cd26d1-10c0.zip/node_modules/@vinxi/server-functions/client.js:3:15
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)
    at /private/tmp/jiti/jiti-reproduction/.yarn/__virtual__/@vinxi-server-functions-virtual-848940c2eb/5/Users/daniel/.yarn/berry/cache/@vinxi-server-functions-npm-0.1.2-ce18cd26d1-10c0.zip/node_modules/@vinxi/server-functions/plugin.js:3:15
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)
    at /Users/daniel/.yarn/berry/cache/@solidjs-start-npm-0.4.9-2d183d4d01-10c0.zip/node_modules/@solidjs/start/config/index.js:1:188
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)
    at /private/tmp/jiti/jiti-reproduction/vite.config.ts:1:177
    at evalModule (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:256443)
    at jiti (/Users/daniel/.yarn/berry/cache/jiti-npm-1.21.0-baebd5985a-10c0.zip/node_modules/jiti/dist/jiti.js:1:254371)
    at file:///private/tmp/jiti/jiti-reproduction/index.js:21:18
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:329:24)
    at async loadESM (node:internal/process/esm_loader:28:7)
    at async handleMainPromise (node:internal/modules/run_main:120:12)

Node.js v21.5.0

Please understand that all I'm trying to do is build a website using solid-js and latest yarn (because solid-js is awesome) and I'm already multiple levels deep of debugging and opening issues. I would be very happy if you could fix this and very disappointed if you don't (although the mlly bug is more severe for me).

To give you some incentive, google "yarn berry" and read about everyone using it - don't you want to enable those people to use your library?

Answer 5 · 2024-01-16T17:59:57.000Z

Thanks for making reproduction 👍🏼

Answer 6 · 2024-01-16T19:13:03.000Z

Some context:

jiti is an ESM+Typescript compatibility layer for Node.js. You might think ESM is stable and fully supported in Node.js but that's sadly not completely true. Node.js provides no stable loader mechanism (to support TS) nor an invalidation API for ESM Modules (to support reloading a config file without full process restart!). Because of this jiti is depending on Node.js CJS modules to resolve this limitations but also pushing ESM forward.

The import.meta.resolve() is also even till today not a stable feature from Node.js to rely on and when it ships, there is a time required until it is backported to other LTS Node.js versions we can rely on. Because of this, jiti uses mlly which uses wooorm/import-meta-resolve that is extracted from Node.js native implementation and works really well in most of the cases.

Adding to the all complexity, yarn (which I love and used since early days!) pnp uses hacky methods to allow resolving from virtual modules and it (recently) supports a compatible polyfill for import.meta.resolve that works within it's environment.

I think the right solution for this might be relying on import.meta.resolve as a fallback but what makes it tricky to just do it as a "fix" is that import.meta.resolve was not stable until very recently and previously it was an async method(!). Considering huge ecosystem (33M downloads per month) doing so is risky but I would definitely consider it in v2 roadmap. PS: We need to hack webpack bundler to say there is a import.meta.resolve native method currently it strips import.meta entirely!

For jiti v1 that is used by solid, we might also enable this fallback support behind a flag but really a simpler solution is that vinxi's package.json could simply use default export condition. Combined with type: "module" that is already in there, this would be a much faster solution to this problem for solid's ecosystem. Also I would love to later talk with team, because /config subpath import is too much expensive for a type helper wrapper. If it was only type helper like Nuxt and Nitro, this issue wouldn't happen. /cc @nksaraf

Also for the follow up, I have tried to directly unplug vinxi and replace import": with "default":, there seems to be more issues within solid. @solidjs/start for example has implicit dependency on vite which breaks yarn pnpm. Fully supporting pnp is hard after all!

Answer 7 · 2024-01-17T09:34:00.000Z

Thanks a lot for looking into this, explaining the context and giving recommendations to the solid-start team @pi0 :)

Great that you don't hate on yarn, I love it too. I just did yarn add vite to get @solidjs/start to work, but ofc best if solid-start adds it to peerDependencies or something.

Nice that you can fallback to import.meta.resolve in v2 and that it's synchronous now. I think the ball is rolling and I have some patches so solid-start works for me right now 🥳

Answer 8 · 2024-09-25T13:43:38.000Z

Tested against jiti v2 nightly with new ESM API. pnpm had one new strange issue (50e4280) and also shows a warning if the extension is explicitly not set in import breaking internal try-catch of jiti and nothing fixable in jiti...

(tested against #204 (comment))

import { createJiti } from "jiti"; // tested against jiti v2 nightly

const jiti = createJiti(import.meta.url);

// Note: Without explicit extension yarn pnp throws error even in internal try-catch block of jiti
const resolvedPath = jiti.esmResolve("./vite.config.ts");
console.log({ resolvedPath });

const mod = await jiti.import(resolve1);
console.log({ mod })