[project-references] TS2307 with direct file imports (imports that bypass an index)

Question

[project-references] TS2307 with direct file imports (imports that bypass an index)

rosskevin opened this issue 6 years ago · 20 comments

TypeScript Version: 3.1.0-dev.20180831

Search Terms:
TS2307 with direct file imports (imports that bypass an index)

Code
Starting from the yarn workspaces learn-a example, change pkg2/index.ts to import the pkg1/foo class directly.

Reproduction: https://github.com/rosskevin/learn-a/tree/yarn-workspaces-TS2307-direct-file-import

// import * as p1 from '@ryancavanaugh/pkg1'; // works
import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work

export function fn4() {
  // p1.fn();
  fn();
}

yields:

packages/pkg2/src/index.ts:2:20 - error TS2307: Cannot find module '@ryancavanaugh/pkg1/foo'.

2 import { fn } from '@ryancavanaugh/pkg1/foo';

Expected behavior:
All direct paths are accessible, just like any standard file resolution.

Actual behavior:
error TS2307: Cannot find module '@ryancavanaugh/pkg1/foo'.

Related Issues:
#26819 is similar (may be the same) but it is focused on esnext usage
#25600

Answer 1 · 2018-08-31T20:46:57.000Z

Some background - the tedious but working method using aliases with a monorepo (tsconfig at the root):

{
  "compilerOptions": {
    "baseUrl": "./packages",
    "paths": {
      "@ryancavanaugh/pkg1/*": ["./pkg1/src/*"],
      "@ryancavanaugh/pkg1": ["./pkg1/src"]
    }
  }
}

Answer 2 · 2018-09-02T13:34:45.000Z

Switching to TypeScript + monorepos has been fantastic, except for this singular issue. You end up having lots of packages for no other reason to namespace things or you have to bubble all of your modules up to the root of the package just so you can access them without exposing the nested directory structure as in:

import { fn } from '@ryancavanaugh/pkg1/src/foo'

Your solution with using the paths configuration will satisfy the compiler, but it ultimately did not work for me because Node was not able to resolve the modules at runtime.

Similar issue: #25682

Answer 3 · 2018-09-02T13:41:36.000Z

Note: I had to revert project references until this is fixed, but I do have path aliases working without project references.

Answer 4 · 2018-09-10T00:49:36.000Z

This is one of a few show-stoppers for project references for us.

Answer 5 · 2018-09-27T20:25:03.000Z

Again this seems like by design. Our module resolution follows what happens as part of node module resolution.

Assume there was no error, then your pkg2/lib/index.js is going to be

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
// import * as p1 from '@ryancavanaugh/pkg1'; // works
var foo_1 = require("@ryancavanaugh/pkg1/foo"); // does not work
function fn4() {
    foo_1.fn();
}
exports.fn4 = fn4;
//# sourceMappingURL=index.js.map

And that's not going to work. You need to import @ryancavanaugh/pkg1/lib/foo for it to work. compiler never transforms the module references that means you need to provide correct module resolution path

Note --traceResolution gives information about how the resolution was done.

cc @RyanCavanaugh

Answer 6 · 2018-09-27T20:34:25.000Z

@sheetalkamat if I build/publish pkg1 project, then use it externally, import { fn } from '@ryancavanaugh/pkg1/foo' will work.

Therefore, assuming we are seeking parity with individual project builds, if I use this from a monorepo with project references via pkg2, it should work as well, yet it does not:

$ tsc -b
packages/pkg2/src/index.ts:2:20 - error TS2307: Cannot find module '@ryancavanaugh/pkg1/foo'.

2 import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work

So how can you claim project-references using standard node module resolution if this does not work inside a project-references monorepo, but does if I use a single project build?

Answer 7 · 2018-09-27T20:39:46.000Z

This has nothing to do with project references at all. Here is the trace from module resolution and why it fails.

======== Resolving module '@ryancavanaugh/pkg1/foo' from 'c:/temp/learn-a/packages/pkg2/src/index.ts'. ========
Module resolution kind is not specified, using 'NodeJs'.
Loading module '@ryancavanaugh/pkg1/foo' from 'node_modules' folder, target file type 'TypeScript'.
Directory 'c:/temp/learn-a/packages/pkg2/src/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/learn-a/packages/pkg2/node_modules/@types' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/learn-a/packages/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
'package.json' does not have a 'typesVersions' field.
Found 'package.json' at 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/package.json'. Package ID is '@ryancavanaugh/pkg1/foo/index.d.ts@3.0.2'.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.ts' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.tsx' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.d.ts' does not exist.
Directory 'c:/temp/learn-a/node_modules/@types' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Loading module '@ryancavanaugh/pkg1/foo' from 'node_modules' folder, target file type 'JavaScript'.
Directory 'c:/temp/learn-a/packages/pkg2/src/node_modules' does not exist, skipping all lookups in it.
Directory 'c:/temp/learn-a/packages/node_modules' does not exist, skipping all lookups in it.
'package.json' does not have a 'typesVersions' field.
Found 'package.json' at 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/package.json'. Package ID is '@ryancavanaugh/pkg1/foo/index.d.ts@3.0.2'.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.js' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.jsx' does not exist.
Directory 'c:/temp/node_modules' does not exist, skipping all lookups in it.
Directory 'c:/node_modules' does not exist, skipping all lookups in it.
======== Module name '@ryancavanaugh/pkg1/foo' was not resolved. ========

Especially note how there is no information in packageJson to redirect foo into lib file

Also i dont know what you mean by packaging as a module and it works? Can you explain what are the changes and what are your doing for it to be available for module resolution ?

Answer 8 · 2018-09-27T20:50:33.000Z

I'm sorry but your comment about resolution does not change the observable/verifiable facts I presented. Quick restatement of the case:

I can build/publish pkg1, then use it as I do from pkg2 if I am not using a monorepo (e.g. any npm package).
As presented in the project-references monorepo reproduction, pkg2 cannot import a file from pkg1 that is a non-index.

I assume that a project-references monorepo seeks parity with resolutions of files in individually published projects (e.g. any package from npm), correct?

As I mentioned above, I can replicate this standard resolution behavior with path aliases, but project references do not replicate the standard resolution.

Answer 9 · 2018-09-27T21:07:52.000Z

I can build/publish pkg1, then use it as I do from pkg2 if I am not using a monorepo (e.g. any npm package).

Can you please provide example of what you are doing. When I just copied pkg1 as is in node_modules of folder pkg2 and I change tsconfig of pkg2 to remove references

{
  "extends": "../tsconfig.settings.json",
  "compilerOptions": {
    "outDir": "lib",
    "rootDir": "src"
  }
}

Still see the same error when building the project (instead of tsbuild)

c:\temp\learn-a>node c:\TypeScript\built\local\tsc.js -p packages\pkg2\tsconfig.json --traceResolution
======== Resolving module '@ryancavanaugh/pkg1/foo' from 'c:/temp/learn-a/packages/pkg2/src/index.ts'. ========
Module resolution kind is not specified, using 'NodeJs'.
Loading module '@ryancavanaugh/pkg1/foo' from 'node_modules' folder, target file type 'TypeScript'.
Directory 'c:/temp/learn-a/packages/pkg2/src/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/learn-a/packages/pkg2/node_modules/@types' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/learn-a/packages/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
'package.json' does not have a 'typesVersions' field.
Found 'package.json' at 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/package.json'. Package ID is '@ryancavanaugh/pkg1/foo/index.d.ts@3.0.2'.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.ts' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.tsx' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.d.ts' does not exist.
Directory 'c:/temp/learn-a/node_modules/@types' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/temp/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Directory 'c:/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'ryancavanaugh__pkg1/foo'
Loading module '@ryancavanaugh/pkg1/foo' from 'node_modules' folder, target file type 'JavaScript'.
Directory 'c:/temp/learn-a/packages/pkg2/src/node_modules' does not exist, skipping all lookups in it.
Directory 'c:/temp/learn-a/packages/node_modules' does not exist, skipping all lookups in it.
'package.json' does not have a 'typesVersions' field.
Found 'package.json' at 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/package.json'. Package ID is '@ryancavanaugh/pkg1/foo/index.d.ts@3.0.2'.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.js' does not exist.
File 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/foo.jsx' does not exist.
Directory 'c:/temp/node_modules' does not exist, skipping all lookups in it.
Directory 'c:/node_modules' does not exist, skipping all lookups in it.
======== Module name '@ryancavanaugh/pkg1/foo' was not resolved. ========
packages/pkg2/src/index.ts:2:20 - error TS2307: Cannot find module '@ryancavanaugh/pkg1/foo'.

2 import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work
                     ~~~~~~~~~~~~~~~~~~~~~~~~~

Answer 10 · 2018-09-27T21:16:25.000Z

@sheetalkamat our packages (and many others like @material-ui/core) are flattened when published to prevent unnecessarily deep import paths.

The rough equivalent of:

cd packages/pkg1
yarn tsc
cd lib
cp package.json .
# modify module/typings properties to be ./index
npm publish

This makes the index and foo siblings at the root of the package, resolvable at the same level (no lib folder). It is also the reason for the two entries in the path aliases above. The path aliases above allow a source monorepo to behave like a published package, which I feel strongly is an essential need for project-references to be usable.

Does this make sense?

Answer 11 · 2018-09-27T21:22:33.000Z

This is where I think the resolution is incorrect, as no package name can have more than one / in it:

Found 'package.json' at '/Users/kross/projects/learn-a/node_modules/@ryancavanaugh/pkg1/package.json'. Package ID is '@ryancavanaugh/pkg1/foo/index.d.ts@3.0.2'.

It seems that it thinks the package is @ryancavanaugh/pkg1/foo instead of @ryancavanaugh/pkg1, which is incorrect. It is then using the wrong directory as the basis for resolution (speculation on my part).

Answer 12 · 2018-09-28T13:23:29.000Z

@sheetalkamat

import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work

compiler never transforms the module references that means you need to provide correct module resolution path

I think this is the heart of the problem (also discussed in #15479). It would be great if tsc did transform the module references, either based on the paths and/or baseUrl or whatnot. Short of that, we have to add another tool to the the chain (Webpack, module-alias, tsmodule-alias, etc...) to get things to work correctly.

Answer 13 · 2018-10-09T21:37:38.000Z

our packages (and many others like @material-ui/core) are flattened when published to prevent unnecessarily deep import paths.

That means there is a step involved at runtime which indicates that module resolution path is different from the current layout on the disk. Thus you need to provide module resolution mapping to indicate where and how the sources would be.

The line you mentioned here is not an issue. The package json is correctly found at: Found 'package.json' at 'c:/temp/learn-a/node_modules/@ryancavanaugh/pkg1/package.json' Its going to look for foo.ts, foo.d.ts and rest at that location. But since the sources are in src folder at that location it cant find it. That's the issue in your example. The compiler hasn't been provided with information on where the sources are going to be. You need path mapping here to ensure it gets redirected in the src folder instead.

Adding @RyanCavanaugh and @DanielRosenwasser to this discussion as well since we have discussed this offline.

Answer 14 · 2018-10-09T21:45:07.000Z

@sheetalkamat

You need path mapping here to ensure it gets redirected in the src folder instead.

This seems like an unnecessary extra step when the file should probably resolved relative to the entrypoint of the module.

BUT, assuming that is not going to happen, what do you mean by that statement because it is a bit ambiguous? Do you mean I cannot use project references and need to manually map everything as I currently do? Or just add a single path mapping on top of the current project references? I think we should refer again to the reproduction as a foundation for my question: https://github.com/rosskevin/learn-a/tree/yarn-workspaces-TS2307-direct-file-import

Answer 15 · 2018-10-09T23:08:32.000Z

In your example, I changed pkg2 tsconfig to which works and there are no errors

{
  "extends": "../tsconfig.settings.json",
  "compilerOptions": {
    "outDir": "lib",
    "rootDir": "src",
    "baseUrl": "./",
    "paths": {
      "@ryancavanaugh/pkg1/*" : ["../pkg1/src/*"]
    }
  },
  "references": [
    { "path": "../pkg1",  }
  ]
}

Output is:

[16:07:48] Building project 'c:/temp/learn-a/packages/pkg2/tsconfig.json'...

======== Resolving module '@ryancavanaugh/pkg1/foo' from 'c:/temp/learn-a/packages/pkg2/src/index.ts'. ========
Module resolution kind is not specified, using 'NodeJs'.
'baseUrl' option is set to 'c:/temp/learn-a/packages/pkg2/', using this value to resolve non-relative module name '@ryancavanaugh/pkg1/foo'.
'paths' option is specified, looking for a pattern to match module name '@ryancavanaugh/pkg1/foo'.
Module name '@ryancavanaugh/pkg1/foo', matched pattern '@ryancavanaugh/pkg1/*'.
Trying substitution '../pkg1/src/*', candidate module location: '../pkg1/src/foo'.
Loading module as file / folder, candidate module location 'c:/temp/learn-a/packages/pkg1/src/foo', target file type 'TypeScript'.
File 'c:/temp/learn-a/packages/pkg1/src/foo.ts' exist - use it as a name resolution result.
======== Module name '@ryancavanaugh/pkg1/foo' was successfully resolved to 'c:/temp/learn-a/packages/pkg1/src/foo.ts'. ========

Answer 16 · 2018-10-09T23:22:02.000Z

Although i've not tried it in this specific case, my experience is that this:

    "paths": {
      "@ryancavanaugh/pkg1/*" : ["../pkg1/src/*"]
    }

will satisfy the tsc compiler, but when you actually try to run the application in node, the runtime packages will not be resolved, because the compiler does not re-write the paths - as is discussed here.

Answer 17 · 2018-10-10T00:14:17.000Z

By providing the path mapping In compiler options you are informing compiler where the module would be at runtime and the extra build step is needed to ensure that guarantee at runtime .. which is done as part of learna project setup at per #26823 (comment)

Answer 18 · 2018-10-10T13:16:53.000Z

Thank you very much @sheetalkamat for taking time to help me with this!

Answer 19 · 2018-10-10T13:29:20.000Z

For anyone that may bump into this, I have updated my Lerna monorepo + yarn workspaces + typescript project references + flattened build repo as a sample that hopefully helps others.

https://github.com/rosskevin/learn-a

Answer 20 · 2019-11-12T14:36:34.000Z

For anyone stumbling across this later I wrote a little script that configures yarn workspaces correctly for mono repos based on their local dependencies. For packages that publish some sub directory add publishConfig.directory to your package.json to tell the script (and also Lerna) how to rewrite imports locally via paths.

https://gist.github.com/jquense/0c6723e1c6e86bd46e1ee18ff27ac4c2

it will update existing tsconfigs safely so you can run this as part of commit hook or post-install script, etc