Project References docs/example are missing rootDirs (?)
alexeagle opened this issue · 8 comments
I'm working on a prototype to rebuild the Bazel TypeScript support on top of Project References, basically following the paragraph
https://www.typescriptlang.org/docs/handbook/project-references.html
Some teams have set up msbuild-based workflows wherein tsconfig files have the same implicit graph ordering as the managed projects they are paired with. If your solution is like this, you can continue to use msbuild with tsc -p along with project references; these are fully interoperable.
where msbuild is replaced by Bazel. Seems very promising so far, however there's a snag.
Looking at Ryan's demo
https://github.com/RyanCavanaugh/project-references-demo
we see that each project uses an outDir setting to avoid sprinkling build outputs in the sources (yay)
https://github.com/RyanCavanaugh/project-references-demo/blob/master/core/tsconfig.json#L4
However in dependent projects there is no matching rootDirs option
https://github.com/RyanCavanaugh/project-references-demo/blob/master/animals/tsconfig.json
According to the README, the empty-sleeves branch deletes the code from the dependent project, so I would expect TS resolves from the .d.ts in the outDir. So I'm not understanding how the compiler could find /lib/core/utilities.d.ts to resolve imports like import { makeRandomName } from '../core/utilities'; that appear in /animals/dog.ts.
Indeed checking out the empty-sleeves branch, it doesn't seem to compile, seems like it's just out-of-date prototype code
tsproject.json:11:5 - error TS5023: Unknown compiler option 'referenceTarget'.
11 "referenceTarget": true
Another way to observe my problem: In my minimal Proof-of-Concept I am forced to specify a rootDirs setting.
https://github.com/alexeagle/ts_composite/blob/master/tsconfig-base.json#L7-L8
and in this case since Bazel creates output directory with the platform name in it, I can't really check in this file and keep the project portable between mac/linux/windows.
So the question is, does TS have some way of resolving the outDir from project references that I'm missing? Or is the rootDirs setting required to make this work?
I think this may be asking the same thing as #27572 which was closed without a resolution.
I am embarrassed to ask whether you meant to discuss rootDirs, or if you actually meant rootDir.
copying from a discussion with Daniel, here's an attempt at a simpler explanation:
src
|- foo/abc.ts
|- bar/def.ts
output
|- foo/abc.js
|- bar/def.js
foo and bar are separate projects where bar references foo.
We are compiling bar, and def.ts contains import from ../foo/abc.
If the sources for foo (src/foo/abc.ts) aren't present, this currently fails.
But foo/tsconfig.json has outDir: "../output/foo" so the compiler could have known to look at ../output/foo/abc.d.ts to resolve the import.
Having chatted on a call, it sounded like you're finding some issue where project references don't play well with a segmented source and output root.
src
├─ foo
│ ├─ tsconfig.json ("outDir": "../../lib/foo"; "references": [{ path: "../bar" }] )
│ └─ abc.ts
└─ bar
├─ tsconfig.json ("outDir": "../../lib/bar")
└─ def.ts
lib
├─ foo
│ ├─ abc.d.ts
│ └─ abc.ts
└─ bar
├─ def.d.ts
└─ def.ts
In this scenario, the idea is that abc.ts imports def.ts with a relative import. With project references, we always recommend that users import in a way that will work at runtime. So given the output directory structure, this is fine:
// src/foo/abc.ts
import * as def from "../bar/def.js";Maybe what you were seeing is that in that case, resolution doesn't seem to be jumping to the output .d.ts files - it seems to resolve to the .ts file.
> tsc -b .\src\foo\ --verbose
[4:27:04 PM] Projects in this build:
* src/bar/tsconfig.json
* src/foo/tsconfig.json
[4:27:04 PM] Project 'src/bar/tsconfig.json' is out of date because output file 'out/bar/def.js' does not exist
[4:27:04 PM] Building project 'PROJECT_ROOT/src/bar/tsconfig.json'...
[4:27:07 PM] Project 'src/foo/tsconfig.json' is out of date because output file 'out/foo/abc.js' does not exist
[4:27:07 PM] Building project 'PROJECT_ROOT/src/foo/tsconfig.json'...
======== Resolving module '../bar/def.js' from 'PROJECT_ROOT/src/foo/abc.ts'. ========
Module resolution kind is not specified, using 'NodeJs'.
Loading module as file / folder, candidate module location 'PROJECT_ROOT/src/bar/def.js', target file type 'TypeScript'.
File 'PROJECT_ROOT/src/bar/def.js.ts' does not exist.
File 'PROJECT_ROOT/src/bar/def.js.tsx' does not exist.
File 'PROJECT_ROOT/src/bar/def.js.d.ts' does not exist.
File name 'PROJECT_ROOT/src/bar/def.js' has a '.js' extension - stripping it.
File 'PROJECT_ROOT/src/bar/def.ts' exist - use it as a name resolution result.
======== Module name '../bar/def.js' was successfully resolved to 'PROJECT_ROOT/src/bar/def.ts'. ========
However, I think that at compile-time we must be doing "the right thing" by rebuilding nothing and rechecking against the .d.ts files.
You can tell that the original .ts files aren't being rebuilt because the file structure of bar isn't getting hoisted into the output file structure, and you can test whether TypeScript is building against the .d.ts files by making bar's files massive, and rebuilding foo after a small change in abc.ts.
let fs = require("fs");
let output = "function foo0() { return 100 }";
for (let i = 1; i < 100000; i++) {
void (output += `function foo${i}() { return foo${i-1}(); }\n`)
}
fs.writeFileSync("./src/bar/def.ts", output + ";export {};") tsc --build ./fooAs a side note: the workaround you were trying for not finding the .d.ts files was to use rootDirs to virtualize the source .ts and output .d.ts files being in the same place. I don't know how well that would work, since if I recall correctly, project references changes resolution to prefer .ts files over .d.ts files if it finds both.
It's true that everything is working as intended when the referenced .ts files are resolved.
My problem is that those .ts files are not present on disk, like in Ryan's empty-sleeves example. That's because my build system (Bazel) allows you to parallelize the build on remote worker machines (or simulates this in a local filesystem sandbox to ensure hermeticity of the build). Each TS project is built in a separate build step and therefore in an isolated sandbox (or remote worker). So the filesystem at compile-time of foo looks like
├─ foo
| ├─ abc.ts
│ └─ tsconfig.json ("outDir": "../../lib/foo"; "references": [{ path: "../bar" }] )
└─ bar
└─ tsconfig.json ("outDir": "../../lib/bar")
lib
└─ bar
├─ def.d.ts
└─ def.js
Additionally Bazel only re-builds steps whose inputs have changed. This is why we avoid including
bar/def.tsas an input when compilingfoo- it would causefooto be re-built for any implementation change ofbarwhich breaks the incrementality model that Bazel (andtsc --build) offer.
My last caveat there raises a possible solution: always make the .ts sources of referenced projects available in the Bazel sandbox/remote worker. To fix the brokenness of the incrementality, we'll have to take extra care to make the tsc actions properly behave with --incremental. Of course for changes that affected .d.ts we already had the problem of Bazel re-triggering these build steps which would benefit from --incremental.
I'll leave this open as I still think it's reasonable to expect that module resolution should succeed given the structure on disk I pointed to, and given that Ryan had thought of such a scenario in that empty-sleeves branch.
It turns out that passing .ts/.tsbuildinfo files between Bazel build steps has a different problem which I'll file separately...
Any tips/workarounds here?
@aryeh-looker if you're asking about using TypeScript with Bazel, you might want to look at https://github.com/aspect-build/rules_ts which no longer needs any rootDirs settings.