microsoft/TypeScript

tsc --build / Project References Feedback & Discussion

RyanCavanaugh opened this issue · 141 comments

RyanCavanaugh commented

Continuation of #3469 "Medium-sized Projects" which has grown too large for GitHub to display comfortably

Please use this thread for feedback, general discussion, questions, or requests for help / "is this a bug" discussions.

Active bugs / PRs in this area:

Possible next steps:

  • (none yet; @andy-ms) Find unloaded downstream projects during rename
  • #25376 Infer project references from monorepo tooling package.json files
  • #25562 Terse mode output for tsc -b
  • (none yet; @weswigham ?) Intermediate build outputs to temp folders
  • (none; stretch) Parallelization / worker processes under tsc -b

Other interesting links:

timfish commented

A couple of things:

  1. watch doesn't currently output an initial build. It waits until the first file change. This differs to non build mode.
  2. How is build order determined? I found I had to order the project references in dependency order to get the build to complete first time without error
RyanCavanaugh commented

@timfish thanks! Re 1 - PR up at #25610. For the other issue, can you sketch out the files you have? The order is supposed to be a simple topological sort that should always yield a correct ordering. I'll try to repro locally in the meantime based on what you've described

Edit: Not sure how to repro - I changed the TypeScript repo's src/tsconfig.json to be in effectively random order and it still figured out a valid build order. Will need details on this.

EisenbergEffect commented

@RyanCavanaugh Were you all able to land the cross-project rename for the RC?

RyanCavanaugh commented

@EisenbergEffect we have "upstream" renames (renaming a usage in client affects declarations in shared) but not "downstream" (renames in shared affecting uses in server and client) yet

EisenbergEffect commented

Gotcha. Any ideas is you'll be able to get the downstream rename in for the final release? or have you already determined that it needs to push out to 3.1 or beyond?

RyanCavanaugh commented

It looks like we can probably get downstream renames for loaded projects in by 3.0 final. Detecting which other projects on your computer need to be loaded to do an "exhaustive downstream rename" is looking dicey - we haven't found any mechanisms that would let us short-circuit that work when the symbol being renamed is exported from the current file.

  • 👍1
  • 🎉1
newtack commented

I am using the API to build Typescript projects, including createWatchProgram and createWatchCompilerHost.

  1. Are these updated to use the new project references and 2) do I need to do anything else in my code other than update the tsconfig.json?
RyanCavanaugh commented

If you're hosting the compiler API, you don't need to do anything new for project references; everything should work as-is. If you want to take advantage of the new --build mode features, the entry point is createSolutionBuilder and you'll need to provide a BuildHost and CompilerHost along with the file timestamp APIs on the latter.

To migrate to using project references in your code itself, updating your tsconfig.json to a) add references and b) add composite: true may be sufficient - if not, you'll see errors informing you what else needs to happen. Writing a comprehensive migration guide has been a difficult task because project setups are so varied - I'm hoping we can find migrations from early adopters as something to point people to, but it's hard to give guidance without seeing specific build layouts.

newtack commented

Thanks @RyanCavanaugh

Are there any examples for "createSolutionBuilder" and "timestamp API"?

timfish commented

@RyanCavanaugh ref: Build order.

I'll try to get a repo reproducing this although it won't be until next week.

If I'm just adding all references to a root tsconfig, it should work out the dependency tree from that?

RyanCavanaugh commented

@newtack the compiler source code itself (src/compiler/sys.ts for timestamp APIs and src/compiler/tsc.ts & tsbuild.ts) are good references.

@timfish

If I'm just adding all references to a root tsconfig, it should work out the dependency tree from that?

Correct

OliverJAsh commented

Trying this out with the 3.0 RC.

declarationMaps
We've also added support for declaration source maps.
If you enable --declarationMap, you'll be able to use editor features like "Go to Definition" and Rename to transparently navigate and edit code across project boundaries in supported editors.

I tried using this but for some reason "Go to Definition" didn't seem to work. I'm probably doing something wrong, but filed an issue with a minimal reproduction case here: #25662

Will this also handle "Find All References"? If not, is there any way that could be supported?

  • 👍1
kohlmannj commented

Based on @RyanCavanaugh and @rosskevin's work to demonstrate Project References within a Lerna-based monorepo123, I've made another not-for-merging Pull Request which uses both Yarn workspaces and Babel 7: RyanCavanaugh/learn-a#4

Hopefully this example helps others learn! I've been working on a Lerna-based monorepo similar to this, and now (I think) I have a better idea of how to configure Project References with it.

—

  1. https://github.com/RyanCavanaugh/learn-a
  2. RyanCavanaugh/learn-a#3
  3. #3469 (comment)
  • 👍2
drew-y commented

I was hoping to get some advice on how to handle path resolutions. My directory currently looks something like this:

- client/
  - assets/
  - views/
  - src/
  - tsconfig.json
  - webpack.config.js
- shared/
  - src/
  - tsconfig.json
- server/
  - src/
  - tsconfig.json

Im struggling to reference the shared directory. I was surprised to learn that the "paths" section along with baseURL doesn't allow for path aliasing like webpack does. So now I'm not sure how to reference the shared directory at all.

For example, the following doesn't work:

// server/src/index.ts
import { mySharedFunc } from "../../shared/src"

// This doesn't work either. "shared" is still shared after compilation
import { mySharedFunc } from "shared"

Because the project compiles down to:

- dist/
  - client/
    - bundle.js
  - shared/
    - index.js
  - server/
    - index.js

As you can see, it is invalid to reference the src dir as it doesn't exist after compilation.

Should I restructure my repo somehow? Is it possible to get my path aliasing to work?

My struggles appear to be related to #25682

  • 👍6
drew-y commented

I've updated my folder structure to have client, server, and shared under one src directory. This resolved my issue.

ryanstaniforth commented

I assumed that with "composite": true, "declaration": true would always be forced. However this only seems to apply with tsc --build.

I found this out while doing a normal build (tsc) and not seeing expected definition files.

Is this expected behaviour? It seems counter-intuitive to me.

sledorze commented

I found myself using the same technique as the references system in our temporary custom incremental build system (comparing d.ts files) to decide what to rebuild in a non incremental scenario.

However, after finding strange results, I've discovered that the typescript compiler (3.0-RC) do NOT generate deterministically d.ts files.
Indeed, when Unions are involded, the displayed order of those seems to depend on the nodeJS run, which makes to system detect false positives.

ex: { status: "400"; body: KoStatus; } | { status: "200"; body: OkStatus; }
vs:
{ status: "200"; body: OkStatus; } | { status: "400"; body: KoStatus; }

I think you may have found yourselves those issues and wonder if that's a know/accepted limitation between runs or if I should open an issue (have not found an existing one yet).

rosslovas commented

Trying it out on the codebase I'm working on, I ran into a lot of Output file '.../_.d.ts' has not been built from source file '.../_.ts' errors. I managed to strip down a repro to almost nothing, and found that when rootDir and outDir are involved as well as a project reference, source files within an individual project that has a dependency project would be compiled in alphabetical order.

The consequence is that an A.ts that imports from B.ts will fail with Output file '.../B.d.ts' has not been built from source file '.../B.ts' but if I rename A.ts to C.ts everything works, as does removing the project reference or not using root/outDir. The problem occurs in 3.0.0-rc as well as next right now.

Edit: In retrospect this is clearly enough a bug to warrant its own issue, so I made #25864.

  • 👍2
  • 😕2
josundt commented

I just tested TS 3 RC project references in a typical npm package setup with included tests. The tests were set up as a TS project referencing the implementation TS project.

I surprisingly found that project references do not adhere to the outDir compilerOption of referenced or referencing projects, and that the relative paths for module imports in the compiled referencing project's JS files therefore are wrong; paths point to the TS folder, not the compiled output folder.

A somewhat simplified view of my PoC:

  • src/
    (contains implementation code, TS project REFERENCED by tests)
  • dist/
    (compiled output from src/; using the outDir compilerOption)
  • test/
    (tests for implementation code in src/; project REFERENCING src/)

Expected behavior:
Relative paths of module imports in the compiled code for the referencing project should be calculated from:

  • relative path from outDir compilerOption of referenced project
  • relative path from outDir compilerOption of referencing project

I put up a small repository to demonstrate the problem:
https://github.com/josundt/tsprojects-outdirproblem

UPDATE: Separate issue created: #26036

gentoo90 commented

Is <TypeScriptBuildMode> msbuild parameter pllaned for 3.0.1? It's not working in 3.0.0, tsc.exe still runs with --project, even if it's set to true.

massimocode commented

@RyanCavanaugh This project thing looks really cool so I decided to give it a try. It hasn't been going so well for me so I thought I'd share my experiences and the code-base in case it's helpful for you guys.

Basically, I have the following folders in my project.

  • back-end - A Node.JS back-end. References code from shared.
  • front-end - An Angular 2 front-end. References code from shared.
  • shared - Vanilla JS code. Doesn't reference any of the other folders.
  • specs - Some tests written in typescript. References back-end, front-end and shared.

My previous set-up (TypeScript v2.x) was to have a tsconfig.json in the root with the most "forgiving" settings and that would give my IDE all the intellisense etc and cross project renames. I then had a compile.json in back-end (which was a tsconfig file in disguise), and another compile.json in the front-end (same concept). The reason for these files is that I wanted to only compile front-end code (for webpack) or back-end code (for ts-node) separately from the rest. Needless to say, they wouldn't have any issues picking up the code that I referenced in the shared folder as the paths were all relative etc. This all worked perfectly fine for my work flow and I've been super happy with it (except about a year and a half ago the Angular CLI didn't like it at all, but I'm doing webpack manually and haven't retried since...)

So, I upgraded TypeScript to the @next version and switched over to this whole tsconfig per folder thing and removed the tsconfig.json that was at the root. I'm using the latest version of the VSCode Insiders build, but it doesn't really seem to work at all. In the sense that when I'm looking at some back-end code that references something in the shared folder, it says that thing simply doesn't exist.

The ts-node for my back-end only works if I compile the shared folder and emit to disk, something that I'd rather not have to do because my back-end compiles down to ES6 and my front-end down to ES5. Although I guess it wouldn't be so bad if it emitted declarations only?

Anyway, I've checked it all in, would you mind taking a look and seeing what you think? I wouldn't mind if you wanted to use the repo (or part of it) as some kind of example project. Thanks!

Here's the link: https://github.com/SMH110/Pizza-website/tree/TS_Projects

The previous working setup not using the new projects functionality is at this commit: SMH110/Pizza-website@9548bdd

  • 👍2
WearyMonkey commented

Hi @RyanCavanaugh! I'm very excited to start using this feature at Canva, so have been having a play. I have run into one bug.

When a project contains a reference to project in a ancestor folder, includes source files must be listed in dependency order.

Sample project: https://github.com/WearyMonkey/ts3-file-order-bug

e.g.

{
  "compilerOptions": {
    "composite": true,
    "declaration": true

  },
  "references": [
    { "path": ".." }
  ],
  "files": [
    "./imports.ts",
    "./exports.ts"
  ]
}

Where imports.ts is:

import { foo } from './exports';

and exports.ts is:

export const foo = 'foo';

Fails with the error

error TS6305: Output file '/Users/toby/dev/ts3_bug/proj1/proj2/exports.d.ts' has not been built from source file '/Users/toby/dev/ts3_bug/proj1/proj2/exports.ts'.

Changing the order of files to:

  "files": [
    "./exports.ts",
    "./imports.ts"
  ]

Succeeds compilation as expected.

I've tried on 3.0-rc, 3.0.1-insiders.20180726, and 3.1.0-dev

Some more general feedback on my experience so far:

  • So fast.. can't wait!
  • The relationship between rootDir, outDir and how that affects cross project imports is quite confusing, and I was only able to find the correct (I think) configuration via trial and error.
  • I often get has not been built from source file error, and theres no help to find the cause, forcing me to fix via trial and error.
  • 👍1
DanielRosenwasser commented

Just as a heads up for everyone: @RyanCavanaugh recently became a father! 🎉 👶 🍼 🎉

He'll be out for a while, but in the meantime if you are running into anything bug-like, I'd encourage you to file separate issues and the team will try to take a look.

Otherwise, this is still a good venue for general discussion. Thanks all!

  • 🎉16
  • ❤️9
JiriZidek commented

This feature looks like holy-grail we were looking for from the very beginning of TS: Since compiling every single .ts into single .js (1:1) was too fragmented (and would had resulted in enormous <link..> tags), we opted for All:1 (single .js file), all our pages use single <link..> that load one jumbo .js file, but browser is getting a lot of code that is not used by specific page (e.g. editor of "Contact" does obtain code for editor of "Email").
Now we will be able to compile shared .ts files into one .js and specific .ts files into another .js file, but still limiting them to reasonable count (like five).
Am I right, that now we can have within single .csproj file (WebAPI) multiple .js targets (accompanied by d.ts) so we can modularize our project ?
I would compare it to analogy: Multitude of .cs files are compiled into DLLs. Up till now we had only one option: to compile all into single EXE.
Do I underestand it correctly ?

Soul-Master commented

I try to create composite project for testing this feature. I think, this feature still need a lot of implement in term of tooling.

In VS Code insider build, I can use renaming tool to rename "add" function in source project. However, I cannot go to definition of reference function in test project. It navigated to definition file instead.

In VS2017 15.8 Preview 5, it didn't change exported function name in test project after I renamed "add" function in source project. Moreover, it showed definition file instead of real source code when I clicked go to implementation in test project.

src/utils.ts

export function add(x: number, y: number) {
    return x + y;
}

export function remove(x: number, y: number) {
    return x - y;
}

export const defaultSettings: AppPlatform.Setting = {
    width: 50,
    height: 50
};

test/mainTest.ts

import { add, defaultSettings } from "../src/utils";

export function testAdd(){
    const result = add(5, 4);

    return result;
}

export function getDefaultSetting() {
    return defaultSettings;
}

https://github.com/Soul-Master/typescript3-composite-project

AnyhowStep commented

I just tried out the composite project feature.
It all compiles, and runs with ts-node successfully.

However, when I try to "Go to Definition (F12)" with VS Code, it just says "No definition found".
Hovering my cursor over a function of the imported project gives the correct type declarations, though.

TS 3.0.1

VS Code:
Version: 1.25.1
Commit: 1dfc5e557209371715f655691b1235b6b26a06be
Date: 2018-07-11T15:40:20.190Z
Electron: 1.7.12
Chrome: 58.0.3029.110
Node.js: 7.9.0
V8: 5.8.283.38
Architecture: x64

[EDIT]
Here's my attempt at using TS 3's project composition
https://github.com/AnyhowStep/TS3ProjectComposition

EisenbergEffect commented

For anyone combing this thread looking for sample setups, Aurelia vNext is now in full swing 😄 You can find our monorepo here: https://github.com/aurelia/aurelia It's setup with TypeScript 3 projects, Lerna 3, Mocha/Chai/Sinon/Karma for unit tests and CircleCI 2.0 for continuous integration.

Big thanks to the TypeScript team for all their hard work. Aurelia vNext is fully committed to TS now and we're loving the new productivity and contribution-pleasant setup we are able to achieve with this new combination of tools. Keep it up!

  • 👍5
  • ❤️6
octref commented

Feedback 1: I'd like tsc -b -w to show what has been rebuilt in the CLI (similar to tsc -p -w). Currently it just silently rebuilds. -v is too noisy.

Feedback 2: Can you have an official example (as in, GitHub project) that illustrates how one should structure a project, linked from the handbook? Nobody is going find the example projects you are linking in this issue.

Feedback 3: The blog post has a misleading comment // Needed for project references. for composite, when composite is only required for referenced project.

Feedback 4: The blog post does not include --watch in the --build mode section (IMO it's the most useful one). This is inconsistent with the handbook.

drew-y commented

Has anybody else had any issues with auto-importing from a project reference? VSCode always wants to import from my dist folder instead of the src folder.

Example:

// Auto imports from here
import { MyFunc } from "../../../../dist/shared";
// Instead of here
import { MyFunc } from "../../../shared";
  • 👍4
ryanstaniforth commented

@drew-y I'm pretty sure this is the intention with --build mode. Because you're essentially referencing a separate TypeScript project (shared in this case), you must import the built output of it.

  • 👎1
drew-y commented

@ryanstaniforth That does make sense. And I now realize that "../../../../dist/shared" technically works. However "../../../shared" is still preferable, at the very least is terms of aesthetics. So it would be nice if there was still some sort of option to prefer that directory. If possible.

ryanstaniforth commented

@drew-y I see where you're coming from, I first assumed I should import from the source directory. If that was the case though, what would be the point of TypeScript ensuring dependencies are compiled first. There must be a reason that we're not aware of for this design decision.

drew-y commented

@ryanstaniforth Agreed. Although technically, in my case, "../../../shared" is an import from the dist directory.

At least for me, my dist directory directly mirrors my src directory. I.E.:

src/
  client/
  server/
  shared/
dist/
  client/
  server/
  shared/

But I'm realizing it would be tough for the typescript compiler to know that. Unless there was a way for me to specify it.

ryanstaniforth commented

@drew-y Oh I see, so you actually only have a single project, i.e. only one tsconfig.json? In that case I see you're problem, it shouldn't favour dist.

dmoonfire commented

Is there an example of using this with a multirepo? My development environment is working great while working across projects but each one is broken into separate repos and NPM packages. When I try to build them via CI, the references aren't pulled down so it complains and refuses to build.

It would be great if there was an option that lets me fallback to normal resolution (via node_modules/), but I don't think there is one.

Or is this really only useful for monorepos?

Thank you for the fantastic feature.

Soul-Master commented

I just tried VSCode 1.26.0 with my composite project. With TypeScript 3.0, nothing works. I cannot go to definition or rename method in all referenced files.

However, it works fine when I switch TypeScript to 2.7.1 for unknown reason.

gotodefinition

rename

https://github.com/Soul-Master/typescript3-composite-project

  • 👍2
jamiewinder commented

I've never seen the need for build before / after hooks for TypeScript until now, but after successfully getting project references working in my project I'm wondering if they're worth considering.

I use NPM scripts to automatically bump the patch version of my packages every time I build, e.g.:

"build": "npm version patch --no-git-tag-version && tsc"

But of course I'm not the one building these packages any more, the TypeScript build mode is.

Some useful hooks for this scenario would be preBuild and postBuild which only run if something has actually changed in the project (which presumably --build can distinguish up front).

natew commented

If I have project A with a reference to project B and I change a single file in project B, I am seeing the following:

tsc "touches" every file in project B outDir.
tsc "touches" every file in project A outDir after that.

In a big project with two different nodemon processes being watched, this is causing some mayhem. For some reason I actually end up in a loop of restarts (debugging that led me to realize the above), but even beyond that, wouldn't this make a lot more sense and be a lot more gentle?

tsc outputs just the changed file in project B to outDir
tsc does nothing in project A

ricklove commented

Having serious problems with only a single depth of referenced projects in vscode:

I am using outDir with a common build folder that mirrors my source folder.

However, vscode seems entirely dependent on the .d.ts files that are generated by tsc -b -w running in the background.

  • One issue this causes is that often vscode will have a completely wrong code comment: i.e. when I mouse-over in the editor, its showing a comment for the wrong element (as if the internal source mapping typescript is using for the current file is old).

  • Another issue is that the type system seems to break quickly: i.e. I can reload vscode and all will be well and all the comments will be correct. Then as soon as I type anthing in the file, some of the type information breaks (some of the types from the dependent project will go to any etc.). This happens even when I can F12 directly to the .d.ts file in the build folder (however, I did notice that although the .d.ts code is correct, the color scheme is wrong for the broken type.)

Anyway, at this point typescript 3.0 references seem to work ok with tsc but vscode is not stable enough to use it.

Update: Import files that reference each other breaks

It seems that my problem had to do with importing 2 files from the referenced project that reference each other:

import React from 'react';

// BACKGROUND: 
// 'common-components' is a referenced ts project
// 'list-editor' imports 'common-deps'

// ERROR: If I try to import a 'common-deps' here, it breaks the type system in vscode (actually it works for a short time immediately after reloading vscode)
// import { CommonComps } from '../../common-components/src/common-deps';
import * as ListEditorModule from '../../common-components/src/list-editor';

// WORKAROUND: However, if I export a 'common-deps' type from within 'list-editor', and bring in the type that way, it works fine
type CommonComps = ListEditorModule.CommonComps;
  • 👍1
ricklove commented

Another issue: It seems the tsc -b -w will not build child (referenced) projects if the main project has an error. I would expect the build tree to execute the dependencies first (and generate their outputs) even if the main project has a problem that prevents the build (assuming the tsconfig references are correct). However, it doesn't seem to be working like this.

With vscode dependent on the outputs of the referenced projects, it is not possible to get code comments and helps to work without manually building the referenced projects.

  • 👍4
ricklove commented

Found the problem and added a bug report: #26703

panjiesw commented

I attempted to use project references in a monorepo, so I don't know if this should be here or the other discussion. I made a fork of learn-a project https://github.com/panjiesw/learn-a to better describe what I found.

One of monorepo pattern using lerna + NPM is to reference subprojects inter dependencies using NPM's own "package": "file:<location>" instead of their version in package.json. The learn-a fork above have been modified to this structure. Some modified preview:

// root package.json
{
  "dependencies": {
    "@ryancavanaugh/pkg1": "file:packages/pkg1", // direct sub-package reference
    "@ryancavanaugh/pkg2": "file:packages/pkg2", // direct sub-package reference
    "@ryancavanaugh/pkg3": "file:packages/pkg3" // direct sub-package reference
  },
  "devDependencies": {
    "lerna": "^3.2.1",
    "typescript": "^3.0.3"
  }
}
// pkg2 package.json
{
  //<... unmodified stuffs ...>

  "dependencies": {
    "@ryancavanaugh/pkg1": "file:../pkg1" // direct inter sub-package reference
  }
}

This way, even without --hoist option in lerna bootstrap, NPM will hoist all the sub-package dependencies in the root's node_modules. Combine this with the newly addition of TypeScript build mode for project reference, we can have a powerful build orchestration from the root package only, instead of repeating build instruction in every package.

There are issues though, in VSCode v1.26.1, the auto-import and refactoring experience is still not ideal for me. For sub-package that has "references" to other package in their tsconfig (pkg2 and pkg3), somehow, the auto-import always use absolute path, even for something (say, interface) in a sibling .ts file. Even if my "typescript.preferences.importModuleSpecifier" setting is set to "auto" or "relative".

// pkg2/src/baz/interface.ts
export interface Baz {
  message: string;
}

// pkg2/src/baz/index.ts
import { Baz } from '@ryancavanaugh/pkg2/src/baz/interface'; // auto-imported

export default function baz(): Baz {
  return { name: 'baz', message: 'Hello Baz' };
}

This doesn't happen in pkg1 which doesn't reference other sub-package

// pkg1/src/bar/interface.ts
export interface Bar {
  message: string;
}

// pkg1/src/bar/index.ts
import { Bar } from './interface'; // correct auto-import

export default function bar(): Bar {
  return {name: 'bar', message: 'Hello World!'};
}

Maybe it has to do with the fact that all sub-packages are referenced in root's package.json so they're all also appeared in root's node_modules as symlink.

EDIT

To run my learn-a fork, you need lerna version >=3.0.0 and do lerna bootstrap instead of npm install for the first time.

rosskevin commented

Usage with esnext fails

@RyanCavanaugh I'm pretty sure I'm missing something, or it's a bug, but project references seem not to work when using esnext. Reproduction based on learn-a yarn workspaces here: #26819. Result is a dreaded error TS2307: Cannot find module, and removing the module/target from compiler options makes the sample compilable again.

rosskevin commented

Usage with direct file imports fail

I filed another issue for direct file imports not working in #26823.

It seems quite unworkable if true that with project references you can only resolve from the index, not direct to a file e.g.

// import * as p1 from '@ryancavanaugh/pkg1'; // works
import { fn } from '@ryancavanaugh/pkg1/foo'; // does not work

export function fn4() {
  // p1.fn();
  fn();
}
  • 👍1
WhiteAbeLincoln commented

@rosskevin The reason you only can resolve from the index is because tsc imports the built files when using project references. This means that your import must be import { fn } from '@ryancavanaugh/pkg1/lib/foo' because the tsconfig for pkg1 has "outDir": "lib" and the package.json has "typings": "lib/index.d.ts". The import would work as you expected if outDir was unset and typings was set to just index.d.ts.

Also, the same problem would occur when importing any package that was built out of source unless a prepackage script changed the location of the built files. In that case the package.json would also have to be changed so that the "main" and "typings" keys did not reference the build directory.

EDIT:
This was my understanding of the issue, and how I resolved your example with learn-a. If I'm missing something, just ignore this I guess

rosskevin commented

That may be true @WhiteAbeLincoln, but I did not try unsetting outDir because build artifacts would pollute the src directory, and that isn't something that is acceptable to us.

WhiteAbeLincoln commented
goloveychuk commented

Should tsc -b --watch rebuild projects incrementally?
Checked now (typescript@3.1.rc), on any change in one file tsc building the whole project and all dependent projects.
Tested with two parallel tsc --watch processes - works much faster, rebuilds only changed file.
So, is incremental rebuild not implemented yet or it's a bug?

  • 👍6
sledorze commented

I think that typescript Types signature exported in d.ts files are not equivalent between nodejs runs (the order of union members can change for example), that may explain some strange rebuild behaviours.

phil-lgr commented

I'm using references and @WearyMonkey's comment (#25600 (comment)) helped me, the referenced projects compile file, but the project I'm pointing to aka tsc --build src/server/tsconfig.json --watch was failing to build itself! I added the problematic file to the files:

image

and now it works! so basically, "include": ["./**/*"] does not work with --build, seems that tsc --build does not know how to build the pointed project in the correct order... or am I doing it wrong?

I tried the dev and rc builds.

Edit: I think #27195 is related

mttcr commented

I tried to use project references with projects that use simple namespaces. The structure looks like that:

a/
 src/
    a1.ts :  namespace test { export class A { public attr = E.A; } }
    a2.d.ts :  declare namespace test { const enum E { A=1, B=2 } }
 tsconfig.json :
		{
			"compilerOptions": {
				"target": "es5", "types": [], "declaration": true, "composite": true,		
				"outDir": "build","declarationDir": "def", "rootDir": "src",
				"declarationMap": true, "sourceMap": true, "noEmitOnError": true,
			}
		}
b/ 
  src/
   b1.ts : namespace test { let b = new B(); b.attr = E.A; let a = new A(); a.attr = E.B; }
   b2.ts : namespace test { export class B { public attr = E.B; } }
tsconfig.json
		{
			"compilerOptions": {
				"target": "es5", "types": [], "declaration": true, "composite": true,
				"outDir": "build", "declarationDir": "def", "rootDir": "src",
				"declarationMap": true, "sourceMap": true, "noEmitOnError": true,
			},
			"references": [{ "path": "../a" } ]
		}

When I compile, I have the following errors:

>node ..\node_modules\typescript\lib\tsc.js -version
Version 3.2.0-dev.20180926
>node ..\node_modules\typescript\lib\tsc.js  -b b -v
[09:30:24] Projects in this build:
    * a/tsconfig.json
    * b/tsconfig.json

[09:30:24] Project 'a/tsconfig.json' is out of date because output file 'a/build/a1.js' does not exist

[09:30:24] Building project 'C:/testnext/test/a/tsconfig.json'...

[09:30:25] Project 'b/tsconfig.json' is out of date because output file 'b/build/b1.js' does not exist

[09:30:25] Building project 'C:/testnext/test/b/tsconfig.json'...

b/src/b1.ts:4:10 - error TS2304: Cannot find name 'E'.

4 b.attr = E.A;
           ~

b/src/b1.ts:5:14 - error TS2552: Cannot find name 'A'. Did you mean 'a'?

5  let a = new A();
               ~

  b/src/b1.ts:5:6
    5  let a = new A();
           ~
    'a' is declared here.

b/src/b1.ts:6:11 - error TS2304: Cannot find name 'E'.

6  a.attr = E.B;
            ~

b/src/b2.ts:5:16 - error TS2304: Cannot find name 'E'.

5  public attr = E.B;
                 ~

Does this works as expected? Is there a way to make it work? For the moment, I'm using a single project with all sub projects embedded. Problem is, we can reference b classes from a project.

yonilerner commented

@RyanCavanaugh I think Im experiencing the same issue as @goloveychuk , where using --watch with --build doesnt seem to incrementally build projects; when a change is detected in any of the projects, only that project is rebuilt, but the entire project is rebuilt, rather than just the changes. --verbose doesnt have any info on this; I'm assuming that this is the case because a recompile upon a change using --build --watch takes about the same amount of time as a full compile of that individual project, 7 seconds, but when running tsc --watch without --build on that project, file changes are recompiled in less than a second

  • 👍6
jonaskello commented

I'm seeing the same thing as @goloveychuk and @yonilerner. I tested on a small project that just takes ~3 seconds to build but I'm seeing that tsc -w -b takes ~3 seconds for the initial build and then also ~3 seconds for every file change while tsc -w -p takes ~3 seconds for the initial build and then ~0 seconds for file changes:

$ tsc -b -w --preserveWatchOutput packages/pkg1

15:44:42] Starting compilation in watch mode...
[15:44:45] Found 0 errors. Watching for file changes.
[15:44:48] File change detected. Starting incremental compilation...
[15:44:50] Found 0 errors. Watching for file changes.
[15:45:12] File change detected. Starting incremental compilation...
[15:45:15] Found 0 errors. Watching for file changes.

vs

$ tsc -w --preserveWatchOutput -p packages/pkg1
[15:45:29] Starting compilation in watch mode...
[15:45:33] Found 0 errors. Watching for file changes.
[15:45:36] File change detected. Starting incremental compilation...
[15:45:36] Found 0 errors. Watching for file changes.
[15:45:38] File change detected. Starting incremental compilation...
[15:45:38] Found 0 errors. Watching for file changes.

Looking for som clarification if this is the intended behaviour of tsc -b -w or if there is a plan to make it behave like tsc -w -p?

  • 👍3
RyanCavanaugh commented

@yonilerner @jonaskello sounds like a bug, though we've had some recent improvements in this area as well that may have addressed it. If you have a repo we can use to test on, we can investigate?

sheetalkamat commented

We do not yet do incremental build at program level in tsbuild watch mode (Like tsc -w) and it’s a todo yet to be addressed and that’s the reason you see that much time to build

yonilerner commented

@sheetalkamat Ah ok, that would explain it. Thanks for the info. Do you have an idea of when that will be implemented?

jonaskello commented

@sheetalkamat Nice to know that this feature is coming :-). Currently it is not really tenable to use tsc -b -w during development for larger projects because it is so much slower than tsc -w -p or ts-loader. Of course I'm also interested to know the roadmap for implementing this.

  • 👍3
ciddan commented

Hey,

We've run into an issue with project references that we can't quite figure out. I'll try to explain the scenario, it's not entirely straightforward.

These are the relevant parts of our monorepo project structure:

platform/
    client/
        ui/
            tsconfig.json
    shared/
        libs/
            ui/
                vuex-extensions/
                    tsconfig.json
    web/
        thing/
            tsconfig.json

client/ui and web/thing both reference shared/libs/ui/vuex-extensions. All three projects depend on the npm module vuex. The vuex-extensions project contains some vuex helper decorators.

When we import the vuex-extensions shared library into the client/ui project or the web/thing project those projects stop successfully compiling, due to the following reason:

import Vuex from "vuex";
import { createDataStore } from "@shared/vuex-extensions/DataModule";

Adding the import statement for the vuex-extensions changes the import Vuex from "vuex" module resolution. Instead of importing from the "local" node_modules, "vuex" is resolved to the shared lib. And it seems as if the ambient type declarations present in the vuex typings aren't applied in this scenario.

See following screenshots:

import_1
import_2

We've found a work-around by forcing the typescript compiler to use the correct path for vuex by adding it to the paths section of our tsconfig:

"paths": {
	"@shared/localization-store": [
		"../../shared/libs/ui/localization-store"
	],
	"@shared/vuex-extensions/*": [
		"../../shared/libs/ui/vuex-extensions/dist/*"
	],
	"vuex": ["node_modules/vuex"]
},

I'm not sure if this is a bug in the module resolution, intended behavior, incorrect usage or all three at once. Any help on the matter would be appreciated. Thanks!

sheetalkamat commented

@ciddan you can use --traceResolution to see what is going on with the resolution in both cases for further investigating

sheetalkamat commented

@yonilerner @jonaskello the --watch behavior change is likely to come in 3.3

  • 🎉10
ciddan commented

@sheetalkamat Thanks for the tip! I'll check that out tomorrow.

axelnormand commented

I'm also struggling to get project references working as expected with a react lerna monorepo project. The idea is to have a publishable component library as used by app1 and app2 but all in one monorepo and share build tools.

Here's a working example: https://github.com/axelnormand/react-typescript-monorepo

I couldn't get it to work in VS Code without also adding that paths bit in the tsconfigs like @ciddan mentioned.

As also mentioned up thread, VS Code can't compile it out the box as the parent projects need the generated d.ts files, so looking forward to future versions when you don't need declaration and declarationMap on.

My solution for VS Code was to use tasks.json to call compile:watch npm target (which just calls tsc -b). Then when VS Code starts up you have to press ctrl+shift+b to get it going:

"tasks": [
    {
      "type": "npm",
      "script": "compile:watch",
      "problemMatcher": ["$tsc-watch"],
      "isBackground": true,
      "group": {
        "kind": "build",
        "isDefault": true
      }
    }
  ]

So my actual question is why do i need paths as well as references?
Paths is then also needed to plugin in to ts-jest settings

ps Thanks for all the great typescript work so far though as love developing react/react native with it.

  • 👍1
unional commented

I get xxx is not in project file list error when enabling composite:

tsconfig.json

{
  "compilerOptions": { "composite": true, ... },
  "files": [ "src/index.ts" ]
}

Before enabling, this would work as tsc will follow the dependencies in src/index.ts.

After enabling, all files referenced throws an error:

error TS6307: File 'xxx' is not in project file list. Projects must list all files or use an 'include' pattern.

If I have to include src, this causes all test files next to the source files included in the build.

Is this a bug? If so, should I file it separately? 🌷

  • 👀1
andrewbranch commented

@unional from what I understand, this is intentional; not a bug:

All implementation files must be matched by an include pattern or listed in the files array. If this constraint is violated, tsc will inform you which files weren’t specified

(from https://www.typescriptlang.org/docs/handbook/project-references.html#composite)

I believe this constraint helps tsc figure out the project dependency tree more quickly/easily. You might find include more convenient than files, and it works best when projects are grouped into folders.

  • 👍1
sheetalkamat commented

Right now we only use declaration maps to jump to source files instead of declarations. Its working as intended at moment but could change in future. #25844 will handle out of date declaration maps to not crash.

AzzkiyOne commented

There is a caveat using project references.

Imagine monorepo setup with packages A and B where A is dependent on B. In tsconfig of A there is a reference to B.

Now when i want to use a different version of B in A (that version will be downloaded from npm) i get in a situation where A references version of B that is located in monorepo but actually uses version of B that is downloaded from npm. This can cause many troubles by providing wrong typings info.

screendriver commented

@yonilerner @jonaskello the --watch behavior change is likely to come in 3.3

Great! The current behavior is especially annoying when you want to run unit tests in watch mode.
Because tsc --build --watch in combination with project references compiles every file again (instead of just the changed ones) and updates the timestamp on every compiled output file. That leads to running the watched unit tests multiple times because every project in my project references gets updater one after another.

  • 👍2
dgreene1 commented

It seems quite unworkable if true that with project references you can only resolve from the index, not direct to a file e.g.

Just wanted to share that this seems like a good thing that you can't use things that weren't made "public." I think of it as an enforcement of no-submodule-imports from ts-lint: palantir/tslint#3051

The ability to prevent coupling future proofs a team like mine without having to resort to separate repos or microservices with brittle CI/CD pipelines.

Anyway, I'm really looking forward to these features. Great work @RyanCavanaugh and the team! TypeScript makes coming to work every day an absolute joy! :)

  • 👍1
pd4d10 commented

How to make tsc --build always emit output files, whether or not there is an error?

It is different from tsc at this point.

Tried to set noEmitOnError: false, but seems not work

sgronblo commented

I've now spent a whole day trying to get the following set up to work. I have a small "monorepo" test repository that contains three "projects" (I've seen many other users mention having similar structures):

  • common
  • frontend -> depends on common, should be built by webpack
  • backend -> depends on common, ideally I would like to be able to build a docker image from this with the required node_modules included

Requirements:

  1. I would like to be able to use yarn workspaces for this but that presents its own challenges in filtering out the node_modules that belong only to the backend project. I guess it's best to keep that part out of this discussion.
  2. I would like to be able to import common with an absolute import like import * as common from "common" so that in the case that we would ever split common out of the monorepo we wouldn't need to fix all the paths.

One of the main questions I have now is how do "project references" interact with "path mappings". I noticed that if I put

paths: {
  "common": ["../common/src"]
}

I am able to do the relative import of common/src/index.ts from backend/src/server.ts through the above mentioned import * as common from "common" however this also gives me the ability to jump immediately to an implementation inside common, even when I don't have declarationMaps turned on for common.

Does that mean that accessing the code through a path mapping bypasses project references and I won't get the speed benefits?

jonaskello commented

@sgronblo I use yarn workspaces in a similar setup and it works well. I don't use any path mappings as they become redundant. However to make it work you cannot use any typescript plugins in webpack, instead just let webpack build/watch the JS files that tsc --build produces. In tsconfig.json make sure to set "outDir": "lib", and "rootDir": "src" for all packages. In package.json also make sure to set "main": "lib/index.js"and "types": "lib/index.d.ts".

I would also recommend that you prefix your own packages with a group name, something like @myapp/common, @myapp/frontend etc. This is only for the name field in package.json. You still keep the folder structure like myapp/packages/common, myapp/packages/frontend etc. Then when you run yarn it will create linked folders in node_modules/@myapp so it is easy to find your packages under node_modules. Becuase of these folder links import * from @myapp/common will just work without any path mappings as the common behaviour is to look upward for node_modules/{packagename}.

Also something I noticed is that it is important to set references correctly in tsconfig.json. If the packages build in incorrect order they will not find each other. For example if frontend builds beforecommon it will not find it as it is not built yet. So there must be a reference from frontend to common.

spion commented

One big blocker we still have for build mode is the requirement that declarations are turned on.

In our monorepo we have multiple "toplevel" packages (several backends, several frontends) that are not libraries, and they share multiple dependencies.

The backends and frontends cannot have declarations turned on. It would be a huge amount of unnecessary busy work to do it, to eliminate a bunch of TS4058 s i.e.

Return type of exported function has or is using name 'X' 
from external module "/path/to/external/module" but cannot be named.

This in turn means we cannot add them to the composite project. As a result, they somehow have to be built and watched separately from "build mode" which complicates the whole setup and negates the benefits for large subset of our code.

Perhaps build mode could take several toplevel (composite) project configs at once as an argument? That way we can make one per backend/frontend and pass them all.

  • 👍2
screendriver commented

With the latest version v3.3.3 we still have the issue #25600 (comment)

tsc --build --watch builds every project (all files within that project) where I changed a single file instead of just that one file. Is that now the expected behaviour?

  • 😕3
SimenB commented

Using this in Jest now while migrating from Flow to TypeScript. My only real complaint constructive feedback is that it's slow (emitDeclarationOnly currently takes a full minute on my beefy 15" 2018 MBP, and we're not done migrating so it'll keep increasing).
It'd be really nice if TSC provided some sort of progress meter. Some sort of 213123/123123123123 thing that updated whilst it was building would be awesome (similar to progress in webpack or yarn).

Incremental builds are fast (and watch works great), though!

  • 👍3
lo1tuma commented

Seems like 7b290fd introduced this behavior to update all last modified timestamps of all output files of a project even if the contents didn’t change. I don’t quite understand the explanation in the commit message and why this is necessary.
Running tsc --build -w --listEmittedFiles confirms this assumption, after changing a single file it lists all files within the project as being emitted.

  • 👍3
vilicvane commented

Enforcing "noEmitOnError" brings a lot of inconvenience, is this caveat temporary?

E.g. we have to create separate tsconfig files for incremental building that disables options like noUnusedLocals and noUnusedParameters.

Also this makes refactoring harder because we are expecting a lot of intermediate errors. And a dependant project will be out of sync with its dependency unless we fix all the errors in the dependency first.

  • 👍1
antl3x commented

Seems like 7b290fd introduced this behavior to update all last modified timestamps of all output files of a project even if the contents didn’t change. I don’t quite understand the explanation in the commit message and why this is necessary.
Running tsc --build -w --listEmittedFiles confirms this assumption, after changing a single file it lists all files within the project as being emitted.

No explanation about this folks?

  • 👍3
RyanCavanaugh commented

update all last modified timestamps of all output files of a project even if the contents didn’t change

Let's say the following sequence occurs on a solution containing projects core and ui, where ui depends on core:

  • 1 PM Clean build
  • 2 PM Modify core implementation (without modifying types)
  • 3 PM Build the entire solution with tsc --build
  • 4 PM Run tsc --build ui

Now, the 4 PM build should do nothing, because at 3 PM we built the entire solution, and nothing has changed.

The question is, how does that happen?

The timestamps on ui, naively, would indicate that its outputs are older than its inputs, because the 1 PM ui output is older than the changed 2 PM core build output upon which it depends.

But when we did the 2 PM build, we realized that ui is "still up to date" because core's types didn't change. At this point we effectively choose to do an instant free build by moving ui's timestamps forward.

It's identical to us doing a real build of ui, it just happens that we already know what the outputs will be because we have realized the upstream inputs are unchanged.

weswigham commented

@RyanCavanaugh I think the point is moreso that we shouldn't be updating the timestamps - while doing so makes our recordkeeping easier, it causes naive tooling watching those files to trigger when they don't necessarily need to (since the contents haven't changed).

sheetalkamat commented

@weswigham tsc -b depends on timestamps and is soley based on timestamps of inputs and outputs so not updating it is just wrong.

weswigham commented

@sheetalkamat How so? Doesn't it just mean that rather than relying solely on a file's mtime, we should be looking at the mtime(s) of the project(s) it depends on? If those projects mtime's are newer, then we determined we didn't need to reemit during the last build, and thus still don't need to?

sheetalkamat commented

@weswigham What if you cancelled your build after project reference was done building. Now on next turn you don't know if you skipped build or were not even gotten chance to determine if build is needed?

weswigham commented

Mmmm... you could avoid that issue if you transactionally update all outputs at once (or nothing). Eg, you output to a cache dir, and only once done do you then rename the cache dir to the output dir. Or you can write a lockfile that you only delete when fully done.

RyanCavanaugh commented

I think the point is moreso that we shouldn't be updating the timestamps - while doing so makes our recordkeeping easier, it causes naive tooling watching those files to trigger when they don't necessarily need to (since the contents haven't changed).

The fact of relying on timestamps is irrelevant to the downstream tool impact scenario. If our compiler could execute in 1ms and build any project instantly, a solution build would absolutely write these files unconditionally, even in the presence of unchanged output file contents. We are now sometimes able to do exactly that.

If those projects mtime's are newer, then we determined we didn't need to reemit during the last build, and thus still don't need to?

All of the timestamp-update-only code is dependent on noticing that the .d.ts content of an upstream project ended up being identical to what was already on disk while we were emitting it. It's not possible to replicate this behavior between invocations by examining timestamps without a sidecar status file, which was something we explicitly scoped out of non-incremental build.

peterjuras commented

I'm running into issues when converting our current code base to project references, because transpiled imports don't respect the outDir of the referenced projects. In this issue this has been declared as "working as intended" but I don't see us moving anywhere near composite projects as long as we can't get them working reliably.

I created an example repo that demonstrates our issue.

There are two projects in it a and b. a is supposed to build the source code in a way that a/index.js can be run with node. Since the build output does not rewrite imports and does not restructure the build output similar to how non-composite projects are resolved (i.e. re-creating the folder structure to make imports work) a/index.js can not be executed.

What is the use case for these project references? I was hoping that they would enable us to reference various TypeScript projects without worrying where they are located.

  • 👍2
octref commented

Can we possibly get an option to emit on error for project references?

Now everyday when I write code, whose intermediate state has unused errors 95% of the time, I can't run or debug the code because nothing is emitted.

http://www.typescriptlang.org/docs/handbook/project-references.html#caveats

if one of your out-of-date dependencies had a new error, you’d only see it once because a subsequent build would skip building the now up-to-date project.

Can't you mark that up-to-date project with some other flag to indicate it's not error free, and rebuild it on change?

My day-to-day pain is if I'm working on a project a that depends on b, c, d, even if I'm not making changes to b/c/d that would cause the above scenario, a never emits when I'm writing code. I literally had to run both tsc -b and tsc -p to get project a to emit.

  • 👍2
yujiang commented

Can we add a option to show the build progress
like: (12/50) abc.ts compiling...

  • 👍18
bajtos commented

A limitation we have recently discovered: "Find all references" and "Rename symbol" don't work across referenced projects, see #30823. Posting the link here to make the other issue easier to find for others.

I apologize for a possibly unwanted notification.

  • 👍4
unional commented

When specifying references, if the references do not resolve to a folder or tsconfig, the process should terminate with an error.

{
  "references": [{
    "path": "not-exist-location"
  }]
}

Currently, the build will succeed.

This makes identifying the root cause of kulshekhar/ts-jest#766 very hard to do.
(I'm suspecting there are some relative path issues for ts-jest running in a monorepo)

  • 👍4
AnyhowStep commented

I have a codebase with 34 sub-projects and I've noticed that there are times where watch will claim to have re-checked and recompiled code but the previous errors still show up.

  1. tsc -b -w /*sub-projects-here*/
  2. tsc complains about error at file X line Y
  3. Fix problem
  4. tsc detects change
  5. tsc claims to have re-checked, re-emitted
  6. tsc STILL complains about error at file X line Y
  7. Keep making changes and scratching head until tired
  8. Ctrl+C to close tsc
  9. Delete output folder
  10. tsc -b /*sub-projects-here*/
  11. Builds successfully

This case happens pretty often to me.
And constantly rebuilding isn't really feasible because a complete rebuild takes 9 minutes.

Without composite projects, I get out-of-memory errors, max instantiation depth errors, and probably other errors I can't think of at the moment. So, composite projects are pretty important to me.

Usually, it's something like,

  1. A depends on B.
  2. A has errors.
  3. Updating B fixes errors on A.
  4. tsc -b -w doesn't realize it and still complains about errors on A after rebuilding

However, when I'm not encountering this issue, tsc -b -w is pretty fast. (Unless I update a project high up on the project dependency graph)

It would also be nice to have #25600 (comment)

When the projects are re-checking/re-building, I have no idea what projects are being checked, and the progress. It isn't a problem when build times are fast. But since building can take up to 9 minutes, being able to monitor the progress with something other than a clock would be nice.

icopp commented

I still don't understand why the project references functionality is requiring me to build .d.ts files that aren't actually used for anything else just to get tsc -b to work for other packages in a working monorepo setup, when type inference in VS Code works fine without ever creating those .d.ts files. I feel like there's some total disconnect here in intended functionality that I just don't get.

  • 👍2
ahnpnl commented

If you're hosting the compiler API, you don't need to do anything new for project references; everything should work as-is. If you want to take advantage of the new --build mode features, the entry point is createSolutionBuilder and you'll need to provide a BuildHost and CompilerHost along with the file timestamp APIs on the latter.

To migrate to using project references in your code itself, updating your tsconfig.json to a) add references and b) add composite: true may be sufficient - if not, you'll see errors informing you what else needs to happen. Writing a comprehensive migration guide has been a difficult task because project setups are so varied - I'm hoping we can find migrations from early adopters as something to point people to, but it's hard to give guidance without seeing specific build layouts.

@RyanCavanaugh I see a few issues with the current typing of createSolutionBuilder

function createSolutionBuilder<T extends BuilderProgram>(host: SolutionBuilderHost<T>, rootNames: readonly string[], defaultOptions: BuildOptions): SolutionBuilder<T>;
  • The name rootNames input is very misleading. This input should be something like tsConfigPaths. Because this works:
const solution = ts.createSolutionBuilder(solutionBuilderHost, ['./tsconfig.json'], {
   incremental: true,
})

but this doesn't:

const solution = ts.createSolutionBuilder(solutionBuilderHost, compilerOptions.fileNames, {
   incremental: true,
})
  • defaultOptions with interface BuildOptions which has the last property is
[option: string]: CompilerOptionsValue | undefined;

This is unclear what else can be the valid build options

marijnh commented

Just to check whether what I'm seeing is expected... I tried converting my medium-size (20k lines) project to use references and tsc -b -w, on the assumption that it'd help the compiler do fast incremental (watch) builds. However, it ended up significantly slowing rebuilds when I edit one of the modules that a lot of other modules depend on—it looks like it unconditionally rebuilds all the dependents in that case, leading to tens-of-seconds rebuilds in some cases. This was a rather big regression compared to just running tsc -w without references (where rebuilds are sub-second). Is that just how this feature works, or am I doing something wrong?

  • 👍11
desmap commented

@RyanCavanaugh no offense but is anyone using project references in production? I am trying to setup a monorepo with this feature and I just don't get there. I looked at many examples but it's just not working. Now, I am hanging at the same bug described earlier: #25600 (comment) which has its own issue here #25864 but is surprisingly closed despite still being there and breaking the entire project reference feature

It's quite a tedious and frustrating experience I am not used to by TS

natew commented

@desmap I've had so many problems with a monorepo it's loony. Nearly constant, all sorts of issues referencing projects, with intellisense. You have to really, really, spend time and hunt down a lot of weird advice, and at best you have it half working.

Wish I'd saved a few more links to help you out. Only thing on the top of my head is look for the "multiple inheritance" tsconfig structure blog post, and typescript-monorepo-toolkit helps manage refs.

And god forbid you his a slowdown on one package/library, you'll spend a few frustrating nights figuring that out.

  • 👍2
vilicvane commented

@RyanCavanaugh has anyone suggested that TypeScript should provide project information of build error? Occasionally we need to know which referenced project, during build, is causing the build error due to different tsconfig.json.

A common scenario:

We have browser projects and Node.js projects using some third-party modules that requires some extra types to work. Most of the projects are configured correctly, but some of them are misconfigured. Now it's not that straightforward to find out the misconfigured ones.

unional commented

@desmap I'm doing this in multiple of my projects, and I don't have much issue. Here are some of these projects:

https://github.com/unional/standard-log
https://github.com/mocktomata/mocktomata

felipeochoa commented

Leaving some constructive feedback: I spent most of the day trying to set up monorepo using yarn workspaces ad tsc- b and finally gave up and went back to the normal build mode. Here's what I was trying to achieve:

3 packages, each with src and tsconfig.json compiling into */build directories

  • server: No dependents
  • frontend: No dependents
  • common: referred to by server and frontend

(I was trying to use yarn workspaces, as in RyanCavanaugh/learn-a#3)

The first wall I hit was when trying to do cross-package imports using import { X } from '@project/common/x'. (Which I think is a very common approach). I gave up after seeing #10866 et al, not wanting to add another build dep just for this.

Since TS won't rewrite the imports, I resigned myself to import { X } from '@project/common/build/x'. But then that led to TS6305: Output file '/common/build/x.d.ts' has not been built from source file '/common/src/x.ts'. I could get tsc happy by instead using import { X } from '@project/common/src/x', but of course that failed at runtime and it was back to square one. (I suspect this is related to #25864)

Curiously, even without project references "go to definition" is able to take me to the source file in common from call sites in server. Not sure if this is due to source-maps + symlinks interacting fortuitously, but that solves one major goal from this effort.

Incremental builds are still challenging, but we just run 3 different build processes for now. It's going to get painful as we add more packages though since we need to manually watch all build directories for nodemon to reload the dev server after any changes.