Angular CLI and monorepos (like lerna, yarn workspaces)

Question

Angular CLI and monorepos (like lerna, yarn workspaces)

dherges opened this issue 8 years ago · 27 comments

Bug Report or Feature Request (mark with an `x`)

- [x] bug report -> please search issues before submitting
- [x] feature request

Versions.

$ ng --version
    _                      _                 ____ _     ___
   / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
  / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
 / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
/_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
               |___/
@angular/cli: 1.3.0-beta.0
node: 6.10.1
os: darwin x64
@angular/animations: error
@angular/common: error
@angular/compiler: error
@angular/core: 4.3.1
@angular/forms: error
@angular/http: error
@angular/platform-browser: error
@angular/platform-browser-dynamic: error
@angular/router: error
@angular/cli: error
@angular/compiler-cli: error
@angular/language-service: error

This in itself manifests the error.

Repro steps.

A full repro can be found at GitHub repo spektrakel-blog/a-glimpse-at-yarn-workspaces

Initially, faced the "You seem to not be depending on @angular/core" error.
This goes down to a sanity check whether Angular is installed as a local dependency.

However, there are more errors with yarn workspaces, when dependencies from a workspace (or "sub-project", or "sub-package") are installed to the top-most node_modules directory of the workspaces root.

Then, it ends up w/ file structure:

Workspace root package.json:

  "workspaces": [
    "packages/*",
    "demo"
  ]

Demo workspace package.json:

{
  "name": "demo",
  "version": "0.0.0",
  "license": "MIT",
  "private": true,
  "dependencies": {
    "@angular/common": "^4.2.4",
    "@angular/core": "^4.2.4",
    "@angular/forms": "^4.2.4",
    "@angular/http": "^4.2.4"
  }
}

Now, dependencies are installed to node_modules and not to demo/node_modules:

$ ls -a demo/node_modules/
.               ..              .bin            .yarn-integrity
$ ls -a node_modules/@angular
.                        common                   forms                    platform-browser-dynamic
..                       compiler                 http                     router
animations               compiler-cli             language-service         tsc-wrapped
cli                      core                     platform-browser

Then, running ng build from the demo folder errors:

$ cd demo
$ yarn build
yarn build v0.27.5
$ ng build
You seem to not be depending on "@angular/core". This is an error.
error Command failed with exit code 2.

As a workaround, it's possible to symlink "@angular/core":

$ mkdir -p ./node_modules/@angular
$ ln -sf ../../../node_modules/@angular/core ./node_modules/@angular/core
$ ls -l node_modules/@angular/core
lrwxr-xr-x  ... node_modules/@angular/core -> ../../../node_modules/@angular/core
$ cat node_modules/@angular/core/package.json
{
  "name": "@angular/core",
  "version": "4.3.1",
  "description": "Angular - the core framework",
  "main": "./bundles/core.umd.js",
  ...
}

But then we only get one error further until:

$ yarn build
yarn build v0.27.5
$ ng build
Hash: a90d0476b33232a953c1
Time: 53409ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {3} [initial] [rendered]
chunk    {1} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {3} [initial] [rendered]
chunk    {2} main.bundle.js, main.bundle.js.map (main) 1.89 MB [initial] [rendered]
chunk    {3} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

ERROR in Error encountered resolving symbol values statically. Function calls are not supported. Consider replacing thefunction or lambda with a reference to an exported function (position 194:50 in the original .ts file), resolving symbol NgModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/core/core.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts

Which suggests that Angular CLI isn't really meant to work w/ "sub-projects" like yarn worksapces.

The log given by the failure.

See above.

Desired functionality.

Angular CLI and webpack should resolve depndencies from "top-level" node_modules.
The sanity check should honor node's module resolution algorithm (looking up recursively up the file tree).

Mention any other details that might be useful.

Related to #6504

A related change was implemented in #6475 (resolve in all available node_modules), but it seems that it does not address this use case.

Answer 1 · 2017-07-22T15:58:13.000Z

At this point, this is me primarily asking:

"How is Angular CLI meant to work with monorepos?"

It the answer is "please use Angular CLI from the monorepo root directory", I can live by that.

Answer 2 · 2017-07-28T16:43:11.000Z

I'm not really familiar with the Yarn workspaces, but the CLI currently attributes meaning to the location both both .angular-cli.json and package.json.

It expects them to be in the same dir, assumes that dir is the project root, and that there's also a node_modules there. These requirements are pretty pervasive throughout the CLI.

So no, I don't think the CLI is currently well equipped to work in monorepos.

But that's not because we don't want it to, just because it wasn't ever much of a concern. If there's a reasonable way of making it work I'm all for it really.

I think some of the CLI users are using it with Lerna, but I don't know the details of it.

The important bits of monorepo support is that it's not really specific to any single setup (like Lerna or these Yarn workspaces), and that it doesn't compromise the current setup.

Answer 3 · 2017-07-29T05:50:25.000Z

Regarding my original post: I need to check whether preserveSymlinks offers an easy solution / workaround. #7194 #7081

Regarding discussion:

What may happen in monorepos (at least in yarn workspaces) that they install "(workspace-)local" and "(project-)global" dependencies. Example:

|- aio
    |- node_modules
        |- @angular
            |- http # <- 4.3.2
    | package.json
|- node_modules
    |- @angular
        |- common # <- 4.3.2
| package.json
|- packages
   |- my-lib
       |- node_modules
           |- @angular
                |- common # <- 4.2.0
       | package.json

Let aside potential version conflicts (which are in the hand of the user).

The difficulty for the webpack build would be to resolve modules "by walking up the tree until it finds one". I remember from past custom webpack configs that you had to pass the location of the node_modules. Was it in resolve?

https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/models/webpack-configs/common.ts#L91

Is this even possible to resolve modules from different directories?

Answer 4 · 2017-07-29T09:29:10.000Z

I think it's possible, yeah. But now that got me thinking how that happens with peer deps. In your example there's different versions of packages that really want the same version of the peer deps. So through node module resolution you'd end up getting the different versions.

But even though the module resolution might work, I think you'd get.... a static analysis error. Which is what you actually got initially:

$ yarn build
yarn build v0.27.5
$ ng build
Hash: a90d0476b33232a953c1
Time: 53409ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {3} [initial] [rendered]
chunk    {1} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {3} [initial] [rendered]
chunk    {2} main.bundle.js, main.bundle.js.map (main) 1.89 MB [initial] [rendered]
chunk    {3} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

ERROR in Error encountered resolving symbol values statically. Function calls are not supported. Consider replacing thefunction or lambda with a reference to an exported function (position 194:50 in the original .ts file), resolving symbol NgModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/core/core.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts

This wasn't just an error finding a module, it was an error finding a module while using static analysis to find lazy loaded modules (we do that on every build, not just AOT).

So I ask... what happens in your setup when you install all the @angular/* packages at a given level, satisfying the peerdeps? It might just work.

And if that works the only thing we need to do to support it is to relax the You seem to not be depending on "@angular/core". This is an error. one, by performing some better @angular/core checks (https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/upgrade/version.ts#L85-L115).

See where it tries to use path join with the project root? We should be able to use node module resolution instead like https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/utilities/require-project-module.ts.

Answer 5 · 2017-07-29T17:25:26.000Z

@filipesilva Ack.

|- demo
    | .angular-cli.json
    |- node_modules
        |- @angular # symlink to  ../../node_modules/@angular
|- node_modules
    |- @angular

$ ng build
WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

So still the same error.

$ ng build --preserve-symlinks
Your global Angular CLI version (1.3.0-rc.3) is greater than your local
version (1.3.0-beta.0). The local Angular CLI version is used.

To disable this warning use "ng set --global warnings.versionMismatch=false".
Hash: d97b10ba4761c0902221
Time: 19229ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {4} [initial] [rendered]
chunk    {1} main.bundle.js, main.bundle.js.map (main) 94.3 kB {3} [initial] [rendered]
chunk    {2} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {4} [initial] [rendered]
chunk    {3} vendor.bundle.js, vendor.bundle.js.map (vendor) 1.8 MB [initial] [rendered]
chunk    {4} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

So with --preserve-symlinks the build is fine!

Also AoT / Prod build!

$ ng build --preserve-symlinks --aot --prod
Your global Angular CLI version (1.3.0-rc.3) is greater than your local
version (1.3.0-beta.0). The local Angular CLI version is used.

To disable this warning use "ng set --global warnings.versionMismatch=false".
Hash: 31d53744fb697d9b9f87
Time: 19407ms
chunk    {0} polyfills.3b4be225e7f6a233ebb3.bundle.js (polyfills) 177 kB {4} [initial] [rendered]
chunk    {1} main.a772681d78c70ca6637d.bundle.js (main) 101 kB {3} [initial] [rendered]
chunk    {2} styles.d41d8cd98f00b204e980.bundle.css (styles) 69 bytes {4} [initial] [rendered]
chunk    {3} vendor.e839d82f8b5326dc3f41.bundle.js (vendor) 761 kB [initial] [rendered]
chunk    {4} inline.6eaed4d70fc63a0f7481.bundle.js (inline) 0 bytes [entry] [rendered]

Answer 6 · 2017-07-31T10:15:41.000Z

Ok, so this is progress. I take it the whole partial dep thing still won't work too well (for peer deps at least) but at least stuff seems to work if we tell webpack to pretend symlinks aren't really there.

BTW --preserve-symlinks was the work of @clydin, so big thanks to him for enabling this.

Answer 7 · 2017-08-10T09:54:49.000Z

This was merged recently in TS and is probably available in nightly so I think the symlinks won't be required anymore: microsoft/TypeScript#16274

Edit: this doesn't solve what I thought it would :(

Answer 8 · 2017-08-14T18:06:34.000Z

Similar to feature request #6083

Answer 9 · 2018-03-04T14:21:15.000Z

@filipesilva

I'm not really familiar with the Yarn workspaces

If you want to make some tests without having to worry about all that, a really simple monorepo for backend/frontend like the following, does not work:

| package.json
| node_modules/
| backend/
| frontend/
  | .angular-cli.json

(then running from root folder ng serve/build/test)

Answer 10 · 2018-03-05T09:52:06.000Z

just fyi, for packages that do not support the module hoisting done in yarn workspaces there is now a nohoist flag to prevent it https://yarnpkg.com/blog/2018/02/15/nohoist/

Answer 11 · 2018-07-02T15:24:21.000Z

Not quite sure if this is related, but I get the following error:

Could not find API compiler-cli, function VERSION

I use lerna with yarn workspaces. I currently have the angular.json inside the subfolder of my web package:

~/packages/web/angular.json

Anyone found a workaround yet?

Answer 12 · 2018-07-03T18:54:01.000Z

working root package.json

{
  "private": true,
  "scripts": {
    ...
  },
  "workspaces": {
    "packages": [
      "libs/*",
      "apps/*"
    ],
    "nohoist": [
      "**/@angular*",
      "**/@angular*/**"
    ]
  }
}

Answer 13 · 2018-08-29T11:17:42.000Z

I'm subscribing here, trying to use Yarn workspaces. nohoist indeed works as a temporary fix - at the expense of disk cost.

Answer 14 · 2019-01-14T12:32:41.000Z

Hello,

I'm working with angular 7.2.x and I think that problem was fixed on this version, as I see here : #11685

But I still can not get it to work, and I would like to avoid using nohoist.
I have built a very simple repo showing the problem here : demo
I am new to angular and even more to yarn so I may have done a mistake somewhere, but the example is very simple.
I created one yarn workspace, containing a new angular application, which can't find the @angular/ packages because the node_modules directory is in the parent (root) directory.

If someone is able to say me what isn't right, or if I still need to use nohoist, that would be very helpfull :)
Thank you !

Answer 15 · 2019-01-17T00:06:13.000Z

I had a similar issue.
I would love to create a symlink with @angular in the package but I did not figure out how to do that.
So I did a global search for node_modules/@angular.
It turns out that I had a few references in the angular.json file.
I set my reference to the package in the root.
Ie.
"styles": [ { "input": "./../../node_modules/@angular/material/prebuilt-themes/indigo-pink.css" }, "src/styles.scss" ].
I did this for the $schema, styles (under test and architect.)
I have compiled my project and this seems to work.

Answer 16 · 2019-06-13T22:02:43.000Z

I'm having an issue I think related to this one. I'm trying Ivy, in a lerna workspace. With ng serve everything works fine. However, when I try to run ng run build throws:

Date: 2019-06-13T21:52:59.215Z
Hash: 55c7e38a65bd19305a57
Time: 9864ms
chunk {0} runtime-es5.9c308a63d02029c20228.js (runtime) 1.41 kB [entry] [rendered]
chunk {1} main-es5.4af9b61479361f268d39.js (main) 128 bytes [initial] [rendered]
chunk {2} polyfills-es5.76fb5c306a2dd7f67a99.js (polyfills) 68.1 kB [initial] [rendered]
chunk {3} styles.691cb89d8238aaa5586f.css (styles) 63.4 kB [initial] [rendered]

ERROR in ../../node_modules/@angular/material/toolbar/typings/toolbar-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of AppMaterialModule, but could not be
resolved to an NgModule class
../../node_modules/@angular/material/toolbar/typings/toolbar-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of AppMaterialModule, but could not be resolved
to an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/button/typings/button-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be resolved to an NgModule class
../../node_modules/@angular/material/card/typings/card-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be resolved to
an NgModule class
../../node_modules/@angular/material/form-field/typings/form-field-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be
resolved to an NgModule class
../../node_modules/@angular/material/input/typings/input-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be resolved to an NgModule class
../../node_modules/@angular/material/select/typings/select-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be resolved to an NgModule class
../../node_modules/@angular/material/tabs/typings/tabs-module.d.ts(8,22): error TS-996002: Appears in the NgModule.imports of SharedMaterialModule, but could not be resolved to
an NgModule class
../../node_modules/@angular/material/button/typings/button-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be resolved to an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/card/typings/card-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be resolved to
an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/form-field/typings/form-field-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be
resolved to an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/input/typings/input-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be resolved to an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/select/typings/select-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be resolved to an NgModule, Component, Directive, or Pipe class
../../node_modules/@angular/material/tabs/typings/tabs-module.d.ts(8,22): error TS-996003: Appears in the NgModule.exports of SharedMaterialModule, but could not be resolved to
an NgModule, Component, Directive, or Pipe class
src/app/shared/shared-material.module.ts(24,14): error TS-996002: Appears in the NgModule.imports of SharedModule, but itself has errors
src/app/shared/shared-material.module.ts(24,14): error TS-996003: Appears in the NgModule.exports of SharedModule, but itself has errors
src/app/shared/shared.module.ts(13,14): error TS-996002: Appears in the NgModule.imports of AuthModule, but itself has errors
src/app/app-material.module.ts(10,14): error TS-996002: Appears in the NgModule.imports of AppModule, but itself has errors

I ran ivy-ngcc with the source set to the lerna root node_modules, and it ran fine. However, seems like the @angular/material it's not being upgraded.

Answer 17 · 2020-04-06T04:40:03.000Z

nohoist - this solution is not works for me.
Because when I do nohoist then node_modules is create in the apps level:

apps
- ng-app
  - node_moduels
packages.

So in packages, I got an error: Cannot find module '@angular/...

This is because @angular doesn't exist in the root node_modules

Any ideas how to solve this problem? please

@filipesilva angular is v9, and still doesn't support lerna/yarn? :(

Answer 18 · 2020-08-27T17:48:06.000Z

I think there have been recent updates to Angular because I'm using Ivy with no problems in a yarn workspace.

The only remaining issue for me was that ng update wasn't working in a monorepo. I opened a PR at #18610 to fix it.

Answer 19 · 2020-10-09T12:27:41.000Z

@andreialecu Do your changes enable ng update to work with a project where dependencies are hoisted to the monorepo root instead of being installed in the same directory as angular.json?

Answer 20 · 2020-10-09T12:34:09.000Z

@diminutivesloop as far as I remember, yes. That was the main reason behind opening that PR.

Answer 21 · 2022-02-17T19:47:09.000Z

Angular now has direct support for monorepos, so that should fulfill this use case: https://angular.io/guide/file-structure#multiple-projects.

We don't directly support Yarn / Lerna workspaces, but Angular's built-in support with the CLI should satisfy the same use cases. If there are significant feature gaps that require Yarn / Lerna, then please file a separate issue about it and we can try to support those use cases directly.

Answer 22 · 2022-02-18T12:06:35.000Z

Angular now has direct support for monorepos, so that should fulfill this use case: https://angular.io/guide/file-structure#multiple-projects.

We don't directly support Yarn / Lerna workspaces, but Angular's built-in support with the CLI should satisfy the same use cases. If there are significant feature gaps that require Yarn / Lerna, then please file a separate issue about it and we can try to support those use cases directly.

The most significant gap IMO is the ability to run multiple projects with one command, in the right order (dependencies) and in parallel if possible.

A while back I wrote a CLI that did exactly that on top of Angular CLI as an exercise, so I am pretty sure it should be doable.

However, that request already exists: #11002

Answer 23 · 2022-02-18T14:49:34.000Z

Can't you already achieve that with nrwl/nx? It comes with a myriad of commands for testing/linting/building only things that were affected, running things in parallel, looking at dep graphs etc

Answer 24 · 2022-02-18T15:45:06.000Z

Can't you already achieve that with nrwl/nx? It comes with a myriad of commands for testing/linting/building only things that were affected, running things in parallel, looking at dep graphs etc

Yes of course, the thread I linked acknowledged nrwl as well I think.

But the question was about a significant feature missing in Angular CLI but is supported in other monorepo environments, so I think my point stands.

but Angular's built-in support with the CLI should satisfy the same use cases

If Angular CLI built-in support claims to satisfy the same use cases, I think a simple command to build a library with its dependencies should be maybe supported? 🤔

NX is good of course, but again if someone would like to use Angular as plain as it gets, I think this feature could be quite beneficial.

Answer 25 · 2022-02-18T18:43:11.000Z

We're aware that having to ng build multiple projects in the right order can be quite annoying and easy to forget. We're planning to investigate what we can do to improve this, but don't have an expected timeline yet.

Answer 26 · 2022-02-18T18:56:23.000Z

We're aware that having to ng build multiple projects in the right order can be quite annoying and easy to forget. We're planning to investigate what we can do to improve this, but don't have an expected timeline yet.

@dgp1130 I'd be more than willing to help in this if help is needed. I can as well publish my private CLI for inspiration.

That CLI basically reads the source code and builds the dependency tree and then from that can execute the libraries the the proper order. As well I added the feature to run the builds in parallel when possible.

Expand to see the examples

parallel-builds.mov