🚀 Breaking changes for Yarn 4

Question

🚀 Breaking changes for Yarn 4

arcanis opened this issue 3 years ago · 36 comments

Answer 1 · 2021-10-30T07:54:00.000Z

Drop Node 12 support (EOL will be 2022-04-30)

This is a no-go for OSS devs. E.g. I run vite-plugin-ssr's test suite against Node 12 to make sure vite-plugin-ssr works with Node 12.

Answer 2 · 2021-10-30T08:09:46.000Z

This is a no-go for OSS devs. E.g. I run vite-plugin-ssr's test suite against Node 12 to make sure vite-plugin-ssr works with Node 12.

We don't intend to support Node versions that aren't maintained anymore by Node themselves. Not only to make our life easier, but also because some of our dependencies do that too, and need to be kept updated (thus making such Node requirements a necessity).

Answer 3 · 2021-10-30T08:14:19.000Z

Ok makes sense. Even if v4 releases a couple of months earlier than 2022-04-30 that would be ok. Sorry for the noise.

Answer 4 · 2022-02-04T16:38:08.000Z

From this comment it sounds like Yarn 4 might also change the data format:

I'm considering switching to a pure JSON format in the next Yarn major (and we'd publish this JSON as a formal spec that third-party resolvers can rely upon)

I went looking for the corresponding issue for that and ended up here. Should that be on this list too?

Answer 5 · 2022-02-05T09:13:37.000Z

I added an item to the list 👍

Answer 6 · 2022-04-15T06:20:19.000Z

We'll be exploring a new resolution algorithm that will prevent most of the attacks similar to the recent color.js hijacking.
from https://dev.to/arcanis/yarn-32-libc-yarn-explain-next-major--o22

Is there any way to follow, the implementation or design or discussion? @arcanis

Answer 7 · 2022-04-17T02:40:22.000Z

The only installation instructions provided in the docs are for corepack. Since corepack is not always an option (security concerns, having no npm, etc.) it would be nice if the manual instructions could be added back.

Answer 8 · 2022-04-17T04:43:11.000Z

@pcjmfranken Corepack doesn't require npm. It's built into Node.js as of v14.19, which everyone should be using since v12 is EOL in a few days.

Answer 9 · 2022-04-17T07:50:32.000Z

@milesj Corepack is actually built into¹ the NPM that ships with the official Node distributions. Also, custom Node builds can opt out of Corepack's inclusion independently of NPM's.

You're right in that corepack should be available in the majority of environments, but it's certainly not a given.

You can see the files being added to the node_modules directory here: https://github.com/nodejs/node/blob/master/tools/install.py#L82-L121 ↩

Answer 10 · 2022-04-22T13:16:16.000Z

The default value change for enableGlobalCache should be well documented when yarn 4 is released. This tripped me up when switching since we sync our local cache folder into our docker dev environment and I was confused why docker didn't work anymore. I expect this might happen to a few other people.

Answer 11 · 2022-12-16T14:44:28.000Z

Looks like #4247, which bumped the cache key, was missed in this list.

Answer 12 · 2022-12-16T19:46:30.000Z

Looks like #4247, which bumped the cache key, was missed in this list.

Bumping the cache key isn't a breaking change.

Answer 13 · 2022-12-16T20:07:00.000Z

Is it just a coincidence that the last released cache key bump was done in a major version, and included a changelog entry? (I ask since that's really all I was going off of.)

Answer 14 · 2023-05-09T04:43:25.000Z

Hey @arcanis, could any of these changes be cherry picked into v3?

Answer 15 · 2023-05-09T13:09:08.000Z

No, breaking changes can't be backported.

Answer 16 · 2023-05-09T15:13:55.000Z

Yarn 4.0 first rc release is now over 1 year old. And there's been 43 rc releases since then. And the 4.0.0 milestone has no other opened issue.

Is there anything that is blocking a final release of Yarn 4.0.0 ? Is there a rough ETA of when this might happen ?

I am aware that we can use rc releases in the meantime, but it does look a bit unusual to have such a long period of time for rc. If it is stable, then it should probably be released, but if it isn't, then we probably should populate the milestone to reflect what is missing, shouldn't we ?

Answer 17 · 2023-05-09T16:46:11.000Z

Is there anything that is blocking a final release of Yarn 4.0.0 ?

Time! It's always time. Most of the work is done, but there's always a couple of smallish improvements we want to make around some "details". Myself I have a couple of things around JS constraints, and @merceyz mentioned he'd like to close #4952 first to make the migration smoother. There's also the website revamp, but it'll likely be shipped separately.

Is there a rough ETA of when this might happen ?

Probably soon, but no ETA.

Answer 18 · 2023-05-10T01:06:20.000Z

Myself I have a couple of things around JS constraints

Could you create issues assigned to 4.0.0 for those things ? then the community might be able to help, at least a bit ?

Answer 19 · 2023-07-08T17:20:50.000Z

Where I can find waiting issues to be done? Milestone is the right place? Because there is only this issue present there. https://github.com/yarnpkg/berry/milestone/2

Answer 20 · 2023-08-03T17:52:37.000Z

everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds

Answer 21 · 2023-08-03T18:08:46.000Z

yarn set version 4.0.0-rc.48

Answer 22 · 2023-08-03T18:15:22.000Z

everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds

yarn set version 4.0.0-rc.48

also yarn set version canary https://yarnpkg.com/cli/set/version

Answer 23 · 2023-08-03T19:48:34.000Z

where can i find the change logs

Answer 24 · 2023-08-28T17:40:22.000Z

Dependabot updates seem broken due to the cache key change, and each PR will include a mass replacement of zips with vendoring enabled.

I know it's technically unrelated, but thought it's useful to have it documented here.

Edit: When a release for v4 is created, they should pick it up as per here:

https://github.com/dependabot/dependabot-core/blob/main/npm_and_yarn/Dockerfile#L6-L7

Answer 25 · 2023-08-28T18:10:16.000Z

Will they? From the look of it they use Corepack, so if your package manager is pinned via either packageManager or yarnPath, you should only get updated PR after you migrate to v4, right?

Answer 26 · 2023-08-28T18:19:39.000Z

Will they? From the look of it they use Corepack, so if your package manager is pinned via either packageManager or yarnPath, you should only get updated PR after you migrate to v4, right?

@arcanis It would have been nice, but my guess is they are still using v3 even with the above configured. This is the behaviour I'm seeing:

With:

"packageManager": "yarn@4.0.0-rc.50" and yarnPath: .yarn/releases/yarn-4.0.0-rc.50.cjs

I will write an issue on their side.

Answer 27 · 2023-09-20T15:40:58.000Z

^ Just an update on the above, it was an oversight on my side that I eventually discovered - it was the difference between darwin & linux in the OS that was causing the replacement, not the key change.

I've set it in .yarnrc now, there's no issue with dependabot.

Answer 28 · 2023-10-22T20:32:02.000Z

Since yarn 4.0 stable is released in https://github.com/yarnpkg/berry/releases/tag/%40yarnpkg%2Fcli%2F4.0.0, this issue can be closed.

Answer 29 · 2023-10-22T22:13:17.000Z

getting an overview of what breaking changes are relevant to different use cases of yarn would be nice, and not just a list of the commits

Answer 30 · 2023-10-22T22:19:15.000Z

We will have a blog post tomorrow getting into the high level changes. I'll keep the issue open until then (we just wanted to tag the release beforehand to avoid having to rush a fix if something broke during the deploy process).

Answer 31 · 2023-10-23T07:15:53.000Z

This release made all of our builds to fail.

Here is an excerpt from one of our docker files:

FROM node:16.19.0-bullseye-slim

RUN yarn config set scripts-prepend-node-path true
RUN yarn set version 3.3.1

This image, seem to include the latest version of yarn (4 from several hours ago) and since we rely on 3.3.1 we try to set it.

This is the error that we see:

Step 7/18 : RUN yarn set version 3.3.1
 ---> Running in c2e50735b75c
Usage Error: This tool requires a Node version compatible with >=18.12.0 (got 16.19.0). Upgrade Node, or set `YARN_IGNORE_NODE=1` in your environment.
Yarn Package Manager - 4.0.0

Changing the line to RUN YARN_IGNORE_NODE=1 yarn set version 3.3.1 does fix the issue, just thought you should know about it

Answer 32 · 2023-10-23T07:17:38.000Z

@herzaso I think the error messages are clear enough. This is an expected behavior.

And do yourself a favor and upgrade Node.js to a version that has not reached end of life yet.

Answer 33 · 2023-10-23T07:19:26.000Z

@wojtekmaj as I said, the error was clear and we managed to fix, but enterprises move slow. We are in the process of moving to Node 20

Answer 34 · 2023-10-23T13:20:18.000Z

Here we go: https://yarnpkg.com/blog/release/4.0 !

Answer 35 · 2023-10-27T19:35:47.000Z

@arcanis Amazing! A few questions though, that aren't covered in the article, perhaps useful for people who are new to Yarn (maybe the post can be updated):

How do you upgrade to 4.0 from 2.x or 3.x? And from 1.x (EDIT: 1.x migration is already covered)? There are no steps listed. What are the migration steps?
- The post makes things a little confusing, because it says:
  
  To that end we used to recommend using the yarnPath setting pointing to a checked-in binary, but this pattern increased friction more than we liked
  
  When looking at the CLI docs, the only hint we see at updating to 4.0 is the yarn set version command. I think to myself, "this must be how we update the version". If we try yarn set version 4.0, that seems to work. However, it sets yarnPath in .yarnrc.yml, which seems to be opposite of what the 4.0 release post says is ideal. Basically, it is a little confusing because now it seems I've updated to 4.0 in the wrong way.
  
  The post then says
  
  Now that Corepack is shipped with both Node 18 and 20 we no longer need to rely on yarnPath, and as a result we updated our installation guide to reflect that
  
  However, when you look at the linked installation guide, it says that updating yarn is done with yarn set version stable. But if I want to ensure that I stay on the new 4.0, and not accidentally get 5.0 later and things breaking, this is why I assume I need to run yarn set version 4.0.0, and this creates a yarnPath in yarnrc, and adds the binary to .yarn/releases/, which contradicts the premise of the post.
  
  So the main question is, how do we update to 4.0, ensure we stay on 4.0, and not use yarnPath?
  
  Then the post says that plugins are shipped by default. Does that mean I must now run yarn plugin remove @yarnpkg/plugin-workspace-tools?
  
  What else?
  
  UPDATE: I now tried yarn set version stable, and just as with yarn set version 4.0.0, it creates the supposedly-unwanted yarnPath config.
  
  UPDATE 2: I undid all changes of yarn set version ____, and instead I updated the packageManager field in package.json manually. Now when I run yarn --version I see 4.0.0 without a binary being installed in .yarn/releases/. Furthermore, upon running yarn install, I see:
```
➤ YN0048: Automatically removed core plugins that are now builtins 👍

➤ YN0087: Migrated your project to the latest Yarn version 🚀
```
  It turns out that Yarn automatically removed the no-longer-needed plugins.
  
  UPDATE 3: There is something a bit odd after running yarn install however: the yarn.lock is modified with a ton of changes, and every single package has its checksum hash modified. This is very unexpected.
  
  Here's a piece of output that is unexpected during that yarn install:
```
➤ YN0013: │ 2838 packages were added to the project, and 2014 were removed (+ 2.95 MiB).
```
  UPDATE 4: Another discrepancy:
  
  The post says that Zero-Installs is no longer default. However, when I try yarn init -2 in an empty folder, and yarn add any-package, I still see pnp files (that happen to be git ignored). This is confusing. When I think of not having Zero-Installs, I think that I will have a node_modules folder, but that is not the case. If I don't have Zero-Installs, but I also delete .pnp files, nothing will work.
  
  What's the point? Why would I want to have .pnp files, and not check them in?? The only reason that I can think of is that I need to have node_modules. That's why I set nodeLinker: node-modules in yarnrc, but I was thinking that it would be the default in new installs (for compatibility with the entire existing JS ecosystem of tooling that largely has no idea what .pnp files are).
  
  The concept of node_modules is widely used by both backend and frontend projects, so the .pnp default seems like a bad default, especially for the growing number of frontend projects that use importmaps and fetch files from node_modules. So by merely gitignoring .pnp files, it seems nothing great was actually achieved, it only makes people need to yarn install (in which case you may as well be compatible with the entire ecosystem by using node_modules in that mode).
Many warnings are gone (according to the article, haven't tried yet), but I thought they were useful because other package managers did not show them, so I enjoyed having more insight into my install setup. How do we restore it?
The post mentions official plugins. However, I cannot find a page on the website describing the purpose of plugins (what does it allow people to do?). Where is the list of official plugins and their descriptions? The only pages I can find via the search bar are the yarn plugin CLI docs (does not explain the purpose of plugins) and a tutorial page on writing a plugin (but knowing how to make the shell for a plugin is not useful without knowing what we will do in writing a plugin).
- EDIT: I found a cache of an old Yarn wesite describing the purpose of plugins, but this seems to be missing from the new website.

It would be great for each release post to also double as a complete migration guide.

Answer 36 · 2023-10-27T21:06:47.000Z

@trusktr this is interesting feedback, but since this thread was specifically to talk about the breaking changes in Yarn 4, I don't think we should start discussing the content of the blog post itself (there are 23+ other people subscribed to this thread, and I don't want to spam them; additionally, an issue is likely the wrong format since you have multiple points that would lead to interlaced and very hard to follow discussion).

Footnotes