๐ Breaking changes for Yarn 4
arcanis opened this issue ยท 36 comments
This issue is just there to keep track of which changes we might want to do once we'll get to the next major release. Doesn't mean it'll be anytime soon ๐ Will be updated as time passes and we think about more things.
- #4196
- #4218
- #4262
- #4261 (Removes
yup
) - #4252
- #4254
- #4302
- #4299
- #4305 (uses
npm:
everywhere) - #4253
- #3004
- #3691
- #4320
- #4589
- #4834
- #4524
- #4982
- Make commands git-aware again
- Make
yarn init
initialize a git repository (already the case in 3.x) - Make
yarn version
create tags for the relevant workspaces
- Make
-
Installer['getCustomDataKey']
: "Move this method intoLinker
so that linkers can use it to save some state useful to findPackageLocator (cf PnpmLinker)." - Make
ReportError
lazily generate report messages when provided a configuration object by the streams; this would allow us to remove theconfiguration
parameter from many helper functions where we only use it for display purposes. - Update
yarn workspaces foreach
recursive configuration; it's not clear at the moment what should happen when-R
and--since
are used together. Perhaps we'd need a--recursive-dependents
and--recursive-dependencies
flag? - #4595
- Cleanup #3612
- #4641
- Discuss the
fetchPackageFromCache
options bag.
- Transfer
arcanis.vscode-zipfs
to the Yarnpkg org - Make Yarn detect whether it's running inside a public repository (GitHub Actions) and, if it is AND the repository uses zero-installs, exit and recommend adding either
--check-cache
or--no-check-cache
. - Write a formal spec for the PnP data files, so that 3rd-party resolvers (in particular native resolvers) can implement their own (originally I pushed for it to be a JS API so that we had more freedom to extend/correct the file format as needed, but in 3.x we didn't need to change it so it might now be stable enough that we can relinquish some control).
- #4671
-
Consider only writing this data file by default, unless an option to generate an old-style(nope).pnp.cjs
file is set.
- #4503
- #4639
- #4644
- #4645
- #4648
- #4649
- #4697
- #4698
- #4773
- #4774
- #4865
- #4898
- #4899
- #5604
- #5608
- #5619
Drop Node 12 support (EOL will be 2022-04-30)
This is a no-go for OSS devs. E.g. I run vite-plugin-ssr
's test suite against Node 12 to make sure vite-plugin-ssr
works with Node 12.
This is a no-go for OSS devs. E.g. I run
vite-plugin-ssr
's test suite against Node 12 to make surevite-plugin-ssr
works with Node 12.
We don't intend to support Node versions that aren't maintained anymore by Node themselves. Not only to make our life easier, but also because some of our dependencies do that too, and need to be kept updated (thus making such Node requirements a necessity).
Ok makes sense. Even if v4 releases a couple of months earlier than 2022-04-30
that would be ok. Sorry for the noise.
From this comment it sounds like Yarn 4 might also change the data format:
I'm considering switching to a pure JSON format in the next Yarn major (and we'd publish this JSON as a formal spec that third-party resolvers can rely upon)
I went looking for the corresponding issue for that and ended up here. Should that be on this list too?
I added an item to the list ๐
We'll be exploring a new resolution algorithm that will prevent most of the attacks similar to the recent color.js hijacking.
from https://dev.to/arcanis/yarn-32-libc-yarn-explain-next-major--o22
Is there any way to follow, the implementation or design or discussion? @arcanis
The only installation instructions provided in the docs are for corepack. Since corepack is not always an option (security concerns, having no npm, etc.) it would be nice if the manual instructions could be added back.
@pcjmfranken Corepack doesn't require npm. It's built into Node.js as of v14.19, which everyone should be using since v12 is EOL in a few days.
@milesj Corepack is actually built into1 the NPM that ships with the official Node distributions. Also, custom Node builds can opt out of Corepack's inclusion independently of NPM's.
You're right in that corepack should be available in the majority of environments, but it's certainly not a given.
Footnotes
-
You can see the files being added to the
node_modules
directory here: https://github.com/nodejs/node/blob/master/tools/install.py#L82-L121 โฉ
The default value change for enableGlobalCache
should be well documented when yarn 4 is released. This tripped me up when switching since we sync our local cache folder into our docker dev environment and I was confused why docker didn't work anymore. I expect this might happen to a few other people.
Looks like #4247, which bumped the cache key, was missed in this list.
Bumping the cache key isn't a breaking change.
Is it just a coincidence that the last released cache key bump was done in a major version, and included a changelog entry? (I ask since that's really all I was going off of.)
Hey @arcanis, could any of these changes be cherry picked into v3?
No, breaking changes can't be backported.
Yarn 4.0 first rc release is now over 1 year old. And there's been 43 rc releases since then. And the 4.0.0 milestone has no other opened issue.
Is there anything that is blocking a final release of Yarn 4.0.0 ? Is there a rough ETA of when this might happen ?
I am aware that we can use rc releases in the meantime, but it does look a bit unusual to have such a long period of time for rc. If it is stable, then it should probably be released, but if it isn't, then we probably should populate the milestone to reflect what is missing, shouldn't we ?
Is there anything that is blocking a final release of Yarn 4.0.0 ?
Time! It's always time. Most of the work is done, but there's always a couple of smallish improvements we want to make around some "details". Myself I have a couple of things around JS constraints, and @merceyz mentioned he'd like to close #4952 first to make the migration smoother. There's also the website revamp, but it'll likely be shipped separately.
Is there a rough ETA of when this might happen ?
Probably soon, but no ETA.
Myself I have a couple of things around JS constraints
Could you create issues assigned to 4.0.0 for those things ? then the community might be able to help, at least a bit ?
Where I can find waiting issues to be done? Milestone is the right place? Because there is only this issue present there. https://github.com/yarnpkg/berry/milestone/2
everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds
yarn set version 4.0.0-rc.48
everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds
yarn set version 4.0.0-rc.48
also yarn set version canary
https://yarnpkg.com/cli/set/version
where can i find the change logs
Dependabot updates seem broken due to the cache key change, and each PR will include a mass replacement of zips with vendoring enabled.
I know it's technically unrelated, but thought it's useful to have it documented here.
Edit: When a release for v4 is created, they should pick it up as per here:
https://github.com/dependabot/dependabot-core/blob/main/npm_and_yarn/Dockerfile#L6-L7
Will they? From the look of it they use Corepack, so if your package manager is pinned via either packageManager
or yarnPath
, you should only get updated PR after you migrate to v4, right?
Will they? From the look of it they use Corepack, so if your package manager is pinned via either
packageManager
oryarnPath
, you should only get updated PR after you migrate to v4, right?
@arcanis It would have been nice, but my guess is they are still using v3 even with the above configured. This is the behaviour I'm seeing:
With:
"packageManager": "yarn@4.0.0-rc.50"
and yarnPath: .yarn/releases/yarn-4.0.0-rc.50.cjs
I will write an issue on their side.
^ Just an update on the above, it was an oversight on my side that I eventually discovered - it was the difference between darwin & linux in the OS that was causing the replacement, not the key change.
I've set it in .yarnrc
now, there's no issue with dependabot.
Since yarn 4.0 stable is released in https://github.com/yarnpkg/berry/releases/tag/%40yarnpkg%2Fcli%2F4.0.0, this issue can be closed.
getting an overview of what breaking changes are relevant to different use cases of yarn would be nice, and not just a list of the commits
We will have a blog post tomorrow getting into the high level changes. I'll keep the issue open until then (we just wanted to tag the release beforehand to avoid having to rush a fix if something broke during the deploy process).
This release made all of our builds to fail.
Here is an excerpt from one of our docker files:
FROM node:16.19.0-bullseye-slim
RUN yarn config set scripts-prepend-node-path true
RUN yarn set version 3.3.1
This image, seem to include the latest version of yarn (4 from several hours ago) and since we rely on 3.3.1 we try to set it.
This is the error that we see:
Step 7/18 : RUN yarn set version 3.3.1
---> Running in c2e50735b75c
Usage Error: This tool requires a Node version compatible with >=18.12.0 (got 16.19.0). Upgrade Node, or set `YARN_IGNORE_NODE=1` in your environment.
Yarn Package Manager - 4.0.0
Changing the line to RUN YARN_IGNORE_NODE=1 yarn set version 3.3.1
does fix the issue, just thought you should know about it
@herzaso I think the error messages are clear enough. This is an expected behavior.
And do yourself a favor and upgrade Node.js to a version that has not reached end of life yet.
@wojtekmaj as I said, the error was clear and we managed to fix, but enterprises move slow. We are in the process of moving to Node 20
Here we go: https://yarnpkg.com/blog/release/4.0 !
@arcanis Amazing! A few questions though, that aren't covered in the article, perhaps useful for people who are new to Yarn (maybe the post can be updated):
- How do you upgrade to 4.0 from 2.x or 3.x? And from 1.x (EDIT: 1.x migration is already covered)? There are no steps listed. What are the migration steps?
-
The post makes things a little confusing, because it says:
To that end we used to recommend using the yarnPath setting pointing to a checked-in binary, but this pattern increased friction more than we liked
When looking at the CLI docs, the only hint we see at updating to 4.0 is the
yarn set version
command. I think to myself, "this must be how we update the version". If we tryyarn set version 4.0
, that seems to work. However, it setsyarnPath
in.yarnrc.yml
, which seems to be opposite of what the 4.0 release post says is ideal. Basically, it is a little confusing because now it seems I've updated to 4.0 in the wrong way.The post then says
Now that Corepack is shipped with both Node 18 and 20 we no longer need to rely on yarnPath, and as a result we updated our installation guide to reflect that
However, when you look at the linked installation guide, it says that updating yarn is done with
yarn set version stable
. But if I want to ensure that I stay on the new 4.0, and not accidentally get 5.0 later and things breaking, this is why I assume I need to runyarn set version 4.0.0
, and this creates ayarnPath
in yarnrc, and adds the binary to.yarn/releases/
, which contradicts the premise of the post.So the main question is, how do we update to 4.0, ensure we stay on 4.0, and not use yarnPath?
Then the post says that plugins are shipped by default. Does that mean I must now run
yarn plugin remove @yarnpkg/plugin-workspace-tools
?What else?
UPDATE: I now tried
yarn set version stable
, and just as withyarn set version 4.0.0
, it creates the supposedly-unwantedyarnPath
config.UPDATE 2: I undid all changes of
yarn set version ____
, and instead I updated thepackageManager
field inpackage.json
manually. Now when I runyarn --version
I see4.0.0
without a binary being installed in.yarn/releases/
. Furthermore, upon runningyarn install
, I see:โค YN0048: Automatically removed core plugins that are now builtins ๐ โค YN0087: Migrated your project to the latest Yarn version ๐
It turns out that Yarn automatically removed the no-longer-needed plugins.
UPDATE 3: There is something a bit odd after running
yarn install
however: theyarn.lock
is modified with a ton of changes, and every single package has itschecksum
hash modified. This is very unexpected.Here's a piece of output that is unexpected during that yarn install:
โค YN0013: โ 2838 packages were added to the project, and 2014 were removed (+ 2.95 MiB).
UPDATE 4: Another discrepancy:
The post says that Zero-Installs is no longer default. However, when I try
yarn init -2
in an empty folder, andyarn add any-package
, I still see pnp files (that happen to be git ignored). This is confusing. When I think of not having Zero-Installs, I think that I will have anode_modules
folder, but that is not the case. If I don't have Zero-Installs, but I also delete.pnp
files, nothing will work.What's the point? Why would I want to have
.pnp
files, and not check them in?? The only reason that I can think of is that I need to havenode_modules
. That's why I setnodeLinker: node-modules
in yarnrc, but I was thinking that it would be the default in new installs (for compatibility with the entire existing JS ecosystem of tooling that largely has no idea what.pnp
files are).The concept of
node_modules
is widely used by both backend and frontend projects, so the.pnp
default seems like a bad default, especially for the growing number of frontend projects that use importmaps and fetch files fromnode_modules
. So by merely gitignoring .pnp files, it seems nothing great was actually achieved, it only makes people need toyarn install
(in which case you may as well be compatible with the entire ecosystem by usingnode_modules
in that mode).
-
- Many warnings are gone (according to the article, haven't tried yet), but I thought they were useful because other package managers did not show them, so I enjoyed having more insight into my install setup. How do we restore it?
- The post mentions official plugins. However, I cannot find a page on the website describing the purpose of plugins (what does it allow people to do?). Where is the list of official plugins and their descriptions? The only pages I can find via the search bar are the
yarn plugin
CLI docs (does not explain the purpose of plugins) and a tutorial page on writing a plugin (but knowing how to make the shell for a plugin is not useful without knowing what we will do in writing a plugin).- EDIT: I found a cache of an old Yarn wesite describing the purpose of plugins, but this seems to be missing from the new website.
It would be great for each release post to also double as a complete migration guide.
@trusktr this is interesting feedback, but since this thread was specifically to talk about the breaking changes in Yarn 4, I don't think we should start discussing the content of the blog post itself (there are 23+ other people subscribed to this thread, and I don't want to spam them; additionally, an issue is likely the wrong format since you have multiple points that would lead to interlaced and very hard to follow discussion).