Dependencies bumped in Jlab core breaking build

[telamonian]

A bunch of dependencies got bumped in the latest head of Jlab core, and the dependencies of dataregistry-extension haven't caught up. This means that for at least some builds (in my case, a sibling build of jupyterlab-hdf5, which has a dependency on dataregistry-extension) you get multiple resolutions for eg @phosphor/widgets (one for 1.8.0 and one for the minor version bump 1.9.0) and the build breaks:

../jupyterlab-hdf/src/dataregistry.ts:81:3 - error TS2345: Argument of type '({ url }: { data: void; url: URL; type: void; }) => { data: () => HdfDatasetMain; type: string; }' is not assignable to parameter of type '(_: { data: void; url: URL; type: void; }) => { data: () => Widget; type: string; } | { data: () => Widget; type: string; }[]'.
  Type '{ data: () => HdfDatasetMain; type: string; }' is not assignable to type '{ data: () => Widget; type: string; } | { data: () => Widget; type: string; }[]'.
    Type '{ data: () => HdfDatasetMain; type: string; }' is not assignable to type '{ data: () => Widget; type: string; }'.
      Types of property 'data' are incompatible.
        Type '() => HdfDatasetMain' is not assignable to type '() => Widget'.
          Type 'HdfDatasetMain' is not assignable to type 'Widget'.
            Types of property 'title' are incompatible.
              Type 'import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/title").Title<import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/widget").Widget>' is not assignable to type 'import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/title").Title<import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/widget").Widget>'.
                Types of property 'owner' are incompatible.
                  Type 'import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/widget").Widget' is not assignable to type 'import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/widget").Widget'.
                    Types of property 'layout' are incompatible.
                      Type 'import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/layout").Layout' is not assignable to type 'import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/layout").Layout'.
                        Types of property 'removeWidget' are incompatible.
                          Type '(widget: import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/widget").Widget) => void' is not assignable to type '(widget: import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/widget").Widget) => void'.
                            Types of parameters 'widget' and 'widget' are incompatible.
                              Type 'import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/widget").Widget' is not assignable to type 'import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/widget").Widget'.
                                Types of property 'title' are incompatible.
                                  Type 'import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/title").Title<import("/Users/tel/git/jupyterlab/node_modules/@jupyterlab/dataregistry-extension/node_modules/@phosphor/widgets/lib/widget").Widget>' is not assignable to type 'import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/title").Title<import("/Users/tel/git/jupyterlab/node_modules/@phosphor/widgets/lib/widget").Widget>'.
                                    Types have separate declarations of a private property '_label'.

I think you can fix this by matching up the dependencies in package.json to the latest ones in Jlab core master and then doing a release.

[kgryte]

Just to clarify, the build breaks because of multiple versions of @phosphor/widgets? And these versions are distinguished by a minor version bump (e.g., 1.8.x vs 1.9.x)?

[vidartf]

Doesn't the semver ranges overlap?

[telamonian]

@kgryte That's what I thought until I read @vidartf comment and dug into it. It actually looks like for some reason the

"@phosphor/widgets": "^1.8.0",

line from the package.json of @jupyterlab/dataregistry-extension resolves in the yarn.lock to 1.9.1, which conflicts with the existing 1.9.0. Regenerating yarn.lock by running

jlpm run clean:slate && rm yarn.lock && jlpm integrity && jlpm install

does appear to fix the multiple resolutions of @phosphor/widgets. Not 100% sure why that requires deleting and recreating yarn.lock, but whatever.

That doesn't solve everything. Now I get a whole bunch of different build errors including:

../../node_modules/@types/react/index.d.ts:2823:14 - error TS2300: Duplicate identifier 'LibraryManagedAttributes'.

2823         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                  ~~~~~~~~~~~~~~~~~~~~~~~~

  ../../node_modules/@types/react-dom/node_modules/@types/react/index.d.ts:2816:14
    2816         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                      ~~~~~~~~~~~~~~~~~~~~~~~~
    'LibraryManagedAttributes' was also declared here.

[telamonian]

Okay, I think I have my head thoroughly wrapped around this. There are two basic problems:

• The yarn.lock for Jlab core now has a resolution for @phosphor/widgets@^1.9.0 to 1.9.0, but no resolution for @^1.8.0. When I try to build a sibling extension against the current release of @jupyterlab/dataregistry-extension, a resolution is added for @phosphor/widgets@^1.8.0 to 1.9.1, which caused the original error. @vidartf This is why there was still an error even though the semver ranges of ^1.8.0 and ^1.9.0 overlap.

• There's currently a weird version issue in Jlab core that prevents regenerating the yarn.lock. @jupyterlab/apputils has an explicit dependency on both "@types/react-dom": "^16.8.4" and "@types/react": "~16.8.18". "@types/react-dom": "^16.8.4" has itself a dependency on "@types/react": "*". When regenerating the yarn.lock the explicit dependency @types/react@~16.8.18 resolves to 16.8.25, while the sub-dependency @types/react@* will now resolve to 16.9.2 (the latest).

[vidartf]

Note that we call out to yarn-deduplicate to help ensure that overlapping semver ranges gets collapsed to as few versions as possible. It might be that this is not invoked for some command orders. E.g. I'm not sure how the sibling commands work, so these might not be covered?

[saulshanabrook]

One thing to note is that yarn.lock files are only for development, they are not used when installing the package.

https://yarnpkg.com/lang/en/docs/yarn-lock/:

Don’t worry about publishing the yarn.lock file as it won’t have any effect on users of the library.

The whole @types/react thing is a real pain, I have hit that before. I'll try to bump the dependencies on this package and test it with the dataregistry extension and see if I can get them to work nicely.

[saulshanabrook]

@telamonian I tried to reproduce your error by creating a new environment with jupyterlab 1.1.2, but it seemed to work OK:

conda create -n tmp -c conda-forge jupyterlab
conda activate tmp
pip install h5py
pip install jupyterlab_hdf
jupyter labextension install jupyterlab-hdf @jupyterlab/dataregistry-extension

The yarn deduping doesn't really work all the time, despite our command that attempts to do it. So often if I install one extension, and then another, it gives different results than installing both at once. To debug this sort of thing I often cd into the jupyterlab staging dir and trying running yarn run @phosphor/widgets and it should tell you why its installing multiple versions.

Could you give a way to reproduce this? I am hesitant to have to release a new version of this package every time JupyterLab releases if I can help it.

[telamonian]

@saulshanabrook It should reproduce if you just do a sibling install of jupyterlab-hdf:

jlpm run add:sibling https://github.com/jupyterlab/jupyterlab-hdf5.git && jlpm build

If deduping is already automatically happening as part of normal labextension installs, then this bug should probably be filed under "yet another stupid problem caused by sibling installs" and mostly ignored. I'll dig into it further if I have time

[saulshanabrook]

I think we can close this now, the most reccent jupyterlab hdf5 seems to work with this.

[telamonian]

I think it's fine to close. I ran into this issue several more times both with sibling installs and with just building @jupyterlab/hdf5 by itself. I was always able to resolve it by running:

jlpm yarn-deduplicate -s fewer

ie the deduplicate script from the top-level package.json in jlab core.