skott installation adds 544 MB to node_modules

[dec0dOS]

Hello! First of all, thanks for this fantastic project! Skott has been incredibly useful for me.

However, I've noticed an issue with the node_modules directory that Skott's installation generates – it's incredibly large. A fresh installation comes in at a whopping 544 MB in dependencies! This seems like quite a hefty price to pay to have Skott as a devDependency for my project.

npm install skott

I've come across some related issues that I'd like to discuss:

1. I believe TypeScript should be placed in devDependencies instead.
2. There are other packages that could potentially be moved to devDependencies as well.
3. The @tabler package takes up a significant amount of space due to its icons, which are likely unused.
4. It might be worth considering bundling the package before publishing it to npm. This could result in substantial improvements for Skott, potentially reducing its size by a factor of 10x to 25x.

I hope we can address these concerns to make Skott even more efficient and developer-friendly

[dec0dOS]

Additionally, I believe it makes sense to consider splitting Skott into two separate packages: Skott-cli and Skott-webapp. Skott-cli could be utilized in CI processes without requiring the installation of the entire web application. This could streamline development workflow and resource usage

[antoine-coulon]

Hello @dec0dOS, thanks a lot for the kind words and for opening that issue!

You're completely right, 544MB is absurd. Most of this comes indeed from a big mistake from me, I forgot to move everything related to the web app in "devDependencies", so there is almost 400MB that we can get rid of right away and very easily as all the dependencies are already inlined in the bundle shipped with skott-webapp 🤦‍♂️

I'm opening a PR right away for that in the first place.

Additionally, I believe it makes sense to consider splitting Skott into two separate packages: Skott-cli and Skott-webapp. Skott-cli could be utilized in CI processes without requiring the installation of the entire web application. This could streamline development workflow and resource usage

Conceptually that's indeed correct, but given that skott-webapp will only be shipped with its bundle which is currently around 1,25MB (unzipped), in my opinion this would not really be worth it regarding the cost of the DX (requiring one package install) for something that I believe most skott users will use.

dist/index.html                          0.71 kB │ gzip:   0.39 kB
dist/assets/index-2f10dd52.css           1.62 kB │ gzip:   0.83 kB
dist/assets/buttons.esm-48f94bc9.js     19.22 kB │ gzip:   6.64 kB
dist/assets/index-b0284b85.js        1,221.83 kB │ gzip: 363.73 kB

There was already a step in that direction I did in #45 to reduce skott's core, but this was for less used display modes no representing most of the use cases.

I believe TypeScript should be placed in devDependencies instead.

Yes indeed TypeScript is huge (50MB), from what I can remember this was required by the TypeScript parser I'm currently using that is https://github.com/typescript-eslint/typescript-eslint, but I might be mistaken so I'll reinvestigate and drop TypeScript from runtime dependencies as well.

With TypeScript + without skott-webapp dependencies we would be around 100MB for skott, without TypeScript this would be even better, around 50MB.

I'll let you know how it goes!

[antoine-coulon]

@dec0dOS I just published a new version of the web app 2.0.1 that should importantly reduce the node_modules size. You can update skott as the new version is in the compatible range of updates or just take the skott@0.31.1.

For "typescript" dependency on skott side, I'll investigate to be check whether we can get rid of it as well.

Also, could you provide me the tool name that you're using to provide a detailed view of node_modules size please? :)

[antoine-coulon]

@dec0dOS, I just tested on my own machine with only skott@0.31.0 installed it seems that skott-webapp@2.0.1 divided by 3,6 the size of the node_modules:

skott-webapp@2.0.0

❯ du -sh node_modules               
520M    node_modules

skott-webapp@2.0.1 or skott@0.31.1

❯ du -sh node_modules    
144M    node_modules

So it nearly reduces the whole tree size by 400MB.

[dec0dOS]

Hello @dec0dOS, thanks a lot for the kind words and for opening that issue!

You're completely right, 544MB is absurd. Most of this comes indeed from a big mistake from me, I forgot to move everything related to the web app in "devDependencies", so there is almost 400MB that we can get rid of right away and very easily as all the dependencies are already inlined in the bundle shipped with skott-webapp 🤦‍♂️

I'm opening a PR right away for that in the first place.

Thank you for the quick response! I truly value your efforts.

Yes indeed TypeScript is huge (50MB), from what I can remember this was required by the TypeScript parser I'm currently using that is https://github.com/typescript-eslint/typescript-eslint, but I might be mistaken so I'll reinvestigate and drop TypeScript from runtime dependencies as well.

I also use typescript-eslint, and the great thing is that it doesn't require TypeScript as a dependency. You can safely move it to devDependencies. In the current published version, it occupies the most space:

I believe it makes sense to also move @typescript-eslint, @babel, and @eslint to devDependencies. You can also consider moving all the other dependencies that aren't directly imported in the Skott to devDependencies. Skott itself can be a handy tool for investigation! 😄

Also, could you provide me the tool name that you're using to provide a detailed view of node_modules size please? :)

Sadly, that is macOS-only application called DaisyDisk.
However, for other platforms, I mainly rely on ncdu.

[antoine-coulon]

I also use typescript-eslint, and the great thing is that it doesn't require TypeScript as a dependency. You can safely move it to devDependencies. In the current published version, it occupies the most space

Just for precision, I'm referring to https://www.npmjs.com/package/@typescript-eslint/typescript-estree which is definitely a required runtime dependency for skott, (this has nothing to do with linting which for sure belongs to the devDeps field).

@typescript-eslint/typescript-estree definitely requires TypeScript at runtime (for the parser API that TypeScript provides) but was not explicitly embedding it in its own dependencies, hence the fact of skott having it listed in prod deps.

But for regarding @typescript-eslint/eslint-plugin and @typescript-eslint/parser (linting related), these one definitely need to get rid of there! I'm not really sure why this is there and how it could have survived there for so much time :D thanks for the heads up, I'm fixing that

[antoine-coulon]

@dec0dOS just published skott@0.31.2 with proper deps declarations and the total size is now 94MB with the second PR.

So from the beginning it basically reduced node_modules size from 520MB to 94MB with low effort, thanks to your heads up, nice!

[dec0dOS]

Thank you, @antoine-coulon!

Adding skott as a development dependency to my backend project only increases the size of the "node_modules" folder by 28 MB as most of dependencies are reused.

[antoine-coulon]

You're welcome and I'm glad to hear that you're using skott 👌