Consider publishing to npm
kimroen opened this issue · 18 comments
It would be very helpful if you could publish this library to npm so that we can install it as a dependency instead of having to download it and include the raw files in our project.
If nothing else, you could include a package.json file in the project, which would allow people to install directly from GitHub.
Thanks for the consideration!
Would a package.json like this work for you?
{
"name": "superpowered",
"description": "Superpowered interactive audio features in JavaScript + WebAssembly.",
"version": "2.0.7",
"browser": "superpowered.js",
"scripts": {
"test": "echo "No test specified." && exit 1"
},
"repository": {
"type": "git",
"url": "https://github.com/superpoweredSDK/web-audio-javascript-webassembly-SDK-interactive-audio.git"
},
"keywords": ["superpowered", "audio"],
"author": "Splice",
"license": "SEE LICENSE IN /license",
"bugs": {
"url": "https://github.com/superpoweredSDK/web-audio-javascript-webassembly-SDK-interactive-audio/issues",
"email": "support@superpowered.zendesk.com"
},
"homepage": "https://superpowered.com/js-wasm-overview"
}
If publishing to npm is out of the question, then yes I do believe that would help 👍 Publishing would be better though, because direct download from Github would require downloading the code on every install instead of having the option to cache it.
Since this is a component to be used in a browser, how often would that install happen and how painful is to download 23 MB?
That is true, but it will be downloaded and built on a build server. The browser will download super powered as part of that built bundle, so this issue isn't related to the end user.
Downloading and building happens around 20 times a day or more at the moment.
Not a big deal, it's just a little unusual for a JavaScript package not to be published to npm 😊
What additional "build" does this require? Why does it have to be included in a larger package, while can be imported with other "traditional" in-browser methods?
I'm not sure it's required to go into detail on that here, but do let me know if that would be useful, and I'd be more than happy to.
In short: We want control over which pages has the library included and which hasn't. Because we bundle our code with a tool like webpack, we can automatically do that based on what pages are using the superpowered package directly or through other code.
Because we do client-side navigation between pages, just including a link in the main html file would download the library on all pages (even the marketing pages that don't use it), and we don't want that.
It's possible we can set it up such that importing the superpowered module would add a link tag in the head or something like that, but we don't have that at the moment.
Hope that clears things up!
So dynamic import is not available in your environment?
Sorry, I don't understand what that means in this context.
You want to control which pages has the library included and which hasn't. I guess the JavaScript code on the pages having Superpowered is different than the pages without. On the pages having Superpowered you don't have dynamic import, so even if the page is loaded dynamically via JavaScript, you can't import JavaScript from a url? Why should Superpowered be packaged directly into some larger JS bundle?
All right, I think publishing the package on npm has value beyond just this issue of bundling (for instance you can use all kinds of tooling like dependabot to be notified when there's a new version), but let's talk specifics around our bundling situation so we can figure out what the best solution would be:
The web frontend for our application is built in React with a library/framework called Next.js. It is deployed by and hosted with Vercel.
Next.js is pretty cool in a lot of ways, but specifically for this situation it's cool in that it allows you to create a file for every "page" in your application, and Nextjs (through webpack) will figure out what code is used on that page and create a javascript bundle for each page. If something is used on multiple pages, it will be bundled together in a single file because the "cost" of separating them is higher than combining them.
Both of these features are showcased right away on the marketing page for Next.js I linked to above, if you'd like to learn more (under the tabs "File-System Routing" and "Automatic Code Splitting").
In addition, each page can have different data needs, and will automatically be compiled to either a static HTML file that is served by a CDN, a static HTML file accompanied by a cloud function that the site automatically fetches data from, or a cloud function that generates the HTML and serves it for every request (server rendering)
All of these scenarios and combinations will result in different needs for the bundle. On top of this, both the server rendering and the build steps happen in Node, so it needs to create separate bundles for handling the Node and browser environments.
For Superpowered, we don't want the code to run when we're in Node (server-rendering), because it's using WebAudio and other things that aren't available in that environment. To do that, we are in fact dynamically loading it. You can see that in our code base here:
Webpack will see this dynamic import statement and know that this file needs that other file.
Why should Superpowered be packaged directly into some larger JS bundle?
I can configure webpack to package Superpowered independently from the other javascript, but I don't see how that makes a difference for whether Superpowered should be published to npm.
To get Superpowered to play well with our setup, we had to download the files and include them in our repository: https://github.com/VoiseyLtd/website/tree/master/libraries/superpowered
If it was on npm (or on github as a package so we could download it with npm), it would be importable with import SuperpoweredModule from 'superpowered' (or something along those lines).
As an aside, I haven't been able to figure out how to get the web worker to work, so the page freezes and becomes unresponsive when Superpowered is analyzing the audio. That's probably just my own fault though, I'll probably figure it out eventually.
Hopefully that gives some more information that would be useful. I would be very glad to find out that I'm just not doing it right and I can get it working easily, so any help would be much appreciated.
Thanks for considering my input!
After seeing this issue, someone sent me an email asking how I managed to load Superpowered in Next.js.
Although not completely related to this issue, we have since made our repo private and the links in my previous comment no longer work, so I thought it could be helpful to outline here both to make the points in my previous comment clearer, and to help other people who run into this.
Here's what we did to make Superpowered load in Next.js (and what I answered this person on email):
There were a few aspects to it. I'm not sure if I have the best setup, but here's what I recall doing to make it load, at least.
To get rid of the fs-issues, add this to next.config.js:
module.exports = {
webpack(config, { isServer, dev }) {
// Fixes npm packages that depend on `fs` module
if (!isServer) {
config.node = {
fs: 'empty',
};
}
return config;
},
};This makes it so we pretend that the fs module doesn't exist when building (in node) for browser environments.
Then, we need to make sure we don't try to server-render anything that imports Superpowered. The way we did this was by lazy-loading the component using the next/dynamic feature of next:
import dynamic from 'next/dynamic';
// Dynamic component that loads and uses Superpowered (importantly, the imported component must be
// in a different file, and not imported statically anywhere
const ComponentThatUsesSuperpowered = dynamic(
() => import('path/to/component'),
// No server side rendering!
{ ssr: false }
);
const MyComponent = () => {
return (
<ComponentThatUsesSuperpowered />
);
}I believe that's what we did, but it's possible I forgot something or there's something else we're doing that actually makes it work.
If this doesn't work for you, then perhaps try to see if you can make it so that MyComponent in this example only conditionally renders ComponentThatUsesSuperpowered - in our actual code we load some things before rendering to the screen.
Hopefully that helps someone!
Thank you @kimroen for the code snippet you put here, it helps a lot 😃 . I used the first trick about the fs-issue, but I didn't need the dynamic trick.
I rebuild the stretching example using the next.js framework on this repo:
https://github.com/frozar/superporewered-next
I close this issue, because we're unable to support (test) every JS context. We're focusing on major web browsers.
That's fair enough, but it still feels very weird for a javascript package not to be published to npm, independently of the build discussions in this thread.
Thanks for considering though!
We don't consider this as the typical Javascript package, especially as it involves heavy WebAssembly inside and audio-processing-specific thinking, outside of Web Audio. It just doesn't work well in Node for example.
We don't consider this as the typical Javascript package, especially as it involves heavy WebAssembly inside and audio-processing-specific thinking, outside of Web Audio. It just doesn't work well in Node for example.
Gotcha! Just in case you weren't aware, npm is used for most frontend projects as well, not just Node. Publishing a package does not imply that it will or can be used in Node.
Well, I'm surprised I must say because releasing a npm package could only be good for you:
- your software is working (at least on my computer),
- you would reach the vast majority of front end developers,
- it's free advertising,
- it's a step toward "building a user community" direction,
- you would get feedback easier,
- and so on...
If you're not sure about the quality of your package in the npm ecosystem, just take it easy and watch this video:
https://youtu.be/PI5wz2pwXIg
I don't want to bother you, but I hope you would reconsider publishing packages on npm, for me, but especially for you!
I agree that an npm package would be great. We are also using webpack and all our other JS code is installed via npm and then bundled with webpack. We are not using Node.js at all.
Not having to download and vendor the superpowered source code into our own repo is just a much nicer and much more typical workflow for JS packages, whether or not it includes a WebAssembly component. It would be easier for us to upgrade and manage alongside all our other dependencies, and keep our own git repo size down.
Attempting to search for and install via npm is the very first thing I tried to do on discovering Superpowered, bypassing the "download" button on your site, because that is just how it's typically done for JS/web.