huggingface/transformers.js

Next.js example breaks with v3

Closed this issue ยท 17 comments

Question

Are there steps documented anywhere for running V3 in your app? I'm trying to test it out via these steps:

  1. Pointing to the alpha in my package.json: "@huggingface/transformers": "^3.0.0-alpha.10",
  2. npm i
  3. cd node_modules/@hugginface/transformers && npm i
  4. copy the webpack.config.js from the repo into the node_modules/@hugginface/transformers dir.
  5. npm run build in node_modules/@hugginface/transformers dir.

I then run my app, and get the following error:

ERROR in ../../node_modules/@huggingface/transformers/dist/transformers.js 42256:34-64
Module not found: Error: Can't resolve './' in '/node_modules/@huggingface/transformers/dist'
webpack compiled with 1 error

Thanks, I'm excited to test out the latest and greatest!

+1 on this error. Also in 3.0.0-alpha.13

Screenshot 2024-08-29 at 4 46 08โ€ฏPM Screenshot 2024-08-29 at 4 42 32โ€ฏPM

Hi there! ๐Ÿ‘‹ What environment are you running in (OS, build tool, browser, node)? ๐Ÿ‘€ I haven't seen an error like this before ๐Ÿค” Can you share your webpack config?

Hi there! ๐Ÿ‘‹ What environment are you running in (OS, build tool, browser, node)? ๐Ÿ‘€ I haven't seen an error like this before ๐Ÿค” Can you share your webpack config?

Sure can!

OS: Sonoma 14.6.1
Build tool: Lerna
Browser: Chrome
Node: v22.3.0

Regarding webpack config, we aren't using webpack. The config I coped into transformers dir was from the main branch.

Webpack Error:

> app@1.0.0 build
> webpack

assets by status 676 KiB [cached] 1 asset
runtime modules 318 bytes 2 modules
orphan modules 1.66 MiB [orphan] 1 module
./static/images/js/worker.js + 1 modules 1.68 MiB [built] [code generated]

ERROR in ./node_modules/@huggingface/transformers/dist/transformers.js 32909:34-64
Module not found: Error: Can't resolve './' in '/app/node_modules/@huggingface/transformers/dist'
resolve './' in '/app/node_modules/@huggingface/transformers/dist'
  Parsed request is a directory
  using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
    using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
      as directory
        existing directory /app/node_modules/@huggingface/transformers/dist
          using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist)
            using path: /app/node_modules/@huggingface/transformers/dist/index
              using description file: /app/node_modules/@huggingface/transformers/package.json (relative path: ./dist/index)
                no extension
                  /app/node_modules/@huggingface/transformers/dist/index doesn't exist
                .js
                  /app/node_modules/@huggingface/transformers/dist/index.js doesn't exist
 @ ./static/images/js/worker.js 6:0-53

webpack 5.93.0 compiled with 1 error in 2902 ms

I ran into this issue when using webpack on worker.js into worker.bundle.js. @xenova/transformers on 2.17.2 worked just fine. I tried upgrading to @huggingface/transformers on 3.0.0-alpha.14.

I'm on:
macOS: 14.3.1 (23D60)
Node: v22.7.0
Browser: Firefox

With the following import statement, it works well

import { pipeline } from '@xenova/transformers';

alongside this webpack.config.js file:

const path = require('path');

module.exports = {
  entry: './static/images/js/worker.js',
  output: {
    filename: 'worker.bundle.js',
    path: path.resolve(__dirname, 'static/images/js'),
  },
  mode: 'production',
  target: 'webworker',
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            presets: ['@babel/preset-env'],
          },
        },
      },
    ],
  },
  resolve: {
    extensions: ['.js'],
  },
};

However, it doesn't work with:

import { pipeline, AutoProcessor, Swin2SRForImageSuperResolution, RawImage } from '@huggingface/transformers';

I ran into the same issue, but Iโ€™ve figured out the solution!
Just add the following to your Webpack configuration file (or Next.js configuration in my case):

webpack(config, { isServer }) {
	config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        .... 
	return config;
	}

It works like a charm now :)

The error happens only with version 3.0.0-alpha.10 or newer. 3.0.0-alpha.9 works fine for me. Was testing with webpack and in this env:

OS: Sonoma 14.6.1
Browser: Chrome
Node: v22.3.0

+1, @sebastien-lempens thanks for the solution!

The error happens only with version 3.0.0-alpha.10 or newer. 3.0.0-alpha.9 works fine for me. Was testing with webpack and in this env:

OS: Sonoma 14.6.1 Browser: Chrome Node: v22.3.0

This was the temporary solve for me. Since we are using Lerna with CRA, the webpack alias didn't fix things.

Not sure what happened between versions alpha 9 and 10, but that's where this issue starts showing up.

Any word on fix this issue in the most current version?

I ran into the same issue, but Iโ€™ve figured out the solution! Just add the following to your Webpack configuration file (or Next.js configuration in my case):

webpack(config, { isServer }) {
	config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        .... 
	return config;
	}

It works like a charm now :)

@jpcoder2 did you try this? Worked for me

Any word on fix this issue in the most current version?

Still the same as of ^3.0.0-alpha.19 but the @sebastien-lempens 's solution works for me.

Are there any updates regarding this?

@laurislopata We're working on fixing this, as well as creating a Next.js example project which should serve as a basis for others to build on.

Which version of Next.js are you using? Next.js v15 just came out, and from initial testing, may not need the workaround shown above.

I might have found a better solution (tested in Next.js v15):

const nextConfig = {
  output: "standalone",
  // https://nextjs.org/docs/app/api-reference/next-config-js/serverExternalPackages
  serverExternalPackages: ["@huggingface/transformers"],
};

Let me know if that fixes it!


Demo: https://huggingface.co/spaces/webml-community/next-server-template
Source code: https://github.com/huggingface/transformers.js-examples/tree/main/next-server

I might have found a better solution (tested in Next.js v15):

const nextConfig = {
  output: "standalone",
  // https://nextjs.org/docs/app/api-reference/next-config-js/serverExternalPackages
  serverExternalPackages: ["@huggingface/transformers"],
};

Let me know if that fixes it!

Demo: https://huggingface.co/spaces/webml-community/next-server-template Source code: https://github.com/huggingface/transformers.js-examples/tree/main/next-server

Is there an equivalent solve for a Lerna/CRA setup? I can't add the webpack fix above, as CRA won't recognize it unless we eject CRA, which would be a large undertaking. I'd like to try out the official 3.0.0 version, but I'm still getting the original error posted above.

ERROR in ../../node_modules/@huggingface/transformers/dist/transformers.js 42568:34-64
Module not found: Error: Can't resolve './' in '/Users/rstinogle/code/viewer-frontend/node_modules/@huggingface/transformers/dist'
webpack compiled with 1 error and 1 warning

I ended up ejecting create-react-app to fix this issue. For anyone that goes that route, ejecting puts all config, including webpack, into a config dir. I fixed the issue by adding this alias:

"@huggingface/transformers": path.resolve(
  __dirname.replace("/config", ""),
  "node_modules/@huggingface/transformers"
),

I'm going to close this out, as my original issue is resolved.

This issue is till there.