huggingface/transformers.js

Module crashes webpack build due to use of new URL('./', import.meta.url)

Opened this issue · 9 comments

System Info

transformer.js#3

Environment/Platform

  • Website/web-app
  • Browser extension
  • Server-side (e.g., Node.js, Deno, Bun)
  • Desktop app (e.g., Electron)
  • Other (e.g., VSCode extension)

Description

The dist folder of transformer.js in node_modules contains the "new URL('./', import.meta.url)", which cause a build crash for webpack:
webpack/webpack#16878

Reproduction

extension.zip

Facing this issue on expo react native

"expo": "~51.0.28",
"react": "18.2.0",
"react-native": "0.74.5",
"@xenova/transformers": "^2.17.2",

SyntaxError: 192689:107:'import.meta' is currently unsupported, js engine: hermes [Component Stack]

facing this issue on next v14

"@huggingface/transformers": "^3.0.0",
"next": "14.2.5",
"react": "^18",
"react-dom": "^18",
 ⨯ ./node_modules/@huggingface/transformers/dist/transformers.js:32776:1
Module not found: Can't resolve './'

Hi there 👋 Thanks for reporting - we're working on fixing this :)

In the meantime, there is a workaround as shown here
(update the webpack config):

webpack(config, { isServer }) {
	config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        .... 
	return config;
	}

I might have found a better solution (tested in Next.js v15):

const nextConfig = {
  output: "standalone",
  // https://nextjs.org/docs/app/api-reference/next-config-js/serverExternalPackages
  serverExternalPackages: ["@huggingface/transformers"],
};

Let me know if that fixes it!


Demo: https://huggingface.co/spaces/webml-community/next-server-template
Source code: https://github.com/huggingface/transformers.js-examples/tree/main/next-server

Hi there 👋 Thanks for reporting - we're working on fixing this :)

In the meantime, there is a workaround as shown here (update the webpack config):

webpack(config, { isServer }) {
	config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        .... 
	return config;
	}

Hi, this config not work for web extension. Although the compilation error is resolved, the web extension cannot access the local path when running.

Now, my temporary solution is to copy dist/transformers.js from node_modules into my project and import it directly.

I just wanted to say thanks for the suggested fixes. I'm working with a web worker and the updates to the next.conf.js file don't seem to work for this.

What I'll do is try to convert this to a route and see but I wanted to mention this scenario isn't fixed with the above solutions.

The current build of Transformers may work but depending on your Webpack set-up, you have to click through one or two warning overlays telling you that “Accessing import.meta directly is unsupported”. This is definitely questionable for newcomers to the project assuming this is a hard error. Keep in mind that renderer-side errors for Webpack are full-screen blocking overlays, not just something flying by in your terminal.

Solved by referring to the configuration below:

/** @type {import('next').NextConfig} */
import path from "path";
import {fileURLToPath} from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const nextConfig = {
    output: 'export',

    webpack: (config) => {
        config.resolve.alias['@huggingface/transformers'] = path.resolve(__dirname, 'node_modules/@huggingface/transformers');
        config.resolve.alias = {
            ...config.resolve.alias,
            "sharp$": false,
            "onnxruntime-node$": false,
        }
        return config;
    },
};

export default nextConfig;