Atome-FE/llama-node

use with Next.js (webpack fails to load binary file)

Opened this issue · 3 comments

eagor commented

I'm trying to use it in Next.js API action, but getting error:

- error ./node_modules/@llama-node/llama-cpp/@llama-node/llama-cpp.darwin-arm64.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/@llama-node/llama-cpp.darwin-arm64.node
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js

Do I need to add some binary loader rule to webpack config, or somehow skip its loading?

rorcde commented

I also have the same error. Were you able to find a solution for it?

rgpass commented

I resolved the original issue by following the node-loader Webpack docs. This resulted in running npm i -D node-loader and changing next.config.ts to be

/** @type {import('next').NextConfig} */
const nextConfig = {
  webpack: (config) => {
    config.module.rules.push({
      test: /\.node$/,
      loader: "node-loader",
    });

    return config;
  },
};

module.exports = nextConfig;

But that only gets you to the next error, which is

Module not found: Can't resolve './@llama-node/llama-cpp.android-arm64.node' in '/Users/gpass/projects/cackle/node_modules/@llama-node/llama-cpp'

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js
./src/app/api/sanitize/run.mjs
./src/app/api/sanitize/route.ts

./node_modules/@llama-node/llama-cpp/index.js
Module not found: Can't resolve '@llama-node/llama-cpp-android-arm64' in '/Users/gpass/projects/cackle/node_modules/@llama-node/llama-cpp'

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js
./src/app/api/sanitize/run.mjs
./src/app/api/sanitize/route.ts

./node_modules/@llama-node/llama-cpp/index.js
Module not found: Can't resolve './@llama-node/llama-cpp.android-arm-eabi.node' in '/Users/gpass/projects/cackle/node_modules/@llama-node/llama-cpp'

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js
./src/app/api/sanitize/run.mjs
./src/app/api/sanitize/route.ts

./node_modules/@llama-node/llama-cpp/index.js
Module not found: Can't resolve '@llama-node/llama-cpp-android-arm-eabi' in '/Users/gpass/projects/cackle/node_modules/@llama-node/llama-cpp'

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js
./src/app/api/sanitize/run.mjs
./src/app/api/sanitize/route.ts

./node_modules/@llama-node/llama-cpp/index.js
Module not found: Can't resolve '@llama-node/llama-cpp-win32-x64-msvc' in '/Users/gpass/projects/cackle/node_modules/@llama-node/llama-cpp'

---REPEATS FOR EACH ARCHITECTURE---
---FOR THE ONE ARCHITECTURE I HAVE (darwin-arm64 on my Apple M2 Pro)---

Import trace for requested module:
./node_modules/@llama-node/llama-cpp/index.js
./node_modules/llama-node/dist/llm/llama-cpp.js
./src/app/api/sanitize/run.mjs
./src/app/api/sanitize/route.ts
- error node_modules/@llama-node/llama-cpp/index.js (141:36) @ webpackMissingModule
- error Error: Cannot find module '@llama-node/llama-cpp-darwin-arm64'
    at webpackMissingModule (webpack-internal:///(rsc)/./node_modules/@llama-node/llama-cpp/index.js:124:110)
    at eval (webpack-internal:///(rsc)/./node_modules/@llama-node/llama-cpp/index.js:124:219)
    at (rsc)/./node_modules/@llama-node/llama-cpp/index.js (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:88:1)
    at __webpack_require__ (/Users/gpass/projects/cackle/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/llama-node/dist/llm/llama-cpp.js:5:79)
    at (rsc)/./node_modules/llama-node/dist/llm/llama-cpp.js (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:809:1)
    at __webpack_require__ (/Users/gpass/projects/cackle/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/api/sanitize/run.mjs:6:90)
    at (rsc)/./src/app/api/sanitize/run.mjs (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:820:1)
    at __webpack_require__ (/Users/gpass/projects/cackle/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/api/sanitize/route.ts:6:66)
    at (rsc)/./src/app/api/sanitize/route.ts (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:693:1)
    at __webpack_require__ (/Users/gpass/projects/cackle/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fsanitize%2Froute&page=%2Fapi%2Fsanitize%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fsanitize%2Froute.ts&appDir=%2FUsers%2Fgpass%2Fprojects%2Fcackle%2Fsrc%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Fgpass%2Fprojects%2Fcackle&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!:16:116)
    at (rsc)/./node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fsanitize%2Froute&page=%2Fapi%2Fsanitize%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fsanitize%2Froute.ts&appDir=%2FUsers%2Fgpass%2Fprojects%2Fcackle%2Fsrc%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Fgpass%2Fprojects%2Fcackle&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D! (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:77:1)
    at __webpack_require__ (/Users/gpass/projects/cackle/.next/server/webpack-runtime.js:33:42)
    at __webpack_exec__ (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:830:39)
    at /Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:831:28
    at Object.<anonymous> (/Users/gpass/projects/cackle/.next/server/app/api/sanitize/route.js:834:3)
    at Module._compile (node:internal/modules/cjs/loader:1255:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1309:10)
    at Module.load (node:internal/modules/cjs/loader:1113:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1137:19)
    at require (node:internal/modules/helpers:121:18)
    at requirePage (/Users/gpass/projects/cackle/node_modules/next/dist/server/require.js:112:75)
    at /Users/gpass/projects/cackle/node_modules/next/dist/server/load-components.js:80:84
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async loadComponentsImpl (/Users/gpass/projects/cackle/node_modules/next/dist/server/load-components.js:80:26)
    at async DevServer.findPageComponentsImpl (/Users/gpass/projects/cackle/node_modules/next/dist/server/next-server.js:434:36) {
  digest: undefined
}
loo-y commented

is there any solution? I have the same issue with Next.js.