Beta 11 can’t build on Netlify
johnridesabike opened this issue · 24 comments
I upgraded to Beta 11 (and upgraded reason-react to @rescript/react) and the build now fails when deploying to Netlify.
A variation of this error appears for every file in my project:
bsb: [5/11] assets/Contact_Form.iast
1:32:55 PM: FAILED: assets/Contact_Form.iast
1:32:55 PM: /opt/build/repo/node_modules/re-formality/ppx: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /opt/build/repo/node_modules/re-formality/ppx)
We've found a bug for you!
/opt/build/repo/assets/Contact_Form.resi
Error while running external preprocessor
Command line: /opt/build/repo/node_modules/re-formality/ppx '/tmp/ppxd38401Contact_Form.resi' '/tmp/ppxd2563aContact_Form.resi’
Reverting to Beta 10 fixes the errors.
I can’t reproduce this on my local MacOS machine. I also tried some basic debugging (clearing Netlify cache, removing node_modules) but it didn’t make a difference.
I suspect it’s either changes in GH Actions env, or esy, or both. I’m not sure I will be able to look into this until the next month, unfortunately.
I'm also having this issue in GH Actions and in Vercel.com
There's a solution on Discord but I will be able to poke on it only next week. If anyone willing to submit a PR earlier, it will be totally welcomed :)
I made some progress in #99 but it still fails with odd error. I'll comeback to it some time this week.
@johnridesabike @vasco3 Would you mind trying out 4.0.0-beta.12?
It still seems broken, although the error is different now
7:59:25 AM: bsb: [2/11] assets/Contact_Form.iast
7:59:25 AM: FAILED: assets/Contact_Form.iast
7:59:25 AM: sh: 1: /opt/build/repo/node_modules/re-formality/ppx: not found
7:59:25 AM: We've found a bug for you!
/opt/build/repo/assets/Contact_Form.resi
7:59:25 AM: Error while running external preprocessor
7:59:25 AM: Command line: /opt/build/repo/node_modules/re-formality/ppx '/tmp/ppxc94e87Contact_Form.resi' '/tmp/ppx4888aeContact_Form.resi’
Deploying to Netlify. That error appears for every file in my project. It still builds fine on my local machine.
@johnridesabike This is really weird. Do you know what linux setup they use? I'd spin up local Docker container to reproduce it locally.
Also, can you make sure that post-install scripts are not disabled on Netlify?
It looks like their build image is here: https://github.com/netlify/build-image
I’m using Ubuntu Xenial 16.04 (default).
AFAIK post-install scripts should work fine, since I believe some other packages I use depend on those. I’ll double-check though.
Thanks! I will try it out tomorrow 👍
Just so you know this happens on Circle Ci as well using this Docker image: https://hub.docker.com/r/circleci/node
The error is the same as above: sh: 1: /home/circleci/project/node_modules/re-formality/ppx: not found. What's really weird is that the file actually exists 😕
Here is the output of
ls -lah /home/circleci/project/node_modules/re-formality
/home/circleci/project/node_modules/re-formality/ppx
total 43M
....
-rwxr-xr-x 1 circleci circleci 22M Mar 10 08:19 ppx
.....
/bin/bash: line 1: /home/circleci/project/node_modules/re-formality/ppx: No such file or directory
So the file exists but when called sh complains it can't be found.
I did some googling and apparently it means the binary still misses a dependency. I was able to reproduce it locally in a docker container but have no clue what's wrong with the way linux binary is built. I'm not very familiar with the Reason's native side of things and Dune though, so I might miss something obvious.
Just weighing in as I had the same problem locally in WSL2
$ ldd ./node_modules/re-formality/ppx
linux-vdso.so.1 (0x00007ffcb4fdf000)
libc.musl-x86_64.so.1 => not found
I was missing musl libc, so I tried installing with apt which still didn't work as it was an older version (1.1.19-1). It spewed errors about secure_getenv: symbol not found.
I then tried compiling http://musl.libc.org/ from source at 1.2.2 and made sure it was symlinked with sudo ln -s /usr/local/musl/lib/libc.so /lib/libc.musl-x86_64.so.1 and then the ppx executable worked!
$ ldd ./node_modules/re-formality/ppx
linux-vdso.so.1 (0x00007ffd5e396000)
libc.musl-x86_64.so.1 => /lib/libc.musl-x86_64.so.1 (0x00007fb0b4a38000)
I suspect the same issues happens in netlify, where perhaps only gnu libc is present, so it has trouble linking with musl libc :)
@thehabbos007 Thanks for the info! Quick question: is there anything I can do on the build side to make it work w/o a need in user linking?
@thehabbos007 Thanks for the info! Quick question: is there anything I can do on the build side to make it work w/o a need in user linking?
I can imagine statically building the ppx executable might be possible to make it work for all on linux?
Only problem would be increased binary size.
Changing the compiler to use glibc somehow for the ppx would also be a possibility, but it might break other usages (for alpine docker containers perhaps?)
I am not that familiar with esy/dune and the compiler, though so I can't give you any anecdotal fixes im afraid :)
Gotcha, thanks for the input! Yeah, I tried to build it statically, still couldn't make it work. But, as I mentioned, I'm too not very familiar with the Reason's native side of things and Dune, so I might miss something obvious.
I did find this, though, teamwalnut/graphql-ppx@v0.4.6...v0.4.9 which might be helpful!
@thehabbos007 Thanks for the link! I actually looked into this and did a similar custom dockerfile but it didn't work unfortunately.
Hey folks, I think I finally fixed the linux build. Please try 4.0.0-beta.15.
@alexfedoseev Can confirm it is working for me. Thank you so much 🙏
I just upgraded a Netlify project to use beta 15 and it works!
It was a stupid mistake that wasn't related to a static linking itself. Worth me a couple of days to solve. On CI, build step used static linking command was followed by the esy x ... test step, which overwrites the binary built with static linking. So this non-static binary was shipped.
Rescript Build Failing in Docker
Hey Folks,
Building my project with rescript and re-formality for the first time in docker and the ppx is failing. This works fine in my local. Using re-formality version -> ^4.0.0-beta.16
Has anyone faced this issue before Or let me know, I am missing something? I have limited experience with Docker.
Cheers
Prateek
The Error Image
corresponding Docker file
FROM node:14.17.0-alpine
RUN apk add --no-cache git python3 g++ make musl-dev
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub
RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.30-r0/glibc-2.30-r0.apk
RUN apk add glibc-2.30-r0.apk
ENV LD_LIBRARY_PATH=/usr/lib:/lib64:/lib
ARG ENV
ADD package.json .
ADD yarn.lock .
ARG GITHUB_READ_TOKEN
RUN echo -e "machine github.com\nlogin Prateek13727\npassword $GITHUB_READ_TOKEN" > ~/.netrc
RUN yarn install --ignore-engines --frozen-lockfile
RUN rm ~/.netrc
ADD . ./
RUN yarn bs:build
RUN yarn build:$ENV
EXPOSE 3003
COPY deploy/scripts/* /usr/bin/
ENTRYPOINT ["bash", "/usr/bin/run.sh"]
corresponding-package.json
{
"name": "pfe-frontend",
"version": "1.0.0",
"description": "frontend for porter-for-business",
"main": "index.js",
"repository": "git@github.com:porterin/pfe-frontend.git",
"scripts": {
"bs:build": "rescript build -with-deps",
"bs:watch": "rescript build -with-deps -w",
"bs:clean": "rescript clean -with-deps",
"build:dev": "webpack --config webpack.development.js",
"build:prod": "webpack --config webpack.production.js && rm -rf public/build/**map && rm -rf public/build/**map*",
"build:staging": "webpack --config webpack.staging.js",
"dev": "webpack-dev-server --config webpack.development.js --hot",
"install:frozen": "yarn install --ignore-engines --frozen-lockfile",
"upgradeI": "yarn upgrade-interactive --latest",
"start:server": "node server.js"
},
"keywords": [
"BuckleScript",
"ReasonReact",
"reason-react"
],
"author": "",
"license": "MIT",
"dependencies": {
"@glennsl/bs-json": "^5.0.4",
"@rescript/react": "^0.10.3",
"@rescript/std": "^9.1.3",
"@sentry/webpack-plugin": "^1.17.1",
"bs-dynamic-import": "^0.4.0",
"bs-webapi": "^0.17.1",
"catalyst": "https://github.com/porterin/catalyst-rescript#catalyst-remove-bs-prateek",
"chunks-2-json-webpack-plugin": "^1.0.4",
"clean-webpack-plugin": "^4.0.0-alpha.0",
"compression-webpack-plugin": "^8.0.1",
"cors": "^2.8.4",
"css-loader": "^3.5.3",
"decco": "^1.1.1",
"dotenv-webpack": "^7.0.3",
"express": "^4.16.3",
"express-static-gzip": "^2.0.6",
"file-loader": "^6.0.0",
"file-saver": "^2.0.5",
"html-webpack-plugin": "^5.3.2",
"materialui-daterange-picker": "^1.1.92",
"moment": "^2.25.3",
"moment-locales-webpack-plugin": "^1.2.0",
"node-sass": "^4.9.3",
"re-formality": "^4.0.0-beta.16",
"react": "^16.8.1",
"react-app-polyfill": "^1.0.1",
"react-datepicker": "^2.9.6",
"react-detect-offline": "^2.4.0",
"react-dom": "^16.8.1",
"rescript": "^9.1.4",
"sass-loader": "^8.0.2",
"source-map-loader": "^0.2.4",
"style-loader": "^1.2.1",
"url-loader": "^4.1.0",
"webpack": "^5.51.1",
"webpack-cli": "^4.8.0",
"webpack-merge": "^5.8.0",
"xlsx": "^0.17.3"
},
"devDependencies": {
"@babel/core": "^7.15.5",
"@babel/preset-env": "^7.15.6",
"@babel/preset-react": "^7.14.5",
"babel-loader": "^8.2.2",
"webpack-dev-server": "^4.0.0"
}
}
This is fixed. The issue was, I was using RUN ADD . ./ command in Dockerfile after RUN yarn install --ignore-engines --frozen-lockfile . Hence the /node_modules from my local were overriding the node_modules in my docker container.
Also, creating a .dockerignore will help