Native Node.js addon

[sussyGaymer]

I was wondering if it was possible for you to integrate libcurl-impersonate into a native Node.js addon so it works in JS out-of-the-box, without the long and tedious process of building node-libcurl with the impersonate patch.
What I'm suggesting is a fork of node-libcurl because I feel like that would take a lot less work than creating a new thing from scratch. While I do understand there are a couple of forks which do that, they are all unmaintained and largely out of date. This repository seems somewhat alive so I hope it won't be an issue keeping it alive. (An occasional pull from upstream should be all of the maintenance it needs.)
I'm not very familiar with the development ecosystem of c++ and such so I'm afraid I won't be of much help, but logically it shouldn't be that big of a deal, right? Surely we can just replace which curl it uses in it's deps. (I understand it may not be this simple, but I really need a reliable impersonate library in Node. Thanks in advance!)

[bjia56]

fwiw, this fork already exists, though hasn't been updated recently: https://github.com/SwapnilSoni1999/node-libcurl-impersonate

[sussyGaymer]

While I do understand there are a couple of forks which do that, they are all unmaintained and largely out of date.

Yes, I acknowledged the other fork(s) but as I mentioned, it hasn't been updated for 2 years. Not sure how to feel about that but I'm definitely not using something like that.

The sole reason I proposed this was to get a regularly maintained lib that works and is up to date. If nobody is willing to do something like this, I'll probably go and bother the original author of the fork you linked above since it seems like the most promising thing.

[yifeikong]

Look at this one, it's quite new, updated 4 month ago. https://github.com/ycq0125/node-libcurl

It seems to be based on curl-7.84.0, which should be curl-impersonate 0.5.4.

To patch node-libcurl is not a big problem, but to maintain an updated fork is another thing. We will have to update as node-libcurl updates, I don't think we have the bandwidth to do that. My suggestion is that you can create a script for automating the the "replace libcurl` process, which should always work no matter how node-libcurl updates.

[sussyGaymer]

I stumbled upon that package as well during my research, but it looked sketchy and unmaintained. I still decided to give it a try and got some build errors when installing it that I'm not willing to look into further since the only error I see is MSBuild.exe failed with exit code: 1 which is extremely frustrating.

I'll try to replace the libcurl package of node-libcurl or even go as far as forking the package and see where that gets me. Hoping for the best.

Can you provide me with the additional options that this fork adds which the original libcurl doesn't have?

[yifeikong]

@sussyGaymer lwthiker#80 (comment)

[sussyGaymer]

Tried that a couple days ago but it didn't work correctly if I remember correctly. I'll try some other things in the meantime. I hope you understand me making this issue was a last resort after nothing else worked.

[yifeikong]

Which platform are you on? Windows?

[sussyGaymer]

Windows 10. Specifically 10.0.19045.3693.

[yifeikong]

I guess it's a little more diffcult for you to use the environment variable trick. But here are a few general tips:

1. You can download the window dll, from this fork's release page.
2. Unlike on Linux, you don't need to patchelf or whatever.
3. Install node-libcurl, find which libcurl.dll it's using, replace it with the dll above.
4. Setup envionment variable the Windows way, depending on your shell or app.

[bonald]

I just do this :)

apt install build-essential pkg-config cmake ninja-build curl autoconf automake libtool golang-go unzip python-is-python3 2to3 libcurl4-openssl-dev build-essential
git clone https://github.com/yifeikong/curl-impersonate.git
cd curl-impersonate
mkdir build && cd build
../configure
make chrome-build
make chrome-install
ldconfig
rm /usr/local/lib/libcurl.so
ln -s /usr/local/lib/libcurl-impersonate-chrome.so.4.8.0 /usr/local/lib/libcurl.so
rm /usr/bin/curl-config
ln -s /usr/local/bin/curl-impersonate-chrome-config /usr/bin/curl-config
echo "/usr/local/lib/libcurl-impersonate-chrome.so" >> /etc/ld.so.preload
ldconfig

cd your-app
rm package-lock*
npm install node-libcurl --build-from-source --curl_libraries='-Wl,-rpath /usr/local/lib -lcurl'
pm2 kill

const { Curl, curly } = require("node-libcurl");
process.env.CURL_IMPERSONATE_HEADERS = "yes";
Curl.defaultUserAgent = null;
curly.defaultUserAgent = null;
const sessionCurl = curly.create();

thanks for keeping this lib up to date !

[AlexRevin]

A lot easier way with pre-combiled binaries

Mind the architecture of your docker environment and latest lib version in the url

WORKDIR /tmp/curl-lib
RUN wget --no-check-certificate https://github.com/yifeikong/curl-impersonate/releases/download/v0.7.0/libcurl-impersonate-v0.7.0.x86_64-linux-gnu.tar.gz \
  && tar zxvf libcurl-impersonate-v0.7.0.x86_64-linux-gnu.tar.gz \
  && cp libcurl-impersonate-chrome.so* /usr/local/lib \
  && echo "/usr/local/lib/libcurl-impersonate-chrome.so" >> /etc/ld.so.conf.d/libcurl-impersonate \
  && ldconfig /etc/ld.so.conf.d

WORKDIR /app
COPY package.json package-lock.json test.js ./
RUN npm rm --save node-libcurl
RUN npm install --save node-libcurl --no-cache --force --build-from-source --curl_libraries='-Wl,-rpath /usr/local/lib -lcurl-impersonate-chrome'

With test.js as

import { Curl, curly } from "node-libcurl";
process.env.CURL_IMPERSONATE_HEADERS = "yes";
Curl.defaultUserAgent = null;
curly.defaultUserAgent = null;
console.log(Curl.getVersion());

( async () => {
	const r= await curly.get('http://httpbin.io/headers');
	console.log(r.data);
})()

you should get smth like

node test.js
libcurl/8.7.0-DEV BoringSSL zlib/1.3 brotli/1.1.0 zstd/1.5.6 nghttp2/1.61.0
{
  headers: {
    Accept: [
      'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'
    ],
....