No proxy detected/used + Authentication
Opened this issue · 4 comments
const fetch = require("fetch-with-proxy");
=> TypeError: fetch is not a function
using
const fetch = require("fetch-with-proxy").default;
=> Works but doesnt use proxy even when set.
my code:
const fetch = require("fetch-with-proxy").default;
process.env.HTTPS_PROXY = "myproxyprovider:port";
fetch("http://www.mikes-marketing-tools.com/whatismyip.php", {
"authenticate":{
"username": "myusername",
"password": "mypassword"
},
"headers": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-language": "en,en-US;q=0.9,cs;q=0.8,sk-SK;q=0.7,sk;q=0.6",
"cache-control": "no-cache",
"pragma": "no-cache",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"sec-gpc": "1",
"upgrade-insecure-requests": "1"
},
"referrer": "http://www.mikes-marketing-tools.com/whatismyip.php",
"referrerPolicy": "strict-origin-when-cross-origin",
"body": null,
"method": "GET",
"mode": "cors",
"credentials": "include"
}).then(function (response) {
return response.text();
}).then(function (html) {
parser = new DOMParser();
doc = parser.parseFromString(html, 'text/html');
fs.appendFile('./content.html', doc.rawHTML, function(err){
if (err) return console.log(err);
console.log("LOADED");
})
}).catch(function (err) {
console.warn('Something went wrong.', err);
});
I am not sure if the authentication is handled correctly and generally the documentation and readme doesnt tell us much.
This package is just a wrapper that use popular packages like node-fetch, proxy-from-env and tunnel-agent.
So I don't know why proxy is not detected, but in your example, there are a redirection to https and actually, the wrapper does not support http to http redirection. The package add a HTTP agent and as you can see at this line,
https://github.com/touv/node-fetch-with-proxy/blob/master/src/index.js#L53-L55
I attempt to detect automatically the protocol. In this case, node-fetch follows the redirect with the same HTTP agent. But this one is no longer compatible...
This package is just a wrapper that use popular packages like node-fetch, proxy-from-env and tunnel-agent.
So I don't know why proxy is not detected, but in your example, there are a redirection to https and actually, the wrapper does not support http to http redirection. The package add a HTTP agent and as you can see at this line,https://github.com/touv/node-fetch-with-proxy/blob/master/src/index.js#L53-L55
I attempt to detect automatically the protocol. In this case, node-fetch follows the redirect with the same HTTP agent. But this one is no longer compatible...
About the http/https I was trying to use full https and was getting tunnel could not be established error so idk.