fetch times out in under 5 seconds

Question

fetch times out in under 5 seconds

crackedpotato007 opened this issue 2 years ago · 29 comments

Bug Description

When trying to fetch a URL a fetch failed error is thrown with the code as
code: 'UND_ERR_CONNECT_TIMEOUT'
, This error is thrown at a request that barely takes 5 - 6 seconds to complete and other HTTP clients like axios and curl perform flawlessly on the same server

Reproducible By

Fetch any discord API url

Expected Behavior

The fetch should complete which is well below the 120s timeout

Logs & Screenshots

/home/arnav/Documents/tej.js/node_modules/undici/lib/fetch/index.js:197
        Object.assign(new TypeError('fetch failed'), { cause: response.error })
                      ^
TypeError: fetch failed
    at Object.processResponse (/home/arnav/Documents/tej.js/node_modules/undici/lib/fetch/index.js:197:23)
    at /home/arnav/Documents/tej.js/node_modules/undici/lib/fetch/index.js:930:38
    at node:internal/process/task_queues:141:7
    at AsyncResource.runInAsyncScope (node:async_hooks:201:9)
    at AsyncResource.runMicrotask (node:internal/process/task_queues:138:8)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at Timeout.onConnectTimeout [as _onTimeout] (/home/arnav/Documents/tej.js/node_modules/undici/lib/core/connect.js:108:24)
      at listOnTimeout (node:internal/timers:561:11)
      at processTimers (node:internal/timers:502:7) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}

Environment

Gentoo, Node v17.6.0

Additional context

Happen on every discord API url and used to work some time back but suddenly just fails everywhere.

Answer 1 · 2022-07-07T06:23:22.000Z

Thanks for reporting!

Can you provide steps to reproduce? We often need a reproducible example, e.g. some code that allows someone else to recreate your problem by just copying and pasting it. If it involves more than a couple of different file, create a new repository on GitHub and add a link to that.

Answer 2 · 2022-07-07T17:20:26.000Z

Hey!
Here you go
a valid auth token can be generated by https://discord.com/developers

 const myuser = await fetch("https://discord.com/api/v10/users/@me", {
      headers: {
        authorization: `Bot xxxxx`,
        "User-Agent": "undici/tej.js",
        encoding: "json",
      },
    }).catch((err) => {
      throw new Error(err);
    });

Answer 3 · 2022-07-07T18:04:14.000Z

Undici fails if your DNS is set to your router which in turn uses google DNS but if i sed the google DNS directly on my machine it would work, this shouldn't be happening i believe.

Answer 4 · 2022-07-07T18:37:16.000Z

Can you create a reproduction without using an external server? Something that we can run locally.

Answer 5 · 2022-07-08T07:44:12.000Z

I can give that a try but I believe that would just run as I believe that undici isn't able to resolve the webpage to an ip address which explains how a direct dns configuration resolved the issue.

Answer 6 · 2022-07-08T07:45:43.000Z

Can you try to replicate it with a external server and the dns set to your router?
For me it was configured as

search bbrouter
nameserver 192.168.1.1

Answer 7 · 2022-07-08T08:23:11.000Z

I have a very similar setup, and there is no problem in contacting the discord API.

Can you try running your script with NODE_DEBUG=net and paste the output? Both when using undici and when using axios.

Answer 8 · 2022-07-08T08:25:05.000Z

Are you sure it's 5 seconds? Our default connect timeout is 10s? I suspect your dns lookup or ssl negotiation is taking more than 10s or something...

Answer 9 · 2022-07-08T10:11:27.000Z

A direct DNS to 8.8.8.8 works flawlessly on undici but a DNS pointing to the router fails. I will get you the timestamp in a few hours

Answer 10 · 2022-07-08T11:09:22.000Z

I don't think this is a undici problem. This works for axios probably because it has a longer (or no) timeout for establishing the connection.

Answer 11 · 2022-07-08T11:18:15.000Z

The timeout should be configurable with:

import { fetch, setGlobalDispatcher, Agent } from 'undici'

setGlobalDispatcher(new Agent({ connect: { timeout: 60_000 } }) )

fetch(...)

Answer 12 · 2022-08-29T07:29:52.000Z

I am also getting UND_ERR_CONNECT_TIMEOUT after ~5 secs when IPv6 is not configured or working properly. With curl, Chrome and other clients, it still works via IPv4. This seems to be because Node does not implement Happy Eyeballs. There is already an issue for this: nodejs/node#41625

Answer 13 · 2022-09-11T05:22:41.000Z

I have a very similar setup, and there is no problem in contacting the discord API.

Can you try running your script with NODE_DEBUG=net and paste the output? Both when using undici and when using axios.
@arnav7633

I'm having the same issue. I'm using discord.js@14.3.0 which uses undici.

NET 56348: pipe false undefined
NET 56348: connect: find host discord.com
NET 56348: connect: dns options { family: undefined, hints: 32 }
NET 56348: _read
NET 56348: _read wait for connection
NET 56348: destroy
NET 56348: close
NET 56348: close handle
ConnectTimeoutError: Connect Timeout Error
    at onConnectTimeout (/home/eduardo/proj/DescicloBot/bot_stable/node_modules/undici/lib/core/connect.js:131:24)
    at /home/eduardo/proj/DescicloBot/bot_stable/node_modules/undici/lib/core/connect.js:78:46
    at Immediate._onImmediate (/home/eduardo/proj/DescicloBot/bot_stable/node_modules/undici/lib/core/connect.js:119:9)
    at processImmediate (node:internal/timers:471:21) {
  code: 'UND_ERR_CONNECT_TIMEOUT'
}
NET 56348: emit close

Is family: undefined correct?

Answer 14 · 2022-10-08T10:31:21.000Z

I encounter the same problem the first time i use it.

Answer 15 · 2022-10-08T11:02:52.000Z

@mcollina maybe we should consider increasing the default timeout?

Answer 16 · 2022-10-08T11:24:10.000Z

That's not the problem. The problem is that all those folks have IPv6 misconfigured and we are missing Happy Eyeballs in Node.js nodejs/node#41625. Hopefully it will land before Node.js v18 goes LTS or shortly thereafter.

Answer 17 · 2022-10-23T08:32:24.000Z

ive gotten into this issues as well, i dont know which lines of code did its so heres all of it
const { Client, Intents, DiscordAPIError, Collection, GatewayIntentBits, REST, Routes } = require('discord.js');
const client = new Client({ intents: [GatewayIntentBits.Guilds] });
require("dotenv") .config();
const CLIENT_ID = 'your client id'
const http = require('http')

const fs = require('node:fs');

const axios = require('axios');

const https = require('https');
const res = require('res');

const data = JSON.stringify({
todo: 'Buy the milk',
});

const options = {
hostname: 'localhost',
port: 443,
path: '/app',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': data.length,
},
};

const error1 = process.env.INVALID_REQUESTED_URL
const title1 = process.env.INVALID_REQUESTED_URL_TITLE

const express = require('express');

const mongoose = require('mongoose');
const { linkSync, writeFile } = require('fs');

const alert = process.env.COTENT_DOWN || "Sorry But the content on this page for this web server is currently down, please try again later"
const al1 = process.env.ERROR || "404 Page Not Found"
const dashboard = process.env.dash || "Sorry But The DashBoard Is Still In Development"

const port = process.env.PORT || 443

const prefix = '/'

// then you may define it like so
client.event = new Collection();
client.commands = new Collection();

const commands = [
{
name: 'ping',
description: 'Replies with Pong!',
},
{
name: 'test',
description: 'Test Running OK',
},
{
name: 'channel',
description: 'Displays the Owners Channel',
},
{
name: 'reload',
description: 'Reload the bot datasets',
},
{
name: 'latest',
description: 'Display the latest video of the owners channel',
},
];

const rest = new REST({ version: '11' }).setToken('your token');

(async () => {
try {
console.log('Started refreshing application (/) commands.');

await rest.put(Routes.applicationCommands(CLIENT_ID), { body: commands });

console.log('Successfully reloaded application (/) commands.');

} catch (error) {
console.error(error);
}
})();

client.on('ready', () => {
console.log(Logged in as ${client.user.tag}!);
});

client.on('interactionCreate', async interaction => {
if (!interaction.isChatInputCommand()) return;

if (interaction.commandName == 'ping') {
await interaction.reply('Pong!');
} else if (interaction.commandName === 'test') {
await interaction.reply('Test Running OK');
} else if (interaction.commandName === 'channel') {
await interaction.reply('Channel not vaild yet');
} else if (interaction.commandName === 'reload') {
await interaction.reply('Reloading Dataset');
} else if (interaction.commandName === 'latest') {
await interaction.reply('Theres No Latest Video In this channel yet');
}
});

  let dataObj = {"id":443, "name":"App", "email":"app@work.org"};
  let data = JSON.stringify(dataObj);
  let index = {"id":443, "name":"index", "text":"Welcome"}
  res.end(data);
} else if (req.url == "/host/app/test") {
  res.setHeader('Content-type',  'application/json');
  res.setHeader('Access-Control-Allow-Or-Deny', "Allow from views");
  res.writeHead(300); //status code HTTP 300 / OK

  let rf2 = alert
  let index = {"id":"443", "name":"test", "text":"test"}
  let d1 = JSON.stringify(index);
  res.end(rf2);
} else if (req.url == "/host/app/test/index/password/1977/code/1") {
  let ht1 = al1
  let rf1 = alert
  res.end(rf1);
} else if (req.url == "/host/app/invalid-request") {
  let erp1 = error1
  let erpt1 = title1
  res.end(erp1);
} else if (req.url == "/host/app/dashboard") {
  let rf2 = alert
  let ht2 = al1
  let dtr1 = dashboard
  res.end(rf2);
}

});
axios.post('http://localhost:443/host/app', {
market: 'Buy the milk',
});
const app = express();

app.use(
express.urlencoded({
extended: true,
}),
);

app.use(express.json());

app.post('localhost:443/host/app', (req, res) => {
console.log(req.body.todo);
});

server.listen(port, () => {
console.log(Server running at port ${port})
});

client.on("message", msg => {
if (msg.content === "/ping") {
msg.reply("pong");
} else if (msg.content == "/test") {
msg.reply("This command is still under development please try again when the bot updated")
} else if (msg.content == "start") {
msg.reply("Unknown Commands, Did you entered correctly, try using /start")
} else if (msg.content == "/start") {
msg.reply("You have started your adventure, on solarishost, you can start mining by doing s!mine")
} else if (msg.content == "/mine") {
msg.reply("This command is still under development, please try again later when the bot has updated")
} else if (msg.content == "/help") {
msg.reply("You can use the command listed here: s!help, s!start, s!test, s!ping, s!mine, s!stop, s!pokemon find")
} else if (msg.content == "help") {
msg.reply("Please use /help to ask for help for this bot instead of using help to ask for help in command list")
} else if (msg.content == "/stop") {
msg.reply("You have stopped your adventure, to get a break")
} else if (msg.content == "/pokemon find") {
msg.reply("Sorry this feature is still in development")
}
});

client.login('Your Token');

Answer 18 · 2022-10-23T08:37:24.000Z

and also here the error

ConnectTimeoutError: Connect Timeout Error
at onConnectTimeout (C:\Users\jedik\IdeaProjects\NextUptime-Bot\node_module
s\undici\lib\core\connect.js:131:24)
at C:\Users\jedik\IdeaProjects\NextUptime-Bot\node_modules\undici\lib\core
connect.js:78:46
at Immediate.onImmediate (C:\Users\jedik\IdeaProjects\NextUptime-Bot\node
modules\undici\lib\core\connect.js:117:33)
at processImmediate (node:internal/timers:466:21) {
code: 'UND_ERR_CONNECT_TIMEOUT'

Answer 19 · 2022-10-26T21:46:21.000Z

i have same problem in undici.request() or undici.fetch().

Connect Timeout Error

But it only happens when i pass dispatcher field to options

import { Agent, request } from "undici";

...

const base64encodedData = Buffer.from(this.#username + ":" + this.#password).toString("base64");
const { body, statusCode } = await request(this.#url, {
  dispatcher: new Agent({
    connect: { rejectUnauthorized: false, timeout: 60_000 },
  }),
  headers: { Authorization: "Basic " + base64encodedData },
  method: "POST",
});

when i dont pass dispatcher to options it works fine without any Connect Timeout Error

import { Agent, request } from "undici";

...

const base64encodedData = Buffer.from(this.#username + ":" + this.#password).toString("base64");
const { body, statusCode } = await request(this.#url, {
  headers: { Authorization: "Basic " + base64encodedData },
  method: "POST",
});

But in my case i needed option rejectUnauthorized: false

update

if i pass globally at top



import { Agent, request } from "undici";
const dispatcher = new Agent({
  connect: { rejectUnauthorized: false, timeout: 60_000 },
})

...

const base64encodedData = Buffer.from(this.#username + ":" + this.#password).toString("base64");
const { body, statusCode } = await request(this.#url, {
  dispatcher,
  headers: { Authorization: "Basic " + base64encodedData },
  method: "POST",
});

app works much better

Answer 20 · 2022-11-17T05:52:40.000Z

Just wanted to make a note for anyone else that experiences this error UND_ERR_CONNECT_TIMEOUT running on Azure App Service, the actual root cause is because App Service enforces a SNAT outbound connection limit which also results in fetch failed if you try to make a lot of outbound requests very quickly.

Inspired by #1531 (comment) I found it is possible to configure the max connections limit (per host) using

import { fetch, setGlobalDispatcher, Agent, Pool } from "undici";

setGlobalDispatcher(
  new Agent({ factory: (origin) => new Pool(origin, { connections: 128 }) })
);

fetch(...)

which alleviates the problem on Azure App Service

Answer 21 · 2022-12-20T02:00:20.000Z

hey, i've run into this issue and I don't understand why. how can i fix it?

Answer 22 · 2023-01-11T14:34:35.000Z

I think I am affected by this issue, too - I can't make any requests to hosts with an AAAA record outside of my network:

> await fetch('https://cloudflare.com');
Uncaught TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11118:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async REPL7:1:33 {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:6625:28)
      at node:internal/deps/undici/undici:6583:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:6614:13)
      at process.processImmediate (node:internal/timers:471:21)
      at process.topLevelDomainCallback (node:domain:161:15)
      at process.callbackTrampoline (node:internal/async_hooks:128:24) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}

It works for services like GitHub, which refuse to implement IPv6 and only have A records:

> await fetch('https://github.com');
Response {
  [Symbol(realm)]: null,
  [Symbol(state)]: Proxy [
    {
      aborted: false,
      rangeRequested: false,
      timingAllowPassed: true,
      requestIncludesCredentials: true,
      type: 'default',
      status: 200,
      timingInfo: [Object],
      cacheState: '',
      statusText: 'OK',
      headersList: [HeadersList],
      urlList: [Array],
      body: [Object]
    },
    { get: [Function: get], set: [Function: set] }
  ],
  [Symbol(headers)]: HeadersList { /* ... */ }
}

Is this expected? Can I do anything for the time being?

Answer 23 · 2023-02-03T17:58:34.000Z

You need to flip the setting in Node.js core:

node --dns-result-order=ipv4first

Actually I think we can ship a new default here and enable autoSelectFamily: true:

https://nodejs.org/docs/latest-v18.x/api/net.html#socketconnectoptions-connectlistener.

and enable happy eyeballs in Node v18.

@ronag wdyt?

Answer 24 · 2023-03-05T15:53:19.000Z

Running into this with the official node:19-alpine docker image running on fly.io. It seems to have something to do with a configuration in the image making it impossible to resolve domains with AAAA IPv6 records. dns-result-order would obviously fix this, but it would also be nice to figure out how to make AAAA records resolve property. I'll post back if I find anything.

Answer 25 · 2023-03-05T17:28:19.000Z

@bcomnes it's not a problem of DNS resolution but rather ipv6 connectivity. contact fly support. They had ipv6 issues in the past and they fixed them, so possibly it's on their side.

Answer 26 · 2023-03-05T22:06:03.000Z

Odd, I just deployed with autoSelectFamily: true: which made connections to domains that resolve to ipv6 work, but on a subsequent deploy it did not work. I will open an issue with them.

EDIT: I believe this was a fly IPv6 connective issue that was resolved.

Answer 27 · 2023-06-28T19:36:20.000Z

This error happened to me while running nodemon without making any fetch request.

My WiFi router emits at the same time a 5G and a 2.4G bands merged/mixed into one wireless network.

Due to coming close to the router, my Mac auto-connected to the 5G and this UND_ERR_CONNECT_TIMEOUT error started showing. I turned off the VPN and it did not, but when I turned it on again, even with a different DNS the error showed up again.

I manually disconnected from the wireless network and reconnected. The error does not show up any longer…

Answer 28 · 2023-08-03T14:02:42.000Z

In my case, switch node from 18 to 20 is solved
(on sveltekit server)

Answer 29 · 2024-03-29T12:20:19.000Z

Here's my workaround (open-source and documented) that I hope that can help you too:

privacy.sexy | force-ipv4: GitHub action to use for GitHub workflows.
privacy.sexy | force-ipv4.sh.: Script for general use.

After days of research and trial/error, this is how I got this working:

Create a script called force-ipv4.sh, that configures system to prefer IPv4 over IPv6, call it to configure the machine. It was not easy to find a reliable cross-platform solution and I went Cloudflare WARP for DNS resolution along with some system configurations.
To easily use the script in GitHub workflows, create GitHub action called force-ipv4 and call it in GitHub runners.
Fixes the IPv6 request issues, and you can happily run e.g. fetch API from Node.

Related commit introducing this fix: undergroundwires/privacy.sexy@52fadcd