-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
Error: read ECONNRESET at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) Emitted 'error' event on ClientRequest instance... #42154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for the report! What is your code exactly using? If more data, such as a minimal reproduction case is possible, that will be really helpful. |
Below code is use for the Accept the request for customer using HTTP and do process the request, fetch the date from DB and response back. CODE
|
Same error here, while trying to retrieve Kubernetes pods logs as stream. |
I am getting the same error trying to run
It's quite random, sometimes I have to run Tried 16.14.0 as well, same result. |
Connection reset is a normal consequence. It can't be avoided. It means the client left before it last sent packet can be acknowledged (how TCP works). I see them with iPhone clients with short url codes that simply send a The problem I'm having is catching all instances of it. I'm getting application crashes with the same console.warn('app starting:', new Date());
// Snip setup phase
// Error Handlers
this.http2Server.addListener('error', (err) => {
console.error('HTTP/2 server error', err);
});
this.http2Server.addListener('session', (session) => {
session.on('error', (err) => {
if (err?.code === 'ECONNRESET') {
console.warn('HTTP/2 session connection reset.');
} else {
console.error('HTTP/2 session error', err);
}
});
});
this.http2Server.addListener('stream', (stream, headers) => {
stream.on('error', (err) => {
if (err?.code === 'ECONNRESET') {
console.warn('HTTP/2 stream connection reset.', headers[':path']);
} else {
console.error('HTTP/2 stream error', headers, err);
}
});
});
// Logic handlers
this.http2Server.addListener('stream', this.httpHandler.handleHttp2Stream);
this.http2Server.addListener('request', (req, res) => {
if (req.httpVersionMajor >= 2) return;
this.httpHandler.handleHttp1Request(req, res);
});
// Catch uncaughtException
process.on('uncaughtException', (error, origin) => {
console.error('UNCAUGHT EXCEPTION');
console.error(error);
console.error(origin);
process.exit(1);
}); Log output: L01 | app starting: 2022-03-19T17:02:47.326Z
L02 | HTTP/2 stream connection reset. /api/events?channels=dispatch
L03 | HTTP/2 stream connection reset. /api/getUserInfo
L04 | HTTP/2 stream connection reset. /api/getCompanyInfo
L05 | HTTP/2 session connection reset.
L06 | Error: read ECONNRESET
L07 | at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
L08 | errno: -104,
L09 | code: 'ECONNRESET',
L10 | syscall: 'read'
L11 | }
L12 | UNCAUGHT EXCEPTION
L13 | Error: read ECONNRESET
L14 | at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
L15 | errno: -104,
L16 | code: 'ECONNRESET',
L17 | syscall: 'read'
L18 | }
L19 | uncaughtException
L20 | app starting: 2022-03-19T17:03:10.409Z I've tried I'm on Edit: Same result with |
I made more progress with Handlers: this.httpsServer.removeListener('error', reject);
this.httpsServer.addListener('error', (err) => {
console.error('HTTPS server error', err);
});
this.httpsServer.addListener('clientError', (err) => {
console.error('HTTPS client error', err);
});
this.httpsServer.addListener('tlsClientError', (err) => {
console.error('HTTPS TLS client error', err);
});
this.httpsServer.addListener('connection', (socket) => {
socket.on('error', (err) => {
console.error('HTTPS connection error', err);
});
socket.on('clientError', (err) => {
console.error('HTTPS connection client error', err);
});
socket.on('tlsClientError', (err) => {
console.error('HTTPS connection TLS client error', err);
});
}); Output: HTTP/2 session connection reset.
HTTP/2 client error Error: socket hang up
at connResetException (node:internal/errors:691:14)
at TLSSocket.onSocketClose (node:_tls_wrap:1080:23)
at TLSSocket.emit (node:events:532:35)
at TLSSocket.emit (node:domain:475:12)
at node:net:687:12
at Socket.done (node:_tls_wrap:580:7)
at Object.onceWrapper (node:events:640:26)
at Socket.emit (node:events:520:28)
at Socket.emit (node:domain:475:12)
at TCP.<anonymous> (node:net:687:12) {
code: 'ECONNRESET'
}
HTTP/2 TLS client error Error: socket hang up
at connResetException (node:internal/errors:691:14)
at TLSSocket.onSocketClose (node:_tls_wrap:1080:23)
at TLSSocket.emit (node:events:532:35)
at TLSSocket.emit (node:domain:475:12)
at node:net:687:12
at Socket.done (node:_tls_wrap:580:7)
at Object.onceWrapper (node:events:640:26)
at Socket.emit (node:events:520:28)
at Socket.emit (node:domain:475:12)
at TCP.<anonymous> (node:net:687:12) {
code: 'ECONNRESET'
}
HTTP/2 stream connection reset. /api/events?channels=dispatch
HTTP/2 stream connection reset. /api/getUserInfo
HTTP/2 stream connection reset. /api/getCompanyInfo
HTTP/2 session connection reset.
Error: read ECONNRESET
at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
errno: -104,
code: 'ECONNRESET',
syscall: 'read'
}
UNCAUGHT EXCEPTION
Error: read ECONNRESET
at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20) {
errno: -104,
code: 'ECONNRESET',
syscall: 'read'
}
uncaughtException
app starting: 2022-03-19T18:59:18.219Z |
I gave up. I've tried every error watcher I can think up and had to resort to: process.on('uncaughtException', (error, origin) => {
if (error?.code === 'ECONNRESET') return;
console.error('UNCAUGHT EXCEPTION');
console.error(error);
console.error(origin);
process.exit(1);
}); Unless I'm missing one, there's a leak somewhere. Edit: I added 'sessionError' and upgraded to |
Hi I was also having the same problem. found a few causes I get these responses in node V14, V16 and V17. I hope this can help anyone solve their problems in the future
|
I was able to solve all my leaks with the following:
(not exactly my code, but should handle all cases and leave no leaks) |
Upgrading Node to 16.15 from 16.14 solved the issue for us. Specifically it was happening on webpack start. |
Hi team, read ECONNRESET I am using the https call module and this happens only when there is connecting from external vendor api which is not internal network and I don't see this issue when there is internal applications with in the network. Currently I am using the Node version in my docker file as14.15.0. So based on the above comments I see this is an bug in Node V14 in other forum. I just want to check is this fix on Node V16 LTS version |
@meznaric I am facing this issue while calling the https call from my GKE application to external vendor API. Yesterday I upgraded my node js version from 14 --> |
@jaiswarvipin Is this issue resolved for you? If yes could you please let me know how it got fix and I am facing the same |
I'm having the same issue 🗡️ The nodejs version doesn't matter, I used nvm and tried a bunch of them. |
Same issue here. With (18.3.0, 17.4.0, 16.14.0, 16.15.0) only 15.9.0 not cause this issue. I'am starting react app with babel. Use node package manager like n and downgrade node to 15.9.0 temporary helps me. |
Again, this isn't a Node issue. If you don't add a catch for client errors, your app will crash whenever a client doesn't close a connection cleanly. I've been running with the logic in #42154 (comment) for over a month with public servers without anymore crashes or memory leaks. (Though I did add a Best of luck to you guys implementing the error catchers. And even more luck to those of you that don't. 😆 |
@gopinaresh we went from 16.14.2 to 16.15.0
Not sure, it probably has multiple causes. My memory is not fresh on this issue, but IIRC upgrade solved the issue on CI and dev machines. I also checked change log of node release and I think something relevant changed (that was the reason why I attempted node upgrade). |
@meznaric Share your To explain better, connection resets are a normal part of all HTTP client/server connections. It's just more probable to happen with TLS instead of plaintext. For example, with HTTP:
With HTTPS:
Even though the connection can drop at any point, with HTTPS (TLS) there are more packets being exchanged. If any point the client decides to leave (aborts request) before the transaction finishes, you'll get a connection reset. It's normal. You just have to catch it. There's also a SYN/ACK phase that's part of TCP, but that happens before/during a connection is made. HTTP/3 does away with that by moving to UDP, so HTTP/3 could technically expose more flow interruptions to the user. |
@clshortfuse I encountered this issue as well |
@coding-chris-kao You probably want to take a look at the Range header and how it works. With files, it's probably not too hard to implement, as long as you don't use Content-Encoding. https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests |
The code is so lengthy. Why node emit a uncatched expecption? If it emit a normal error event on server object, it would be more safe and easy to handle. |
It doesn't. There's a missing 'error' event listener somewhere in the application. |
I am also having similar issues when trying to make a GET Requests via https.
|
In our case we needed to use internal kubernets urls instead of the external link when making requests |
I'm going to close this out because (as has been mentioned several times in this thread), this is not a node bug. ECONNRESET means the peer closed the connection (i.e., a fact of life you simply need to deal with) and should be handled by adding 'error' event listeners in the right places. |
According to the HTTP specification, this error is related to a "data race" but no specific solution is provided.
in our case, we decided to retry the request on reusedSocket when saw ECONNRESET error. |
Same issue here. I'm making https client from react running on localhost:9001 to my nodejs app running on localhost:8080 (http only). When I make api call, the response frombackend is received but in case of graphql query, we are not getting any response though graphql-client is able to build and send response I cross-checked in debug-mode. what could be the issue ?
|
Refs1:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448436401 Refs2:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448436201 Refs3:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448291065 Reason: Unknown, maybe the network issue (for more similar issues at: nodejs/node#42154)
Refs1:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448436401 Refs2:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448436201 Refs3:https://github.com/eggjs/egg/actions/runs/5233068028/jobs/9448291065 Reason: Unknown, maybe the network issue (for more similar issues at: nodejs/node#42154)
What about a bit more helpful log messages?
For my opinion it says almost nothing. |
This is absolutely correct...
...but the real issue here isn't the ECONNRESET. It's the fact that the error object does not contains enough information to identify the "right places" to add the needed listeners. For example, this is the entire call stack for the one that restarted my service just now:
I suspect that those worthless call stacks are not a nodejs problem, but rather a Javascript runtime problem. But even so, could nodejs do something to help? For example, provide default error listeners that just log information about which listener caught the error before terminating the process. I'm too new to nodejs (and Javascript) to really have an opinion about how to mitigate this issue, but as someone who is stuck developing atop nodejs I would very much appreciate a workaround for those worthless call stacks. |
I am studying nextjs from Vercel's courses: Learn Next.js .Everything is ok in first several days ,until this ECONNRESET error occurs . I didn't figure out why this happen now ,because I think I didn't change things on the project except the net env (cause I use VPN),but I found this link help me to keep going :Cannot seed database . Maybe try 'vercel dev' instead of 'npm run dev' will work . |
With no stack trace in the exception, I have no idea where to catch this exception in my code. |
Version
17.6.0
Platform
No response
Subsystem
No response
What steps will reproduce the bug?
Sending 1000 TPS request
How often does it reproduce? Is there a required condition?
When TPS increase then, its break and server get restart
What is the expected behavior?
Process getting terminated, in ideal case, it should not terminate the process to stop
What do you see instead?
due to the process break, kubernetes POD getting crash, resulting all that movement in progress requests are getting fail, nd most importly server is not available
Additional information
No response
The text was updated successfully, but these errors were encountered: