Quick Summary: Node.js app on Linux 5.4 experiencing PG connection resets under high load? This SRE guide uncovers the obscure kernel/Node.js interaction causing...

You're staring at the logs again, aren't you? Another Node.js ECONNRESET from your database connection pool. But only sometimes. Only under load. And only after 3 AM, Tuesdays, when Jupiter aligns with Mars. Sound familiar? Good. Because I've been there, pulling my hair out, convinced the universe hates my database server. Spoiler: it wasn't the database. It was far, far worse.

This isn't your garden-variety network flake. This isn't a firewall rule. This isn't even a misconfigured max_connections on your PostgreSQL instance. This is an insidious, long-tail problem, a nasty interplay between specific Linux kernel versions, Node.js's default network stack behavior, and TCP Fast Open. Yes, the "optimization" feature that's supposed to make things faster. Sometimes, fast also means broken.

The Symptoms

Your Node.js application, usually a stalwart, starts spewing ECONNRESET errors or EPIPE when trying to write to what should be an active database connection. This happens intermittently. It spikes under heavy concurrent load. Your monitoring screams about connection churn on the Node.js side, but PostgreSQL itself looks fine – no runaway queries, no OOM, plenty of available connections. You might even see a spike in "backend closed connection" messages on the PG side, but the PG logs provide no specific reason, just the sudden termination.

Restarting the Node.js service sometimes provides temporary relief, but the problem always resurfaces. You've checked network cables, increased file descriptor limits, tweaked Node.js connection pool settings to be more conservative. Nothing. It just keeps happening, driving your error rates through the roof and your SRE team to the brink.

The Triggering Environment

This particular flavor of hell seems to thrive in a very specific matrix of OS and Node.js versions. If you're running this combination, pay close attention.

Operating System	Kernel Version	Node.js Version (LTS)	Observed Database Target
Ubuntu 20.04 LTS	5.4.0-x-generic (e.g., 5.4.0-109-generic)	14.17.x, 14.18.x, 14.19.x	PostgreSQL 12.x, 13.x, 14.x
Debian 10 (Buster)	4.19.x or 5.4.x (backports)	14.17.x - 14.19.x	PostgreSQL 12.x, 13.x, 14.x
AWS EC2 Instances (m5.x, c5.x)	Kernel 5.4.x (Ena driver)	14.17.x - 14.19.x	PostgreSQL (RDS or self-managed)

What Didn't Work (And Why It's Misleading)

• Increasing ephemeral port range: You think you're running out of ports. You're not. The problem isn't availability, it's how they're being managed internally.
• Tuning tcp_tw_reuse, tcp_max_syn_backlog: Standard network tuning. Good practice, but doesn't touch the core issue here.
• Database-side tuning (max_connections, wal_buffers): The database is often an innocent bystander, its logs merely reflecting the client's self-inflicted wounds.
• Upgrading pg or pg-pool versions: While good for security and features, this specific issue is lower in the stack.

The misleading part is how ECONNRESET makes you instinctively look at the server or general network issues. But in this case, the RST is originating from the client's own kernel due to a subtle state machine race, not the server actively closing the connection or a firewall blocking traffic.

The Root Cause

Alright, let's get to the ugly truth. This mess boils down to a specific, poorly handled interaction involving TCP Fast Open (TFO) on certain Linux kernel 5.4.x branches. TFO is designed to speed up successive TCP connections by sending data in the initial SYN packet if a TFO cookie is available. Great in theory.

The bug: Under high connection churn and rapid ephemeral port recycling, a race condition exists in these kernels where the TFO mechanism, when coupled with how net.Socket (via libuv) requests new connections, can lead to the client's kernel prematurely sending an RST packet. This happens because the kernel attempts to use a stale TFO cookie, or gets confused by the rapid reuse of ephemeral ports, causing its internal TCP state machine to become inconsistent before the full handshake completes. Instead of cleanly dropping the connection attempt, it just... resets it. From its own side. This manifests as ECONNRESET in your Node.js application.

The reason Node.js 14.x is particularly susceptible is that its underlying libuv doesn't explicitly disable TFO for outgoing connections, nor does it necessarily employ specific workarounds that later Node.js versions or different client libraries might. This is a classic case where an "optimization" creates an obscure failure mode that only manifests under stress in very specific environments. It's the kind of subtle failure that requires digging deep into network stack behavior, a level of detail often necessary when engineering sub-millisecond trading APIs or sub-microsecond algorithmic trading architectures where connection reliability is paramount.

The Fix (Finally, Some Relief)

The solution is deceptively simple: tell Node.js to stop trying to be clever with TCP Fast Open for outgoing connections to your database. You achieve this by explicitly setting the enableTCPFastOpen option to false when establishing your database connections. This often means adjusting your connection string or the options passed to your pg or pg-pool client.

Here's how you'd typically modify your Node.js pg client configuration. This is a global override if you're using pg.Pool, or per-client if you're creating individual Client instances:

const { Pool } = require('pg');

const pool = new Pool({
  user: 'youruser',
  host: 'yourhost',
  database: 'yourdatabase',
  password: 'yourpassword',
  port: 5432,
  max: 20, // max number of clients in the pool
  idleTimeoutMillis: 30000, // how long a client is allowed to remain idle before being closed
  connectionTimeoutMillis: 2000, // will try to connect for 2 seconds before timing out
  // The magic line to fix ECONNRESET on Linux 5.4.x kernels:
  stream: {
    enableTCPFastOpen: false
  }
});

pool.on('error', (err, client) => {
  console.error('Unexpected error on idle client', err);
  process.exit(-1);
});

// Example usage:
// async function queryDatabase() {
//   const client = await pool.connect();
//   try {
//     const res = await client.query('SELECT NOW()');
//     console.log(res.rows[0]);
//   } finally {
//     client.release();
//   }
// }
// queryDatabase();

Note: The stream option with enableTCPFastOpen is not always directly exposed by all pg client versions or wrappers. You might need to dig into the documentation of your specific ORM or connection pool library to find where to pass options directly to the underlying net.Socket constructor. For the base pg client, this works as shown.

Why This Works

By setting enableTCPFastOpen: false, you instruct Node.js's net module (and thus libuv) to explicitly disable TFO for that specific outgoing TCP connection. This bypasses the problematic code path in the Linux kernel 5.4.x series that leads to the race condition and spurious RST packets. Your connections will still establish, just without the potential (and in this case, detrimental) "fast open" optimization. Given the latency of typical database interactions, the performance impact of disabling TFO is negligible compared to the chaos of constant ECONNRESET errors.

Final Thoughts

This is a classic SRE nightmare: an obscure interaction bug between layers of the stack, manifesting intermittently under load, and pointing fingers everywhere but the actual culprit. It wastes days, if not weeks, of valuable engineering time. Understand your entire stack, from application code down to the kernel and network drivers. That's how you find the ghosts.

Now, go patch your damned Node.js services before they make you throw your keyboard across the room. You're welcome.