Quick Summary: Solve the maddening Node.js TLS server hang when serving large files on specific older Linux kernels in Docker. Understand sendfile()...

Alright, listen up. You’ve hit that wall. The one where your Node.js HTTPS server, happily serving smaller files, just hangs when you try to push anything substantial. No errors. No crashes. Just silence. Your client times out, the connection sits there, and you’re left pulling your hair out. I’ve seen this countless times. It’s a subtle, infuriating beast, and it’s likely not your code.

This isn't some garden-variety memory leak or CPU spike. This is a low-level interaction between Node.js, your operating system’s kernel, and OpenSSL. It’s a silent killer, and it’s been known to drive good engineers to rethink their career choices.

The Symptoms of the Silent Killer

Let's confirm you're seeing the right kind of hell:

• Large Files Only: Smaller files (under ~50-100MB) transfer just fine. The problem only manifests with bigger payloads, especially downloads.
• HTTPS Only: Your HTTP server, running the exact same file serving logic, works without a hitch. Switch to HTTPS, and BAM, it locks up.
• Specific Environments: This isn't universal. It hits hard on particular older Linux kernels, often when running Node.js inside Docker containers whose host kernel is, let's say, 'seasoned'.
• No Visible Errors: Node.js doesn't crash. There are no uncaught exceptions. Your application logs look clean. Your client just waits, then gives up.
• Stuck Connections: A quick lsof -i :<your_port> will show established connections, but zero bytes transferred after the initial handshake.

You’ve probably already tried the usual suspects: checking network configurations, proxy settings, fiddling with Node.js buffer sizes. You might even have tried strace, only to see sendfile() calls either succeeding (for small amounts) or blocking indefinitely, offering no real insight into why. Stop wasting your time. The issue is deeper.

The Cursed Environments

This problem is particularly prevalent in environments where older kernel versions are still in play, often masked by modern Docker setups. Here’s a typical trigger matrix:

OS Version	Kernel Version	Node.js Version (affected)	Containerization	Status
Ubuntu 16.04 LTS (Xenial)	3.10.0 - 3.19.x	10.x, 12.x	Docker (host kernel)	TRIGGERS
CentOS 7	3.10.0-x.el7	10.x, 12.x, 14.x	Bare Metal / Docker	TRIGGERS
Debian 8 (Jessie)	3.16.x	10.x, 12.x	Docker (host kernel)	TRIGGERS
Ubuntu 18.04 LTS (Bionic)	4.15.x	10.x, 12.x, 14.x	Docker / Bare Metal	WORKS (usually)
Linux Kernel >= 4.9	4.9.x+	10.x - 18.x	Docker / Bare Metal	WORKS (typically)

The Root Cause

Let's dissect this mess. Node.js, when you do something like fs.createReadStream(filePath).pipe(res), tries to be smart. Really smart. It attempts to use the operating system's sendfile(2) system call. This is a brilliant optimization: it allows the kernel to directly copy data from a file descriptor to a socket descriptor, completely bypassing user-space buffers. This zero-copy mechanism significantly reduces CPU overhead and improves execution latency for high-throughput file serving.

The problem arises with TLS (HTTPS). When you send data over a secure connection, that data must be encrypted by OpenSSL (or an equivalent TLS library) before it hits the network. This encryption process happens in user-space. The kernel's sendfile() call, however, is designed for raw data transfer, not pre-encrypted buffers.

On specific older Linux kernels (especially in the 3.x series, prior to more robust fixes around 4.9), the kernel’s implementation of sendfile(), combined with Node.js's libuv layer trying to optimize, interacts poorly with TLS sockets. Node.js knows it shouldn't use sendfile() for TLS sockets because the data needs encryption. It tries to detect this.

But here’s the rub: on these older, quirky kernels, the sendfile() detection logic (or the kernel's behavior itself when presented with a TLS-backed socket) can get confused. It attempts to perform a sendfile() operation directly to what it perceives as a regular socket, even though it's wrapped by OpenSSL. The kernel tries to move raw file data, bypassing OpenSSL. OpenSSL, of course, never gets the data to encrypt. The kernel then enters a state where it's waiting for the socket to be ready to accept raw data, while the application (Node.js/OpenSSL) is waiting for encrypted data to be sent.

It's a classic deadlock. The kernel thinks it's being smart, Node.js thinks the kernel is handling it, and OpenSSL is completely out of the loop. The connection just hangs, silently, indefinitely, because neither side can make progress. This is similar to how other obscure kernel issues, like the one described in The Invisible Assassin: Why Your Node.js Child Processes Vanish on Older Linux Kernels in Docker, can cause silent failures that defy conventional debugging. It's a reminder that kernel versions matter, especially when containerizing.

The Fix: Disable sendfile()

The solution is brutally simple and profoundly irritating: force Node.js to not use sendfile(). This ensures all file data is read into Node.js’s user-space buffers, then passed through OpenSSL for encryption, and only then written to the network. It bypasses the problematic kernel optimization altogether.

You lose the zero-copy benefit, yes. But for most applications, the overhead of user-space buffering and encryption is negligible compared to the alternative: a completely hung server. Unless you're pushing petabytes and measuring network latency in nanoseconds for a critical algorithmic trading platform, you won't notice the difference. Your sanity, however, will.

Set this environment variable:

NODE_NO_SENDFILE=1

You can set this in your shell before starting Node.js, in your Dockerfile, or your systemd service file. Here’s an example for a Dockerfile:

FROM node:14-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
ENV NODE_NO_SENDFILE=1
CMD ["node", "server.js"]

Final Thoughts

This issue is a stark reminder that abstractions, while powerful, can sometimes hide critical low-level interactions that come back to bite you. The Node.js team has continually improved sendfile() detection, and newer Linux kernels are generally better behaved. But in the world of legacy systems and diverse deployment environments, these 'long-tail' problems persist.

Always consider your underlying kernel version when deploying Node.js applications, especially if you're containerizing. What looks like a simple Node.js application issue can often be a complex interplay of user-space libraries and kernel quirks. Debugging these requires a deep dive, patience, and sometimes, just knowing the magic environment variable.

Save yourself the headache. If you're seeing this phantom hang, especially on older kernels, apply NODE_NO_SENDFILE=1. It's not a performance hack; it's a stability fix for a fundamental mismatch.