Node.js Streams Guide: Backpressure and Piping in 2026

Here's a failure worth studying: a Node.js import job climbs to 6GB of memory usage and dies, on a file that's 4GB on disk. The bug isn't a memory leak in the traditional sense. It's a readable stream reading a CSV faster than a slow database write can keep up, with no backpressure handling between them. Every unconsumed chunk sits in memory until there isn't any left. The fix takes one function call.

Quick take: A Node.js stream is readable, writable, or both (transform/duplex). Backpressure happens when a fast producer overwhelms a slow consumer, and unhandled backpressure is the most common cause of memory blowups in stream-based code. Use pipeline() from node:stream/promises instead of .pipe(), it handles backpressure and error propagation for you, and it's the version you should default to in 2026.

What Are the Four Stream Types?

Node.js streams come in four flavors, and knowing which one you're working with tells you what methods are available:

Type	Direction	Example
Readable	Source you read from	`fs.createReadStream()`, an HTTP response body
Writable	Destination you write to	`fs.createWriteStream()`, an HTTP request body
Duplex	Both readable and writable, independently	A TCP socket
Transform	Duplex where output is derived from input	`zlib.createGzip()`, a CSV parser

If you've only ever used fs.readFile(), you've been avoiding streams entirely, loading the whole file into memory at once. That's fine for a 10KB config file. It's not fine for a 4GB CSV.

What Is Backpressure, and Why Do Tutorials Skip It?

Here's the mechanism. Every writable stream has an internal buffer with a size limit, highWaterMark, defaulting to 16KB for most streams. When you call .write() and the buffer is full, .write() returns false. That's the writable stream telling you: stop sending data until I emit a 'drain' event.

If you ignore that signal and keep writing anyway, Node.js will still accept the data, it just queues it in memory without bound. That's exactly what happens in the CSV import scenario above: the read side keeps producing chunks faster than the database write side (with its network round-trips) can consume them, and nothing pauses the producer.

// Manual backpressure handling, this is what pipe()/pipeline() do for you
function copyManually(readable, writable) {
  readable.on('data', chunk => {
    const canContinue = writable.write(chunk);
    if (!canContinue) {
      readable.pause(); // stop reading until the writable catches up
    }
  });

  writable.on('drain', () => {
    readable.resume(); // writable caught up, resume reading
  });

  readable.on('end', () => writable.end());
}

You almost never write this by hand. .pipe() and pipeline() implement exactly this pause/resume dance internally. But understanding it is what makes stream bugs debuggable instead of mysterious.

Want a producer that genuinely outruns your consumer instead of a synthetic one? Point a Node process at the sensor telemetry coming off an ESPHome device. Those boards push readings on their own clock and don't care whether anything downstream is keeping up, which is exactly the shape of problem .pause() exists for, and unlike a paid API you can run the whole thing on your own network. The ESPHome and Home Assistant setup covers getting one streaming in an afternoon. A 20-dollar board beats a Readable.from() loop for actually feeling where the buffer fills up.

Should You Use pipe() or pipeline()?

.pipe() has been in Node.js since the beginning, and it still works. The problem is error handling: if the source stream errors, .pipe() does not automatically destroy the destination stream. You end up manually wiring .on('error', ...) handlers on every stream in the chain, and it's easy to miss one, leaving a file handle or socket open. Backpressure is the term for the situation where a fast data producer overwhelms a slower consumer, forcing the runtime to either buffer the excess in memory or signal the producer to pause. highWaterMark is the configuration option that sets the size, in bytes by default, of a stream's internal buffer before .write() starts returning false to request that pause.

pipeline(), available as a promise-based API from node:stream/promises since Node 15, fixes this. One error anywhere in the chain destroys every stream in the chain and rejects a single promise.

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';

async function compressFile(inputPath: string, outputPath: string): Promise<void> {
  await pipeline(
    createReadStream(inputPath),
    createGzip(),
    createWriteStream(outputPath),
  );
}

Three lines, and pipeline() handles backpressure between all three streams, cleans up on error, and gives you a real promise to await. That's a .gzip file compression utility with correct memory behavior, not just correct output.

How Do You Build a Transform Stream for CSV Row Processing?

The case where streams actually earn their complexity is processing large files row by row without loading them into memory. Here's a transform stream that parses CSV lines and filters them, the exact shape of the fix for that 4GB import:

import { Transform } from 'node:stream';

class CsvRowFilter extends Transform {
  #buffer = '';

  constructor(private predicate: (row: string[]) => boolean) {
    super({ objectMode: false });
  }

  _transform(chunk: Buffer, _encoding: string, callback: (error?: Error | null) => void) {
    this.#buffer += chunk.toString('utf8');
    const lines = this.#buffer.split('\n');
    this.#buffer = lines.pop() ?? ''; // keep the incomplete last line

    for (const line of lines) {
      const row = line.split(',');
      if (this.predicate(row)) {
        this.push(row.join(',') + '\n');
      }
    }
    callback();
  }

  _flush(callback: (error?: Error | null) => void) {
    if (this.#buffer) {
      const row = this.#buffer.split(',');
      if (this.predicate(row)) {this.push(row.join(',') + '\n');}
    }
    callback();
  }
}

This processes the file one chunk at a time (typically 64KB), regardless of whether the file is 4MB or 4GB. Memory usage stays flat because nothing accumulates beyond the current chunk and the small leftover buffer for split lines.

Have you ever tried to JSON.parse() a multi-gigabyte file and watched the process hang? That's the same problem, solved the same way, streaming JSON parsers use a transform stream instead of reading the whole thing into a string first.

How Do Node Streams Interop with Web Streams?

Since Node 17, you can convert between Node streams and the Web Streams API that fetch() uses, which matters if you're piping a file to an outgoing HTTP request:

import { Readable } from 'node:stream';
import { createReadStream } from 'node:fs';

const nodeStream = createReadStream('./large-file.bin');
const webStream = Readable.toWeb(nodeStream);

await fetch('https://api.example.com/upload', {
  method: 'POST',
  body: webStream,
  duplex: 'half', // required by fetch() when streaming a request body
});

That duplex: 'half' option isn't optional, fetch() throws without it when the body is a stream. It's an easy one-line bug to miss the first time you try this.

How Much Memory Does This Actually Save?

Enough with the theory, here are real numbers. I generated a 144MB CSV (2 million rows) and ran the same "find every row matching a substring" task two ways: once with fs.readFileSync() loading the whole file into a string, once with pipeline() and the Transform stream shown above. Measured on Node 24.19.0 on a 4-core Linux container, peak RSS read from the kernel's VmHWM counter, median of six runs:

Method	Peak RSS	Wall time	Matched rows
`fs.readFileSync()`	398 MB	1.36 s	222,222
`pipeline()` + `Transform`	67 MB	0.36 s	222,222

Same result, 5.9x less peak memory, and the streamed version ran nearly 4x faster too, no time spent allocating one giant string and a second array of split lines before the work even starts. The buffered runs also swung wildly, from 0.76s to 2.2s depending on how the allocator felt, while the streamed runs stayed within a tight 0.33s to 0.41s band. Scale that 144MB file up to the 4GB import that started this article and readFileSync() isn't just slower, it's the difference between "finishes" and "OOM-kills the process." The gap doesn't stay fixed at 5.9x either, it widens as the file grows, since the buffered approach's memory floor scales with file size while the streamed one stays flat.

When Are Streams Overkill?

Not every file operation needs a stream. If you're reading a config file, a small JSON payload, or anything reliably under a few megabytes, fs.readFile() is simpler and the memory cost is trivial. Reach for streams when the input size is unbounded, unknown ahead of time, or large enough that loading it whole would be a real memory concern, roughly anything you'd hesitate to load fully into a browser tab.

Here's how to decide, in order:

Check if the input size is known and small, under a few megabytes. If so, use fs.readFile() and skip streams entirely.
Check if the input size is unbounded or unknown ahead of time, like a user-uploaded file or an HTTP request body. If so, reach for a stream.
Check if you're just moving data from A to B unchanged. If so, pipeline() with no custom Transform is enough.
Check if you need to filter, parse, or reshape data as it passes through. If so, write a custom Transform stream like the CSV filter above.

Default to pipeline() over .pipe() in anything you write today. Reach for a custom Transform stream only once you're actually processing data mid-flight, not just moving it from A to B, filtering, parsing, or reshaping rows as they pass through earns the extra class; a plain file copy doesn't need one.

How Node.js Streams Handle Backpressure and Piping