bhylu.com

Advertisements

Node.js Stream Pipelines

Streams move chunks through a workflow without requiring one complete in-memory value at every stage. A pipeline coordinates errors, completion and pressure from a slower consumer.

How it works

Use the promise-based pipeline API when the caller should await the whole transfer. A source does not become bounded simply because it is called a stream; the source and each transform still need sensible limits. This fixture transfers three known chunks to an owned writable collector. The collector is intentionally tiny and is not a pattern for accumulating arbitrary uploads without a maximum size.

Run the code

Run this example locally with Node.js. The exact sources are reviewed on Node 22.15.1 and Node 24.22.1 unless a specialised lab states otherwise. Save the complete source as stream-pipeline.mjs in a separate empty exercises folder and run node stream-pipeline.mjs. There is no Node.js browser runner on Bhylu; the plain JavaScript playground does not provide these filesystem, server or process APIs.

Download the complete source. Ordinary Node scripts need no Apache, PHP or MySQL service. File examples own their temporary folders; network fixtures use loopback and available temporary ports.

Example

stream-pipeline.mjs

import { Readable, Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
let text = '';
const target = new Writable({
  write(chunk, encoding, callback) {
    text += chunk.toString();
    callback();
  },
});
await pipeline(Readable.from(['tripod', ':', '3']), target);
console.log(text);
console.log(target.writableFinished);

Expected result

tripod:3
true

The awaited pipeline does not finish until the writable has completed. For a large export, replace the collector with a real bounded destination rather than retaining every chunk in a JavaScript string. Errors must still be handled by the caller.

Use the Node.js API reference for the precise version contract.