Open7

Stream API

ytaisei（たいせー）ytaisei（たいせー）

概念

ストリーミングでは、ネットワーク経由で受信するリソースを小さなチャンク（塊）に分割し、少しずつ処理する。ストリームの開始または終了の検出、ストリームの連鎖、エラー処理と必要に応じたストリームのキャンセル、ストリームの読み取り速度への対応が可能。

Readable streams

Readable Streamはunderlying sourceから流れてくるJSのオブジェクト。underlying sourceには二つのタイプがある。

  1. Push sources
  • constantly push data at you when you've accessed them, and it is up to you to start, pause, or cancel access to the stream.
  1. Pull sources
  • require you to explicitly request data from them once connected to.

A chunk can be a single byte , or it can be something larger such as a typed array of a certain size.

  • The chunks placed in a stream are said to be enqueued

    • internal queue keeps track of the chunks that have not yet been read

  • The chunks inside the stream are read by a reader

    • The reader plus the other processing code that goes along with it is called a consumer.

  • There is also a construct you'll use called a controller

    • each reader has an associated controller that allows you to control the stream

Only one reader can read a stream at a time.

  • when a reader is created and starts reading a stream (an active reader), we say it is locked to it.
ytaisei（たいせー）ytaisei（たいせー）

Teeing

it is possible to split a stream into two identical copies, which can then be read by two separate readers. This is called teeing.

You might do this for example in a ServiceWorker if you want to fetch a response from the server and stream it to the browser, but also stream it to the ServiceWorker cache.

Writable streams

This serves as an abstraction over the top of an underlying sink — a lower-level I/O sink into which raw data is written.

The data is written to the stream via a writer, one chunk at a time.

ytaisei（たいせー）ytaisei（たいせー）

Pipe chains

The Streams API makes it possible to pipe streams into one another using a structure called a pipe chain.

Backpressure

this is the process by which a single stream or a pipe chain regulates the speed of reading/writing.

To use backpressure in a ReadableStream, we can ask the controller for the chunk size desired by the consumer by querying the ReadableStreamDefaultController.desiredSize property on the controller.

Internal queues and queuing strategies

Internal queues employ a queuing strategy, which dictates how to signal backpressure based on the internal queue state.

In general, the strategy compares the size of the chunks in the queue to a value called the high water mark, which is the largest total chunk size that the queue would prefer to manage.

high water mark - total size of chunks in queue = desired size
ytaisei（たいせー）ytaisei（たいせー）

Using readable streams

// Fetch the original image
fetch("./tortoise.png")
  // Retrieve its body as ReadableStream
  .then((response) => response.body)
  .then((body) => {
    const reader = body.getReader();
    // …
  });

Invoking this method creates a reader and locks it to the stream — no other reader may read this stream until this reader is released

reader.read().then(({ done, value }) => {
  /* … */
});

three types.

  • If a chunk is available to read, the promise will be fulfilled with an object of the form { value: theChunk, done: false }.
  • If the stream becomes closed, the promise will be fulfilled with an object of the form { value: undefined, done: true }.
  • If the stream becomes errored, the promise will be rejected with the relevant error.
ytaisei（たいせー）ytaisei（たいせー）

If you want to stop iterating the stream you can cancel the fetch() operation using an AbortController and its associated AbortSignal:

const aborter = new AbortController();
button.addEventListener("click", () => aborter.abort());
logChunks("http://example.com/somefile.txt", { signal: aborter.signal });

async function logChunks(url, { signal }) {
  const response = await fetch(url);
  for await (const chunk of response.body) {
    if (signal.aborted) break; // just break out of loop
    // Do something with the chunk
  }
}
ytaisei（たいせー）ytaisei（たいせー）

Using readable byte streams

Readable byte streams are readable streams that have an underlying byte source of type: "bytes", and which support efficient zero-copy transfer of data from the underlying source to a consumer (bypassing the stream's internal queues).

Overview

In a normal readable stream, data from the underlying source always passes to a consumer through the internal queues.

A readable byte stream differs in that if the internal queues are empty, the underlying source can write directly to the consumer

The main difference between ReadableByteStreamController and the default controller (ReadableStreamDefaultController) is that it has an additional property ReadableByteStreamController.byobRequest of type ReadableStreamBYOBRequest.