Batch Plugin

Combine multiple requests into a single batch and receive their responses together, reducing the overhead of separate requests.

Setup

Set up batching on both the server and the client. The server plugin handles incoming batch requests, and the client plugin groups outgoing requests into batches.

import { BatchHandlerPlugin } from '@orpc/server/plugins'

const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin(),
  ],
})

import { BatchLinkPlugin } from '@orpc/client/plugins'

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})

Streaming Responses

The server starts the batch response once every procedure has responded, so it can carry their headers. The response streams if one of them returned an AsyncIteratorObject or a ReadableStream. If your client cannot read streamed responses, keep these procedures out of the batch with filter.

Keep-Alive Timer

A streaming batch response may stay idle for long periods. The server plugin can send keep-alive frames to keep the connection alive.

const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin({
      keepAlive: {
        /**
         * If true, a keep-alive frame is sent periodically while the stream is idle.
         *
         * @default true
         */
        enabled: true,
        /**
         * Interval (in milliseconds) between keep-alive frames after the last message.
         *
         * @default 15000
         */
        interval: 15000,
      },
    }),
  ],
})

Compression

A batch response might be left uncompressed by the Response Compression Plugin, depending on the shape it takes. On Node.js, add the Batch Response Compression Plugin when you know your batch responses are all compressible. It covers every one of them, without giving up streaming:

import { BatchResponseCompressionHandlerPlugin } from '@orpc/node'

const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin(),
    new BatchResponseCompressionHandlerPlugin(),
  ],
})

Groups

Only requests in the same group are batched together. Each group also defines a context, as described in client context.

The following example batches requests by cache policy:

interface ClientContext {
  cache?: RequestCache
}

const link = new RPCLink<ClientContext>({
  method: ({ context }) => {
    if (context?.cache) {
      return 'GET'
    }

    return 'POST'
  },
  plugins: [
    new BatchLinkPlugin({
      groups: [
        {
          condition: ({ context }) => context?.cache === 'force-cache',
          context: { // used for the rest of the request lifecycle
            cache: 'force-cache',
          },
        },
        { // Fallback for all other requests. Keep this last.
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
  fetch: (url, init, { context }) => globalThis.fetch(url, {
    ...init,
    cache: context?.cache,
  }),
})

Now, calls made with cache = 'force-cache' use that cache setting whether they are batched or sent individually.

Filtering Requests

Use filter to skip batching for specific requests before group matching runs. Requests for which filter returns false continue through the link chain individually.

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      filter: ({ path }) => !path.includes('upload'),
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})

Wait Time

By default, only requests made in the same event loop tick are batched together. Set wait to keep collecting requests for a number of milliseconds, trading a little latency for fewer, larger batches.

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      wait: 10, // ms
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})

The wait starts when the first request is queued, so no request waits longer than wait, even while more keep arriving.

Scope

On the server, requests from different users can be batched together. If your transport reads per-request data (in fetch, origin, and so on), set scope to a value unique to the incoming request.

import { AsyncLocalStorage } from 'node:async_hooks'

const requestStorage = new AsyncLocalStorage<Request>() // set per incoming request

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      scope: () => requestStorage.getStore(),
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})

Response Headers

The batch response carries the cookies and Vary fields of its subresponses, and the headers they all share. Other headers stay on their subresponse. To change what stays on each subresponse, set the server plugin’s mapSubresponse option.

Learn More

See the BatchHandlerPlugin source code and the BatchLinkPlugin source code for implementation details.