Middleware · Getting Started

Quick Start

Install the TypeScript middleware package and its LangGraph.js peer dependencies:

npm install @threadplane/middleware @langchain/core @langchain/langgraph

The package exposes its JavaScript API from @threadplane/middleware/langgraph. For Python LangGraph, install threadplane-middleware and follow the Python guide.

Add client-tool state channels

import { Annotation, END, MessagesAnnotation, StateGraph } from '@langchain/langgraph';
import {
  bindClientTools,
  clientToolsChannel,
  clientToolsRouter,
} from '@threadplane/middleware/langgraph';
 
const State = Annotation.Root({
  ...MessagesAnnotation.spec,
  ...clientToolsChannel(),
});

clientToolsChannel() adds both tools and client_tools. The middleware reads tools first and falls back to client_tools.

Bind tools per run

Call bindClientTools() inside the graph node, not once at module load. The browser sends the catalog with each run, so the tool list is request-scoped.

const SERVER_TOOLS: unknown[] = [];
const serverToolNames: string[] = [];
 
async function agent(state: typeof State.State) {
  const llm = bindClientTools(baseLlm, SERVER_TOOLS, state);
  const response = await llm.invoke(state.messages);
  return { messages: [response] };
}

SERVER_TOOLS is where your server-owned LangChain tools go. Client tools from state are appended as model-visible function stubs.

Route after the agent

const graph = new StateGraph(State)
  .addNode('agent', agent)
  .addEdge('__start__', 'agent')
  .addConditionalEdges('agent', clientToolsRouter(serverToolNames), ['tools', END])
  .compile();

When the last model message calls a server tool, the router returns the server tools node. When the last model message calls only browser-declared client tools, the router returns END so the frontend can execute the call and resume.

Complete skeleton

import { Annotation, END, MessagesAnnotation, StateGraph } from '@langchain/langgraph';
import { ChatOpenAI } from '@langchain/openai';
import {
  bindClientTools,
  clientToolsChannel,
  clientToolsRouter,
} from '@threadplane/middleware/langgraph';
 
const State = Annotation.Root({
  ...MessagesAnnotation.spec,
  ...clientToolsChannel(),
});
 
const baseLlm = new ChatOpenAI({ model: 'gpt-4o-mini' });
const serverTools: unknown[] = [];
const serverToolNames: string[] = [];
 
async function agent(state: typeof State.State) {
  const llm = bindClientTools(baseLlm, serverTools, state);
  const response = await llm.invoke(state.messages);
  return { messages: [response] };
}
 
export const graph = new StateGraph(State)
  .addNode('agent', agent)
  .addEdge('__start__', 'agent')
  .addConditionalEdges('agent', clientToolsRouter(serverToolNames), ['tools', END])
  .compile();

Frontend pairing

On the frontend, declare client tools with @threadplane/chat and send them through an adapter that forwards the tool specs into the run input. The middleware consumes those specs on the backend; the browser remains responsible for executing the actual local function, view, or ask tool.

Next steps