Extend the models with tools

A single function call answers one question. Real tasks often need several steps: search, read, calculate, then answer. This guide shows how to put tools in a loop so a model can work through a task.

What tools are here

Tools on the gateway are yours. You describe them, the model decides when to use them, and your code runs them. The gateway does not offer built-in tools such as web search or code execution, and it never runs anything on your behalf. That keeps your data and credentials on your side.

Good tools are small and specific: look up a record, fetch a file, run a query, send a draft for review.

A complete tool loop

This example gives the model two tools and keeps going until it stops asking for them.

import { connect } from './gateway.js';

const gateway = await connect();

const tools = [
	{
		name: 'search_docs',
		description: 'Search the product documentation. Returns matching page titles.',
		parameters: {
			type: 'object',
			properties: { query: { type: 'string' } },
			required: ['query']
		}
	},
	{
		name: 'add',
		description: 'Add two numbers.',
		parameters: {
			type: 'object',
			properties: { a: { type: 'number' }, b: { type: 'number' } },
			required: ['a', 'b']
		}
	}
];

const handlers = {
	search_docs: async ({ query }) => ({ results: [`Getting started with ${query}`] }),
	add: async ({ a, b }) => ({ sum: a + b })
};

async function runAgent(question, maxSteps = 8) {
	const messages = [{ role: 'user', content: [{ type: 'text', text: question }] }];

	for (let step = 0; step < maxSteps; step++) {
		const turn = await gateway.turn({ provider: 'auto', model: 'auto', tools, messages });

		if (turn.toolCalls.length === 0) return turn.text;

		// Record what the model asked for, including any text it wrote first.
		messages.push({
			role: 'assistant',
			content: [
				...(turn.text ? [{ type: 'text', text: turn.text }] : []),
				...turn.toolCalls.map((c) => ({ type: 'tool_call', id: c.id, name: c.name, input: c.input }))
			]
		});

		// Run every requested tool and send all the results together.
		const results = await Promise.all(
			turn.toolCalls.map(async (call) => {
				try {
					const output = await handlers[call.name](call.input);
					return { type: 'tool_result', id: call.id, output };
				} catch (error) {
					return { type: 'tool_result', id: call.id, output: { error: String(error) } };
				}
			})
		);
		messages.push({ role: 'tool', content: results });
	}

	throw new Error('The model did not finish within the step limit.');
}

console.log(await runAgent('What is 19 plus 23, and which docs page covers getting started?'));
gateway.close();

Each loop is one turn. The conversation grows by an assistant message with the calls and a tool message with the results, so the model always sees everything that happened.

Tips for reliable loops

• Cap the steps. A step limit stops a confused model from looping forever, and from spending your budget.
• Return errors, do not throw. Send { "error": "..." } as the output so the model can recover.
• Keep outputs small. Large results use input tokens on every following step. Return only what the model needs.
• Be careful with side effects. A model can call a tool twice or with wrong arguments. Make tools that change data safe to repeat, and ask a person to approve anything irreversible.
• Use each call once. Key your work on the call id. This matters if you reconnect and replay events.
• Describe tools well. The name, description and parameter descriptions are all the model knows about them.
• Mind your budget. Every step is a turn that costs tokens. Watch budget on the stream, and read Errors and handling for what happens at a limit.

Routing for agents

When your loop calls the gateway many times, give routing a hint with agent_context. Use a low difficulty for easy steps and a higher one when the model must reason. Set is_subagent: true for helper tasks that can use a cheaper model.

{ provider: 'auto', model: 'auto', tools, messages, agent_context: { difficulty: 4 } }

Because the full history is resent each step, a model switch between steps is safe.

Next steps

• Errors and handling
• Conversation state