Function calling

Function calling lets a model ask for something it cannot do itself, like looking up an order or reading the time. The model names a function and fills in its arguments. Your code runs it and returns the result.

The gateway never runs your functions. It only passes the request to you and the result back to the model. You stay in control of what executes.

The flow

1. You declare functions in tools and send a message.
2. The model replies with one or more tool_call events and the turn ends with stop_reason: "tool_use".
3. You run each function.
4. You start a new turn with the original messages, the model’s calls, and your results.
5. The model writes its final answer.

Declare a function

Each tool has a name, an optional description, and parameters, which is a JSON Schema object.

const tools = [
	{
		name: 'get_weather',
		description: 'Get the current temperature for a city.',
		parameters: {
			type: 'object',
			properties: {
				city: { type: 'string', description: 'City name, for example Paris' }
			},
			required: ['city']
		}
	}
];

Names may use letters, digits, underscores and hyphens, up to 128 characters. parameters.type must be "object". A clear description helps the model decide when to call the function.

Call it and send the result

import { connect } from './gateway.js';

const gateway = await connect();

async function getWeather({ city }) {
	return { city, temperature_c: 18 }; // call your real service here
}

const messages = [
	{ role: 'user', content: [{ type: 'text', text: 'What is the weather in Paris?' }] }
];

// 1. Ask. The model may request a call.
const first = await gateway.turn({ provider: 'auto', model: 'auto', tools, messages });

if (first.toolCalls.length > 0) {
	// 2. Repeat the model's request in the history.
	messages.push({
		role: 'assistant',
		content: first.toolCalls.map((call) => ({
			type: 'tool_call',
			id: call.id,
			name: call.name,
			input: call.input
		}))
	});

	// 3. Run the functions and add one result per call.
	const results = [];
	for (const call of first.toolCalls) {
		const output = await getWeather(call.input);
		results.push({ type: 'tool_result', id: call.id, output });
	}
	messages.push({ role: 'tool', content: results });

	// 4. Ask again with the results. Keep sending the same tools.
	const second = await gateway.turn({ provider: 'auto', model: 'auto', tools, messages });
	console.log(second.text); // "It is 18°C in Paris."
}

gateway.close();

The tool_call event looks like this:

{ "type": "tool_call", "index": 0, "id": "call-1", "name": "get_weather", "input": { "city": "Paris" }, "seq": 3, "...": "..." }

input is already parsed. id ties a call to its result: the tool_result you send must use the same id as an earlier tool_call in the history, or the request fails with invalid_request.

Rules to know

• Run each call once. Use the final tool_call event, not the partial JSON in tool_call block deltas. Deltas are a live preview only. If you reconnect and replay events, track the call id so you do not run it twice.
• Several calls at once. A model can request more than one function in a turn. Return every result in the same tool message.
• Keep sending tools. The model needs the declarations on every turn that might call them.
• Routing. Declaring tools means only models that support tools can serve the turn. If a model you named cannot, the turn fails with no_capable_model.
• No switch after delivery. Once a tool call has reached you, the gateway does not move that turn to another model, because the work may already have run. If that turn then fails, you get an error instead of a fallback.
• Return errors as results. If your function fails, send output: { "error": "City not found" }. The model can explain the problem or try again.

Next steps

• Extend the models with tools: let the model call functions in a loop.
• Conversation state