When AI has too many tools — what a long list costs you

You connect a task system to your AI assistant. It works. You add email, since you are already there. Then the calendar, then the document store.

Three months later the same assistant answers worse than it did at the start, the bill is higher, and nobody can point to the moment something broke. Usually nothing broke. The tool list simply grew.

Where the long list comes from#

Connecting one system is not one tool.

A server exposing a task database typically offers a dozen or more separate operations: list tasks, search, fetch one, create, update, delete, comment, attach a file, list fields, change permissions. Each is its own entry on the list the model sees.

The second system adds another dozen. The third adds more. Nobody in the company ever decides "let's add a fiftieth tool" — the decisions sound like "let's connect the calendar too", and each one on its own is sensible.

What one tool takes up#

The model does not see your systems. It sees a description of them.

Every tool is a name, a sentence or two about what it does, and a parameter schema: what the parameters are called, their types, which are required, what they mean. Without that the model could not call it correctly.

That description goes to the model on every request, before the first word of your question. Including when you are asking about something the tool has nothing to do with.

Say, for illustration, that one description takes half a page. Five tools means two and a half pages of preamble before the model hears the question. Sixty means thirty pages — every time, on every message. The figures are illustrative; the proportion is not.

Two things follow. The first is cost: you pay for that preamble on every request, whether or not anything gets used. The second matters more — the longer the list, the harder it is to pick the right entry. The model compares descriptions, and across several dozen operations from four systems, plenty sound alike.

How to tell it has become a problem#

The symptoms are recognisable, but none looks like a tool-count problem. They look like the model getting worse.

The model picks the neighbouring operation. You ask it to find one specific job, you get a list of all of them. "Search" and "list" are two different things to you; to the model they are two similarly worded descriptions.

The same kind of question is slower and dearer than it was a month ago. The questions did not change. What the model reads before them did.

The model loses what was agreed at the start of the conversation. The room the preamble took is no longer available for the rest.

A task that took one step now takes three. The model tries, hits the wrong operation, corrects itself. Each attempt is a separate call and a separate charge.

The test takes half an hour: write down five typical questions with today's answers, disconnect half the systems, then ask the same five. Better answers on the shorter list tell you where the problem sits.

Three ways to shorten the list#

A set per role#

Not everyone needs everything. Service works on jobs and the schedule, sales on customers and quotes, finance on documents.

Instead of one configuration for the whole company, several smaller ones, one per role. It is the simplest of the three, needs nothing beyond a decision, and in a company of 11–200 people it is usually enough.

One condition: someone has to review those sets periodically. Roles change, and lists do not shorten on their own.

Splitting across servers#

Instead of one server exposing everything, several narrow ones, connected separately. The dividing line runs along areas of work rather than vendors: jobs, documents, mail and calendar each on their own.

The advantage is practical: whole areas go on and off with one switch rather than editing a list entry by entry. It is also easier to answer what is actually connected right now.

The cost: more things to configure and keep working.

Searching for tools per task#

Instead of the whole list, you send a small permanent set — including one tool the model uses to find the others. When a task needs something outside it, the model searches and loads only the descriptions it needs.

The preamble stays short no matter how many tools are connected. It is the only one of the three that does not stop working as things keep growing.

The cost: an extra round before the model starts on what you asked. With five tools that is wasted time. With several dozen it is a saving that grows with every addition.

A shorter list is not the same as less access#

Worth keeping these two apart — they get confused often, and here the confusion is expensive.

The model does not get access to the database. It gets a list of tools and can ask for them to be called; the application does the calling, on behalf of the signed-in user. So the permissions are the permissions of the account whose credentials were used — the protocol restricts nothing on its own. That exchange is broken into steps in connecting company data to an AI client.

Taking a tool off the list reduces the chance of it being called by mistake. It does not reduce what that account can do. To genuinely limit scope you work on the account and its permissions in the source system, not on the list the model sees.

Summary#

Every connected tool takes up room and attention on every request, used or not. At a dozen that does not matter. At several dozen it does, and it shows first as worse answers, only later on the invoice.

The signal is not the count but the behaviour: similar operations confused, slower answers to the same questions, tasks breaking into attempts. Start with sets per role — that costs one decision. Split across servers if the areas of the business are clearly separate. Reach for tool search when the list grows faster than anyone can tidy it.

The wider context — how a model gets connected to company data in the first place, and whose permissions it runs with — is in the guide on connecting AI to company data.