Defining use cases

Each time a request is seen by the semantic router, it identifies in which use case that request belongs. It does this by creating an embedding from the request and comparing that embedding against the centers of previously discovered use cases. No LLM is used to detect clusters. The comparison is vector arithmetic, so it incurs no token cost.

If a request is seen that identifies its own new use case (it is far away from any of the existing centers of use case groups), then the use case boundaries are re-evaluated. To keep clustering fast, the gateway router does not assign or infer labels for use cases. Humans look at the exemplars of a given use case and use those to decide on labels.

Take a look at the following list of representative prompt inputs from users of an online store application. The store has five identified use cases and the router matches each incoming request to only one of them.

The match is on meaning, so requests that share no keywords can still reach the same use case. "What did I buy in March?" and "Cancel the order I placed yesterday morning." have no words in common, yet both map to the orders use case. Remember that this label is only useful for humans. The router does not know what "orders" means beyond it being a label humans have supplied to one of its identified use case clusters.

If a request can potentially belong to multiple categories ("Will this arrive before Friday and are there any more in stock?"), use an autonomous or orchestrator agent to answer them as two separate questions against different models.

At first glance, tools and use cases might seem similar since both require some kind of meaning to be derived from the incoming request. Use cases are a much broader scope than tools, and a tool can be used within multiple use cases.

Consider the shipping check use case. Given the exemplars in this use case, you might expect the following tools to be used:

Here track-package and shipping-events are tools that clearly belong within the use case of shipping check. However, the calendar tool is far more general purpose and the majority of prompts in many use cases are likely to use this tool ("I last complained about my order on Monday").

Prompts, use cases, and tools all exist at different levels of the hierarchy. Exemplar prompts are individual requests, a use case is the cluster they fall into, and a tool is a capability that one or more prompts within use cases draw on.