An AI router

One of the biggest opportunities in software today is to become the router. In simple terms, a “router” is like your brain — it takes a request and decides what to do next.

This concept isn’t new: integrations have long been a value generator for software businesses. Two examples:

  • Zapier enables anyone to connect different apps. They’ve built a very large, valuable business on “if X, do Y”.
  • Mulesoft, now part of Salesforce, helps organisations connect different pieces of technology together.

The demand for this will only go up with AI. Our value from AI today is based on its ability to create: text, audio, video or image. To get that value, you still need to be prescriptive about what you want. You can ask ChatGPT to give you a recipe for a salad with lettuce, walnuts and cheese, or write an itinerary for Bali.

Whilst that is incredibly valuable, it does require you be explicit in what you want.

These are my thoughts on how AI will progress and why being the router will be increasingly powerful.

The progression

Beyond generating content, the obvious next step is for AI to start taking actions. You can see the early signs of this with ChatGPT plugins.


For plugins to work well, establishing intent, i.e. figuring out what the user wants to do, is critical.

I asked ChatGPT how it decides what plugin to use. At a high-level, it comes down to figuring out the intent from the user, matching this to the list of plugins that the user has enabled, prioritising and then picking one.


In addition to choosing the right plugin, you also need to figure out what action to take within that plugin. If you determined that the groceries plugin is the right one, you still need to figure out whether you need to add items to the basket or place the order.

Once we’re able to take actions with AI, the next step is for AI to help us passively. You tell it to research a topic and it does it while you’re sleeping. It determines that milk is running low in your house and prompts you to order it. It reminds you that you have a dinner booked with a friend on the weekend, and provides 5 restaurants to choose from.

By the time we get here, we’ll have specialised models to do a variety of tasks. For example, a model specifically designed to answer medical queries. If you believe any version of this, you will need a router on top to decide where to send the request to.

The router becomes critical at every step.


What makes a good router

Large LLMs like OpenAI, Google’s Bard or Anthropic’s Claude will capture a major part of this.

But there is room, lots of room. Here’s what makes a good router:

  1. Latency: If you are making decisions at each part of the user’s journey, you will want to be the fastest at it. You are the bottleneck for the entire user experience, and it compounds at every step. This means that you may not want a model as large as OpenAI.
  2. Cost: You are going to have to make many of these decisions, which means your costs need to be low.
  3. Vertical specific: we will see routers that specific to verticals. Within a vertical, the set of actions you need to take is limited. By constraining your set of actions, you can provide an accurate response.
  4. Reinforcement learning: for your router to be the best in the world, you want it to learn continuously. You need a dataset with three components: a question or request (e.g. make me a website), intent (e.g. code generation) and feedback (e.g. this was good, that was bad). With this, you end up becoming the best in the world at figuring out what the user wants, i.e. being the router.

Who’s leading the charge?

In addition to the ChatGPT’s plugins, these are the other candidates competing to become the router; most of these are horizontal.

Every LLM

As above, every single LLM is competing to become the router.

For many use cases, this may simply not suffice because it’s either too slow, too broad or not accurate enough. And then, there’s the problem of multiple LLMs. What if Bard is better at writing code, ChatGPT is better at reasoning and Claude is better at writing copy? Someone’s going to build software that routes requests to each of those LLMs based on your question.


Zapier’s new AI based automation allows you to simply describe what you want to do, and then makes it happen. With this product, they will start collecting data on: a) what a user wants to do, and b) the recommended action.


As this compounds, they are going to have incredible data on intent, proposed action and feedback on that action: the reinforcement learning loop.



Langchain is an open source project that has taken off like a rocket ship. It allows you easily set up AI tools. For example, you can set up a tool that allows you chat with any document or knowledge base. You can set up a tool that scrapes websites or does a different task.

I don’t think Langchain is a router. It’s a tool that allows you to easily set up a router. For now, LangChain is not building a dataset on question, intent and action. Until it does, it won’t be as good as providers like Zapier.

They are much more focussed on helping people build the tools to get something done, which is equally valuable.

Every enterprise with data

Incumbents with data on questions and intent are sitting on a gold mine.

If you have the ability to map what a user asked, and what action was taken, you can use this to become the “router” for your domain.

The challenge here is that you need enough “coverage”. If your company sells sales software, you likely only have this data for sales. In my opinion, this is less useful than having all the data for a specific domain like Finance.

The best example I can provide is Bloomberg. For years, they have collected data on what customers are looking for and have (hopefully) recorded when they found it. You can use this data to build the router for Finance, and they have shown signs of wanting to do so when they shipped BloombergGPT.

To close

It’s going to be incredibly valuable for you to become the router for your vertical. If you do end up becoming the router, you have the luxury of choosing between two valuable options. The first is being the integrator and letting other businesses build on top of you. The second, is to become the router and the action taker.