AiKA Modes for Low-Latency Developer Portals | Algoramming

You spin up a modern AI coding assistant to help refactor a complex, multi-service backend. You expect deep architectural insights, but instead, the assistant spends eighty percent of its time reading configuration files. It reads the same deployment YAML over and over. It consumes thousands of tokens just to ingest boilerplate patterns. When the final bill arrives, you realize you just paid premium frontier model rates for basic file reading.

We have all been there. At Algoramming, we manage these token costs daily. When we design advanced digital platforms for our clients, efficiency is everything. We cannot let valuable engineering budgets melt away into unoptimized API calls. That is why a recent engineering breakthrough from Spotify caught our attention. In September 2026, Spotify platform engineers revealed they slashed Claude Code token consumption by an average of ninety percent.

They did not do this by restricting developer access or writing less code. Instead, they routed routine, input-heavy tasks to cheaper models using declarative agents called AiKA Modes within Spotify Portal, their managed internal developer portal built on Backstage. This architecture is a masterclass in modern agentic design. It solves a pain point that every scaling engineering team faces today. If you want to optimize your developer experience without breaking the bank, this routing pattern is the way forward. In this guide, we will walk you through exactly how to implement this architecture for your own developer portals.

How do you implement AiKA Modes for low-latency developer portals?

To implement AiKA Modes for low-latency developer portals, configure lightweight, declarative agents on Spotify Portal's serverless runtime, then route routine file reads to cheaper models using the open-source shunt plugin. This intercepts large-file ingestion via PreToolUse hooks, keeping frontier models focused on reasoning.

This routing strategy relies on three main layers: declarative agent modes, client-side tool interception, and the Model Context Protocol. By setting a file-size threshold (typically 350 lines), you prevent expensive frontier models from wasting tokens on large reads. The cheaper worker model processes the file, generates a concise summary, and feeds it back to the primary agent. This cuts token consumption by 82% to 94% on bulk reads, while keeping latency low. In our own work building developer platforms, we have seen that this hybrid approach is essential for scaling AI tools across large engineering organizations. It ensures that developers get the speed they need without the runaway costs.

The Token Inflation Problem in AI-Assisted Engineering

Tokens cost money. In fact, they cost a lot of money when scaled across hundreds of developers. Many engineering leaders are beginning to realize that the seat license for an AI tool is not what hurts. The real pain comes from the underlying token usage. By 2028, some industry forecasts suggest that AI coding costs could surpass the average developer's salary if left unmanaged. Already, a quarter of engineering teams spend between two hundred and five hundred dollars per developer every month on tokens. In some high-volume environments, that figure climbs past two thousand dollars.

The core of the problem is that AI coding agents spend most of their time doing things that do not require complex reasoning. They read files. They search directories. They look at test files to copy existing assertions. They check build configurations. Every time an agent opens a five-hundred-line file to answer a simple question, thousands of tokens disappear. Worse, because these agents maintain a running conversation history, they read those same files again and again in subsequent turns. This is known as prefix re-reading, and it can easily account for over sixty percent of an unoptimized AI billing statement.

We are feeding frontier models that are wildly overqualified for basic tasks. You do not need a multi-billion-parameter frontier model to read a standard configuration file or generate boilerplate code. You need a fast, cheap worker model. If you can route the routine work to a lower-cost model while saving the expensive model for high-value reasoning, you can slash your expenses. The Spotify team realized this and built a system that enforces this division of labor. Let's look at how their internal developer portal makes this possible.

Monorepos make this problem even worse. When a developer works inside a massive repository, the AI agent often has to traverse dozens of files just to understand the context of a single method. If the agent ingests all of those files into its active context window, the token count explodes instantly. This is not just a financial issue; it is also a performance bottleneck. Large context windows slow down response times, leading to a sluggish developer experience. Developers want immediate answers, not a thirty-second wait while a frontier model digests a megabyte of source code. This is where the concept of intelligent routing becomes critical.

Introducing Spotify Portal and the AiKA Assistant

To understand how this routing works, we must first look at the platform that hosts it. Spotify is famous for its focus on developer experience. Years ago, they created Backstage, an open-source framework for building internal developer portals. Today, Backstage is the industry standard for centralizing software catalogs, documentation, and API schemas. Spotify Portal is their managed, enterprise version of Backstage. It provides a hosted, fully supported portal that eliminates the overhead of maintaining the infrastructure yourself.

One of the standout features of Spotify Portal is AiKA, which stands for AI Knowledge Assistant. AiKA is a portal-native AI assistant designed to help developers find information across their organization's ecosystem. Because it is integrated directly into the portal, AiKA has access to the software catalog, technical documentation, and ownership metadata. Instead of searching through multiple slack channels, wikis, and repositories, a developer can simply ask AiKA where a service is hosted or who owns a particular API.

When we build modern digital products, we often recommend centralizing knowledge this way. In our web application design & development projects, we have seen how much time developers waste just trying to find the right documentation. AiKA solves this by using retrieval-augmented generation to provide context-aware answers. It acts like a knowledgeable teammate who never sleeps. But Spotify did not stop at simple search. They realized that by exposing the portal's catalog and tools to AI agents, they could create a powerful automation engine. This led to the development of AiKA Modes.

At KubeCon Europe, Spotify platform engineers shared that AiKA has cut the time required to resolve internal support requests by forty-seven percent. This is a massive win for platform engineering teams who are constantly bogged down by repetitive questions. By bringing AI directly into the portal, they created a single source of truth for both human engineers and AI agents. Now, with the release of AiKA Modes, they have made this assistant highly customizable, allowing teams to build purpose-built agents for specific workflows.

What Are AiKA Modes?

An AiKA Mode is a declarative, lightweight agent that runs on an ephemeral, serverless runtime. You can think of it as AWS Lambda, but specifically designed for AI agents. Instead of writing complex backend code, managing API keys, or provisioning long-running servers, platform engineers can define an agent's behavior using a simple YAML configuration file.

Each mode defines three main components: instructions, tools, and models. The instructions tell the agent exactly what it should do, what its boundaries are, and how it should format its responses. The tools connect the agent to external systems, such as the software catalog, search endpoints, or custom APIs. The model field specifies which underlying large language model should power the agent. This means you are not locked into a single provider. You can use Anthropic's models for complex reasoning, and switch to Google's Gemini or OpenAI's models for faster, cheaper tasks.

Modes can be public, meaning they are shared with the entire engineering organization, or private to a specific developer or team. They are accessible through the Portal user interface, a command-line interface, or an API. This flexibility allows you to integrate them directly into your existing development workflows, including local terminal sessions and integrated development environments.

In our analysis of how Spotify Portal AiKA Modes cut Claude costs, we saw that the true power of this architecture lies in its ability to delegate work. Instead of using a single, expensive frontier model for an entire development session, you can use a high-end model as the orchestrator and spin up cheap, specialized AiKA Modes as workers. This is the foundation of the shunt pattern.

This declarative nature is what makes them so powerful. Traditionally, building a custom AI agent required writing Python or TypeScript code. With AiKA Modes, you write a few lines of configuration, and the portal handles the rest, ensuring that all agents respect the organization's security and access control policies.

The Shunt Pattern: Intercepting the Frontier Model

The shunt pattern is a clever way to implement model routing without modifying the underlying AI coding assistant. Instead of relying on developers to manually switch models, or hoping that the assistant will follow routing instructions in a prompt, the shunt pattern enforces routing at the tool execution level.

The implementation uses an open-source Claude Code plugin called shunt, which is available in the spotify/portal-ai-plugins repository. Shunt operates by registering PreToolUse hooks within the coding assistant's runtime environment. These hooks intercept tool calls before they are executed. Specifically, when Claude Code attempts to read a file, the check-file-size hook intercepts the request and inspects the target file's size.

If the file exceeds a configurable threshold, which defaults to 350 lines of code, the hook blocks the read operation. Instead of returning the file contents to Claude, the hook returns a message explaining that the file is too large and directs Claude to use a specialized bulk-reader skill. This skill invokes a lightweight AiKA Mode running on the Portal CLI.

The bulk-reader mode is powered by a cheaper model, such as Google's Gemini 2.5 Flash. It reads the large file, extracts the relevant information, and returns a highly condensed summary to Claude. Because the summary is a fraction of the size of the original file, Claude consumes far fewer tokens. This prevents the frontier model's context window from being flooded with boilerplate, saving up to ninety percent of token costs on large reads.

This approach is highly effective because it targets the most expensive part of the agentic workflow. In a typical development session, an agent might read a five-hundred-line file ten times as it iterates on a solution. By shunting those reads to a model that costs a fraction of the price, you protect your budget while keeping the primary agent focused on the difficult reasoning tasks.

The plugin also intercepts standard bash commands like cat, head, tail, less, and more when they are run on large files. This ensures that developers cannot accidentally bypass the routing logic by running shell commands instead of using the assistant's built-in read tools. However, targeted piped commands, such as running a grep search on a file, are allowed to pass through, as they only return a few lines of relevant context. This granular control is what makes the shunt pattern so elegant. It blocks the expensive, low-value operations while allowing targeted, high-value queries to execute without friction.

Declarative Configuration of Custom Modes

To set up this architecture, you must configure the custom AiKA Modes inside your Spotify Portal instance. This is done declaratively, which means you define the desired state in a configuration file, and the platform handles the underlying orchestration. You do not need to write any code to provision runtimes or manage API endpoints.

Let's look at how to define the two primary modes used in this routing pattern: the bulk-reader and the code-writer. In your configuration, you specify a unique name, a description, and the system instructions that guide the agent's behavior. You also select the target model, set the temperature, and attach any required tools.

For the bulk-reader, the instructions should tell the agent that it is a specialized