How to Build an AI Agent Into Your App: Architecture and Cost

Codixus Team01/07/2026

You want an AI feature that does more than answer one question. It should read the user's data, call your API, take an action, and return a result. That is an agent, and building one into a shipping mobile app is a different job from wiring up a chatbot. This is the build-and-cost guide: the architecture you deploy, the loop underneath it, and how to put a real number on the monthly bill before you commit.

Two things this post is not. It is not a pep talk about why agents matter, and not a lecture on prompt design, because we wrote those. If your last attempt stalled, read why your AI agent project failed first. If you are wrestling with what to actually feed the model, context engineering vs prompt engineering covers that ground. Here we stay on the build.

What an in-app agent actually is

Strip away the marketing and an agent is a small idea: a language model that uses tools in a loop. Anthropic, whose team ships agents in production, defines them plainly as "systems where LLMs dynamically direct their own processes and tool usage," and describes them as "just LLMs using tools based on environmental feedback in a loop."

Compare that to a plain chat completion: text in, text out, one turn, done. In an agent, the model can decide it needs more information, ask your code to run a function, read the result, and decide again. That decide-and-act cycle is the trick, and it runs two to ten times per request depending on the task. Each pass is a full model call you pay for.

Anthropic's team is also clear that you should "add multi-step agentic systems only when simpler solutions fall short." If one model call with the right context answers the question, ship that, and reach for a loop only when the task truly needs the model to gather information it lacks up front.

The reference architecture

Here is the shape that works, and the one we build on: the phone talks to your backend, and only your backend talks to the model and the tools. The client, your Expo app, sends the user's message and context to an endpoint you own. Your backend, a Bun service in our stack, holds the system prompt, defines the tools, runs the loop, executes each tool call, and streams the answer back. The model provider, Claude or OpenAI, is the reasoning layer. Supabase holds state, auth, and any data your tools read or write.

Why the API key never ships in the app

Put your model API key in the mobile binary and you have handed it out: anyone can pull strings from a shipped app, and a leaked key means a stranger runs up your bill. The key lives on the server, full stop. The app authenticates the user to your backend, and the backend authenticates itself to the model. That also gives you one choke point to enforce rate limits, log usage per user, and swap models without an app update.

The orchestration layer

The backend owns the loop: send the conversation and tool definitions to the model, receive a tool request, run the matching function against your database or a third-party API, feed the result back, and repeat until the model returns a final answer. Server-side, you can change a tool, tighten a guardrail, or move to a cheaper model the same afternoon, with no App Store review in the path.

Want a second set of eyes on your agent architecture before you write code? Book a 30-min studio call and we will map the loop, the tools, and the cost with you.

Tool calling: how the loop runs in code

Tools are the hands of the agent. You describe a function to the model as a name, a plain-language description, and a JSON schema for its inputs, and the model decides which one fits. OpenAI lays the flow out in five steps: send the request with the tools the model could call, receive a tool call, run the code on your side, send a second request with the tool output, and receive the final response or another tool call.

Claude's tool use works the same way. The model returns a stop reason of tool_use and one or more tool_use blocks, and per the docs "your code executes the operation and sends back a tool_result," which the model uses to continue. The wording differs by provider, the mechanic is identical. Write clear tool descriptions, because the description is what the model reads to decide. A vague one is the most common reason an agent calls the wrong tool, or none at all.

Streaming so it does not feel slow

An agent that thinks, calls two tools, and writes a paragraph takes several seconds. If the screen sits blank that whole time, users assume it broke. Stream instead: send tokens to the app as the model produces them, and show the intermediate steps while tools run. The total time is the same, but the perceived wait is a fraction of it. On the client it is a streamed HTTP response your Expo app reads chunk by chunk; on the backend, a passthrough of the model's stream plus your own status events between tool calls.

Guardrails and eval

An agent that can act needs limits. Two areas matter most.

First, treat tool output as untrusted. Anthropic's docs are blunt: tool results "often carry content from sources outside your control: web pages, inbound email, user uploads, third-party APIs," and an attacker who controls that content can embed instructions that try to hijack the model, a technique called indirect prompt injection. Keep that content inside tool_result blocks, never in the system prompt, and never let a tool take an irreversible action, like charging a card or deleting data, without a confirmation the user sees.

Second, build an eval set before you scale. Collect twenty to fifty real requests, write down the correct outcome for each, and re-run them every time you change the prompt, a tool, or the model. Without this you are guessing whether a change helped or quietly broke something. This is where the two posts linked above earn their keep: most failures we see are not model failures, they are context and scope failures an eval set catches early.

The real cost drivers

Model APIs bill per token: input (everything you send, from the system prompt and tool definitions to the history and every tool result) and output (what the model writes). Four things move your bill.

Model choice. This is the biggest lever. Provider pricing is tiered: a small, fast model can cost a fraction of a flagship one for the same tokens. Anthropic's published rates show its lightweight Haiku tier priced well under its top Opus tier, and the pattern holds across providers. Use the cheap model for routing and simple steps, and reserve the expensive one for the reasoning that needs it.
Token volume. An agent loop re-sends the growing conversation on every pass, so a five-step task can send your system prompt and history five times over. Trim context, summarize old turns, and do not stuff your whole database into the prompt.
Caching. If your system prompt and tool definitions are stable, prompt caching lets the provider reuse them instead of reprocessing them each call. Anthropic prices cache reads at 0.1x the base input rate, roughly a tenth of a fresh read, and other providers discount cached input similarly. For an agent with a large fixed system prompt, that is a real cut.
Retries. Every failed call you retry, and every tool error the model recovers from, is more tokens. A factor of 1.2 to 1.5 over your happy-path estimate is realistic.

A back-of-envelope monthly estimate

You can size the bill without a spreadsheet. Estimate the tokens one full request consumes, input plus output, across every pass of the loop. Multiply by your model's input and output rates, then by requests per user per month, then by active users. Add a retry factor, and subtract what caching saves on the fixed part of your prompt.

Concretely: if a task runs three passes and each averages a few thousand input tokens and a few hundred output tokens, one completed request lands on the order of ten-thousand-plus tokens. At a mid-tier rate that is fractions of a cent per request; at a flagship rate, several times that. The formula matters more than any single number, because published prices change often. Price it against today's rates, and design so you can drop to a cheaper model for the simple steps.

FAQ

What is the difference between an AI agent and a chatbot?

A chatbot answers in one turn: text in, text out. An agent runs a loop, deciding when to call tools, read the results, and act, before it answers. It can look up data, hit your API, and take steps on the user's behalf; the chatbot only talks. Most agent features that fail are really chatbots that needed one tool call, or chatbots dressed up as agents when they should have stayed simple.

Should I put the model API key in my mobile app?

No. A key in a shipped binary can be extracted and used to run up charges on your account. Route every model call through a backend you control, authenticate the user to that backend, and keep the key server-side. That is also the only clean place to rate-limit and log usage per user.

Claude or OpenAI: which model should I use?

Both support the same tool-calling loop, so you can build against one and swap later. The better question is which tier per step. Use a small, cheap model for routing and simple extraction, and a stronger model only for the hard reasoning. Many production agents mix providers and tiers within a single request to keep quality up and cost down.

How long does it take to build one?

A focused in-app agent with a few tools, streaming, and guardrails is a matter of weeks, not months, on a modern stack. The time goes into tool design, the eval set, and the guardrails, not the model call itself. Scope it to one clear job, ship it, and expand once you can measure it works.

Build it right the first time

An in-app agent is a backend that runs a model in a loop, a handful of well-described tools, streaming, guardrails, and a cost model you understand before you scale. Get those five right and the rest is iteration. That is exactly the work we do in AI mobile app development, on the same stack we ship our own apps on.

Sources

Need this built? Book a 30-min studio call.