Calling a model API is a weekend. Making it stream instantly, survive a reconnect halfway through an answer, and not cost more every single turn is the actual build. Agencies quote $50,000 to $250,000. We build a working core in one week for $5,000.
Not the screens. These are the parts that decide whether an AI chat product works or falls over:
Time to first token. Nobody waits at a blank screen. What matters is how fast the first word appears, not how fast the whole answer finishes.
The stream has to survive the network. Answers take tens of seconds. Connections drop and servers redeploy, and a user should not lose half an answer because of it.
Context cost grows every turn. Replaying the whole conversation into the model on each turn means cost and latency climb with every message, until eventually it does not fit at all.
Compute is the scarce resource. Inference is the expensive part. With no admission control, a handful of heavy users starve everyone else.
Cancellation is real money. A generation nobody is reading keeps burning compute until something explicitly stops it.
The specific decisions that make an AI chat product hold up, and that the obvious version gets wrong:
One-way streaming, not a duplex socket. Tokens only flow one direction, so server-sent events are the right transport. They reconnect natively and avoid the overhead of a channel nothing needs.
Decouple the worker from the connection. Tokens are published to a per-run stream any server can read, so the machine generating the answer need not be the one holding the user connection. A reconnect replays from the last token seen rather than starting the answer again.
The run is a first-class object. Every generation gets an ID and a lifecycle: queued, streaming, then done, cancelled or failed, with token counts attached. Cancellation, quotas and billing all hang off it.
Queue in front, batch on the worker. Requests queue and workers pull when they have capacity, running many conversations in a single batch. Generation is bound by memory bandwidth rather than raw compute, so batching is what turns one expensive worker into dozens of concurrent chats.
Cost-aware quotas, not request limits. A flat requests-per-minute cap cannot tell a one-line prompt from a 30,000-token one. Budgets are measured in tokens and weighted by tier, so heavy use degrades that user instead of the whole system.
A rolling summary plus a cached prefix. Older turns are compacted into a summary and the stable start of a conversation is cached, so a long chat stops getting more expensive with every message.
Explicit cancel, never a closed tab. Stopping is a signal on a control channel that the worker checks between batches. A dropped connection deliberately does not cancel, because the user is probably just reconnecting.
The gap
A studio quote for an AI chat product usually lands at $50,000 to $250,000 over three to nine months. We build a launchable core in one week for a flat $5,000, deployed on your own accounts, with every line of code yours to keep.
A working product you can put in front of real users:
Streaming chat with sub-second time to first token
Saved conversations, resumable, with context carried across turns
Reconnect mid-answer without losing the stream
A queued worker pool with backpressure in front of the model
Per-user token budgets, tiering and usage tracking
Stop generation, and a full message history
And what we would leave out. Being straight about this is the only way a fixed price means anything:
Training or hosting your own frontier model
Image, audio and voice input
Tool calling, browsing and custom agents, unless that is the product
Content moderation at scale
You are getting the core of the same problem the big product solved, scoped to something launchable. Not a clone of the finished article.
No. We build the product around a model you call through an API. Training a frontier model is a different business with a different budget, and almost nobody actually needs to.
Yes, retrieval over your content is a common addition. We scope it properly because the quality of the retrieval matters far more than which model you pick.
A free 20 minute call. We will tell you if an app like ChatGPT is a fit for one week, and what it takes.
Start here: what it costs to build an app in 2026, or how the week works.