OpenRouter makes the case for an agent execution stack

OpenRouter shows why an agent execution stack must choose where requests run, not merely which model answers them. Alex Atallah told woshipm that GLM is run through inference providers such as Fireworks and Together rather than hyperscale clouds. The same model behaves materially differently across providers, he said, because provider optimizations affect speed. They also alter price and output quality. For large models, those changes can arrive as often as every five minutes.

OpenRouter shifts more traffic when it detects an improvement. That pattern frames the larger problem: agents need controlled routes. They also need bounded actions, usable documentation, and measurement across the execution path. woshipm's author makes a parallel case for customer service, arguing that request complexity should determine the processing path rather than a more advanced Agent framework.

Route requests before choosing a model

Choosing a model is an operational decision that can change after deployment. According to woshipm, Alex Atallah says users often run GLM through inference providers including Fireworks and Together rather than hyperscale clouds. The provider is part of the result: its inference optimizations can alter speed and price. They can also change output quality even when the underlying model is the same.

For large models, those provider differences can shift as often as every five minutes, Atallah says. A fixed model choice can therefore become stale quickly.

OpenRouter's approach is to route more traffic toward a provider when its detected quality improves. It can also respond when speed or price improves. That frames routing as continuous selection, with fallback paths available when a provider's performance changes. Atallah also argues that enterprises will inevitably use multiple models, since different training data and methods produce different capabilities and ideas. The relevant question is not which model wins permanently, but which current route fits the request.

woshipm's author makes the same case in customer service: requests with different complexity should take different processing paths. A Router-Agent can identify a specialized domain and pass context to a vertical Agent with separate knowledge and tools. It also has its own conversation flow and evaluation. Those boundaries only hold when intents are distinguishable and knowledge is relatively independent. Tool permissions must also differ. Evaluation must then cover routing and retrieval. It must also cover Agent behavior. Execution and business outcomes must be assessed as well.

Keep language models away from business-state changes

An agent can interpret a customer's request without being the component that changes the customer's account. That distinction matters when a task can issue refunds, alter tickets, cancel orders, file loss reports, or change configuration. According to woshipm, Amazon MARCO separates information queries from operational execution and out-of-scope requests: queries go to RAG, while execution goes to multi-Agent orchestration.

The model's role is to understand natural language. It should request missing details and choose a next step. Deterministic APIs should perform the actual state change. woshipm describes those API sequences as tools, wrapped so the model can invoke them without inventing business logic. Before a tool runs, guardrails should verify the output format. They should confirm that the requested function exists, trace parameters back to the conversation, and apply domain rules. Permissions belong here too.

Write operations need idempotent handling. woshipm recommends a unique request identifier for every write, followed by a result query after a timeout before any retry.

That pattern addresses duplicate execution, but it also makes partial success visible rather than letting an agent guess. woshipm identifies six breakpoints in an execution chain. These include mistaken intent classification and hallucinated functions. They also include invented parameters, permission overreach, duplicate actions, and partial completion. The remedy is not to grant broader access; it is to expose current state through controlled queries and escalate high-risk cases to people. As woshipm's author argues, a higher handoff rate can signal that the system is correctly refusing unsafe autonomy.

Isolation can help contain code that agents need to run, but it does not replace these controls. ifanr notes that Microsoft's Execution Containers remain in early preview.

A controlled execution stack therefore keeps the language model at the decision boundary. Tools govern business-state changes. Policies govern them as well, while humans provide oversight.

Local compute is a privacy tier, not a complete stack

Local compute changes where sensitive context can live; it does not remove the need to choose providers or manage external capabilities. AMD's Personal AI framing keeps tasks on-device where possible, while reserving cloud and data-center resources for larger-scale work, according to ifanr. That is a privacy tier within a hybrid system.

Hardware capacity makes that tier less theoretical. AMD says its Ryzen AI 400, based on Gorgon Point, can run 24 billion-parameter models locally. It also says a Strix Halo system with 128GB of unified memory can run 200 billion-parameter models, while Ryzen AI Max 400 raises unified memory to 192GB and supports local models up to 300 billion parameters. ifanr reports that AMD demonstrated Zhipu AI's 320 billion-parameter GLM-5.3-Flash on Gorgon Halo.

The economic case can be direct. AMD estimated that 10 million output tokens from Claude Sonnet 5 in the cloud cost about 90 euros, compared with zero for local execution.

But local capacity is not a reason to freeze model choice. Alex Atallah told woshipm that inference providers are severely supply-constrained and almost always lack enough compute capacity. He also says identical models can differ materially between providers because deployment optimizations change speed and price. They can also affect output quality. OpenRouter responds by directing more traffic to a provider when it detects improvement in any of those dimensions.

A practical design therefore keeps proprietary context and suitable recurring work local, then routes outward when a task needs larger-scale capacity or a stronger available deployment. Atallah expects enterprises to put proprietary models into core workflows while retaining external models for model selection and supplementary tasks. The boundary should follow the work's sensitivity and requirements, rather than treating local execution as a complete replacement for cloud routing.

Publish documentation that an agent can act on

Documentation becomes part of an agent's decision path when the agent must determine, before integration work, whether a product fits the task. According to geekpark, Addy Osmani cited a study of HTTP behavior across nine mainstream coding agents: they could decide to use or ignore a site after one GET request, within 400 milliseconds. An opaque documentation homepage is therefore a compatibility problem, not merely a poor reading experience.

Put the essential answer at the top. Osmani recommends that the first 500 tokens explain what the product is. They should then explain what it can do and how to begin. Markdown helps because, geekpark reports, equivalent HTML carries extra div tags. It also carries CSS class names, ARIA attributes, and inline styles. Those tokens burden the machine's initial scan.

An llms.txt file at the domain root gives that scan a map. It can provide a Markdown list of documentation pages. The list can describe their coverage and give approximate token counts. Osmani recommends keeping it below 5,000 tokens. A skill.md can state core capabilities and required inputs or credentials. It can also identify constraints such as rate limits and token expiration, then point to key documentation. This is a compact contract, not promotional copy.

Availability matters as much as format. Agents may check robots.txt before crawling and skip a site when their user agent is blocked; lightweight HTTP clients can also receive an empty page from documentation that depends on JavaScript. geekpark notes that Anthropic and Cloudflare offer a Copy for AI feature supplying clean Markdown. Osmani's sequence is to audit robots.txt. Then add llms.txt and expose token counts for measurement. Write skill.md for the three most important APIs before adding that copy feature.

Four layers of a controlled agent execution stack

Choose the execution path by risk, ambiguity and state change

• The request is a repeatable question with an existing answer, such as business hours, membership benefits, refund policies or feature instructions. Use an answering path built from traditional NLP, rules, small models and RAG rather than an action-capable agent. Measure knowledge hit rate, answer accuracy, version validity and handoff-to-human rate; maintain document splitting, question-answer associations, invalid-content removal, promotional-rule time ranges and user access controls.
• The request needs contextual judgment but does not yet change business state-for example, choosing the appropriate product guide and gathering usage scenario, dimensions and budget. Route it to a specialized vertical agent only where intents are distinguishable, knowledge is relatively independent and tool permissions differ. Keep each vertical agent's knowledge, tools, conversation flow and evaluation set separate; use a DAG when multiple capabilities must collaborate through data-dependent steps.
• The request changes real business state, including refunds, ticket changes, loss reports, order cancellations or configuration changes. Treat the agent as an untrusted operator around deterministic tools. Use LLMs for language understanding, missing-information collection and next-step selection, but encapsulate deterministic API sequences as tools. Apply checks for output format, function existence, conversational provenance of parameters and domain-rule validity; give every write operation a unique request identifier and query after a timeout before retrying.
• A workflow spans multiple model providers, or the same model is available from different inference providers. Use routing rather than assuming a model name guarantees a fixed service level. Provider optimizations can alter speed, price and output quality, and performance for large models can change as often as every five minutes. For core workflows, the cited view is to place proprietary models in the core while using external models for model selection and supplementary tasks.
• Sensitive personal context can remain local, or cloud token volume makes continuous remote inference expensive. Prefer a local-first split where practical: handle local tasks on-device and reserve cloud and data centers for larger-scale requirements. AMD says Ryzen AI 400 series processors can run 24 billion-parameter models locally, while the Ryzen AI Max 400 series can run local models with up to 300 billion parameters. Do not infer that local execution eliminates the need for policy controls, isolated execution or human handoff for high-risk cases.

Measure the whole execution path

Model benchmarks isolate the component least able to describe an agent's operational value. A mixed local-and-cloud stack should instead be tested from request classification through retrieval. It should then be tested for tool invocation. A person should review completion. The question is whether the chosen route produced a valid business result at an acceptable cost.

Cost needs to be measured at the provider and workload level. According to woshipm, token prices have fallen about 90% over the past 18 months, while GPT-5.6 Luna's OpenRouter price fell 10 times within two weeks as usage grew 13 times. Those shifts can change a routing decision quickly. ifanr reports AMD's estimate that 93% of enterprises are overspending AI budgets; its cloud example of 15 million output tokens per active user per day costs about 300 euros per day.

Cheap inference is not proof of a safe execution path.

The scorecard should separate routing accuracy from knowledge hit rate. It should also measure answer accuracy and version validity, which woshipm identifies for answering systems. It should record tool-call safety. Permission failures should be recorded separately. The scorecard should also track duplicate actions, partial successes, and the rate at which a person must take over. woshipm identifies six failure points in task execution, including intent misclassification. Other failure points include invented functions or parameters. They also include permission overreach, duplicate execution, and partial success.

Recovery deserves its own test cases. woshipm recommends a unique request identifier on every write operation, followed by a result query after a timeout before any retry. That measures whether the stack can recover without repeating a consequential action. Compare cloud spending with local execution where appropriate: ifanr cites AMD's estimate of about 90 euros to run 10 million output tokens on Claude Sonnet 5 in the cloud, versus zero for local execution. The final measure is business outcome. Execution quality and human-review demand should also be measured.

Put deterministic business actions behind ordinary tools, leaving natural-language interpretation to the model. For multi-step work with data dependencies, use a DAG rather than trusting chat history to preserve state. woshipm identifies intent misclassification as a failure point. Parameter hallucination can also cause failure, as can permission overreach. Duplicate execution and partial success are additional failure points; log each transition so they can be inspected.

Require tool-call checks for format and function existence. Checks should also verify parameter provenance and domain rules. Give every write operation a unique request identifier, then query its result after a timeout before retrying. Measure routing outcomes. Measure execution outcomes and business outcomes. Do not force down human handoff when a high-risk case needs a person: according to woshipm, that transfer can be the correct result.

For readers outside China

• Availability: Geographic availability, supported countries and service eligibility are not disclosed in sources. The source material identifies OpenRouter as a multi-provider inference layer, Microsoft Execution Containers as an early preview, and Project Zenith as first available on Ryzen AI Halo devices; it does not cover purchase channels or availability outside China.
• Pricing: The only directly sourced platform pricing is OpenRouter's pay-as-you-go 5.5% fee. Enterprise plans based on committed spending do not charge that fee on committed spending, and the relevant fee is waived when an enterprise brings its own inference provider or API key. AMD stated that processing 15 million output tokens per active user per day in the cloud costs about 300 euros per day, and that 10 million output tokens on Claude Sonnet 5 in the cloud costs about 90 euros while local execution costs zero. Hardware prices are not disclosed in sources.
• Closest Western equivalents: OpenRouter is the clearest referenced example of a multi-provider model-routing layer.; Anthropic and Cloudflare documentation sites are cited as having a "Copy for AI" feature that provides clean Markdown documentation.; Microsoft's Execution Containers are a referenced example of isolated execution environments for agents, though they remain in early preview.; AEO is presented as an agent-facing analogue to SEO: make machine-readable product capabilities and documentation easy for agents to discover and use.
• Data residency: The local-compute argument is framed as keeping sensitive data and personal context on-device where possible, with cloud and data centers used for larger-scale requirements. This is an architectural direction, not a disclosed compliance, residency or retention guarantee. The sources do not say where OpenRouter providers process data, whether prompts are retained, or which regions are available.

Sources

• ifanr DeepSeek、千问、智谱轮番登场，PC 厂商终于等到了它们的弹药 https://ifanr.com/1678938
• woshipm 智能客服 Agent 架构怎么选：从 RAG、Router 到多 Agent，什么时候该用哪一种？ https://woshipm.com/ai/6448538.html
• geekpark 判断用不用你的软件，Agent 只需 500 个 Token https://geekpark.net/news/369960
• woshipm 对话OpenRouter CEO：Harness 正在取代超级 App，未来所有软件都只是 Agent 的后台工具 https://woshipm.com/share/6461267.html