Choosing the AI model: what it costs, how long it lasts, how easy it is to change.
The model is the part of an AI application that changes fastest: new versions come out, prices move, services are withdrawn. For the business, three things matter: the cost of the whole service, how long it will keep working and what it costs to change course. The result of a public ranking comes after that.
Start with the task
Which is the least expensive model that reliably meets this task's requirements?
I first establish minimum quality, response time, number of requests, privacy and continuity. Choosing a provider comes later. A model that excels overall may be unsuitable for a particular activity; a smaller one may be sufficient, if representative tests demonstrate it.
The organisational and economic assessment comes before this choice. Achieving technical answer quality requires separate development and verification work.
The cost of the whole service
The price of the model, which is paid according to use, is one part of the bill. Data preparation, infrastructure, retries, checks carried out by people, monitoring and maintenance also count. Your own server requires hardware and someone to run it; with light or irregular use, a pay-as-you-go service may be more economical.
Pay-as-you-go prices are quoted per token, the units of text a model reads and writes. The price list does not say how many tokens it takes to get a useful result: that depends on the task, the length of the documents supplied, the steps in the workflow, retries and corrections, and it varies from model to model. Some models "reason" before answering: that reasoning text is billed as output even when it is not shown, so the bill does not match what you read in the answer. A study of coding agents (eight models, real software-fixing tasks) found that different runs of the same task consumed tokens in amounts differing by up to 30 times, and that more tokens do not guarantee higher accuracy. No formula leads from the price per token to the cost of the result: it is measured case by case, with trials on real requests, and repeated when the model, data or workflow changes.
In McKinsey's May 2026 survey, 93% of the 75 qualified respondents reported exceeding their AI budget. The sample does not represent all companies, but draws attention to the costs of adoption at scale.
A case made public in October 2026: according to 404 Media, which saw an internal communication, since 1 October Accenture has paid $29 a month per employee for Claude; spending above that must be charged to a project or team. Some employees had been spending up to $1,000 a month. Accenture has not publicly explained the decision, but the case shows how much spending can vary from person to person with the same tool, and one possible response: tying the cost to the activity that generates it, where it can be compared with the result.
A workflow can route different activities to different models: more capability where the problem requires it, cheaper or specialised models for clearly defined steps. Each route must respect data constraints and pass the same quality checks.
Assess the choice to wait, too
In my judgement, non-adoption also merits assessment: a competitor might automate an activity or develop expertise before us. This is a strategic scenario to examine, not a prediction that applies to every business.
I propose active caution: experiment within a limited scope, measure value and costs, maintain skills and decide on your own evidence. A project can be postponed or rejected when the benefit does not justify the investment and risks.
External service, dedicated cloud or your own server
An "open" (open-weight) model can be downloaded and run on your own machines. That does not mean free of charge, free of conditions of use or necessarily economical; and the same model can also be used through an external service.
Epoch AI's May 2026 analysis finds that, since January of that year, the leading open models lagged the leading proprietary models by an average of about four months on its own index. This describes a period and a chosen measure: it does not say they are equivalent on every task.
Pay-as-you-go external service
Services such as OpenRouter, the OpenAI API or managed cloud provider APIs can offer straightforward access to models and scalable capacity. They may suit a project when its data can be used under the service's terms and reducing operational work matters.
Check costs, data retention and location, use for training, and the applicable agreements. With an intermediary such as OpenRouter, the model provider also matters, particularly if routing can change. Managed cloud APIs may offer different network and geographic options: the service's name alone says little about its privacy arrangements.
Dedicated cloud resources
Another option is to run a model on dedicated cloud resources, such as machines with dedicated graphics cards (GPUs) on Azure or AWS. This gives more control over the model, network and access, but means operating the model and paying for the infrastructure. Privacy and compliance still depend on configuration and applicable agreements.
Server on your premises
A model on your own infrastructure can give more control over the data's path, but requires hardware, updates, monitoring and maintenance. GPU memory limits which models will run; response time and capacity also depend on computing power, request length and concurrent users. Before choosing this route, measure response times and workload on the proposed hardware with representative tests.
The practical consequence: build the system so that the model can be changed when it pays to do so, without spending more on that freedom than the project justifies.
How long does it need to keep working?
With a model run on your own machines, a configuration can be retained while it meets requirements. A new model release does not require a migration; security updates and component maintenance remain necessary.
An external service has a different lifecycle: OpenAI and Anthropic publish the dates on which models are retired. Distinguish a mandatory migration, because a service disappears, from a beneficial one, because an alternative improves quality, costs or performance.
I therefore budget for reassessment and plan for continuity. Model independence is an investment: its value grows with the probability and cost of needing a replacement.
Don't stay tied to a provider
I keep the part that depends on the model provider separate from the rest of the application: the business's rules, permissions and data stay outside it. That way, replacing the model affects a limited part. It does not make it free: different models behave differently, and have to be tested again.
I would organise the system into parts with distinct responsibilities:
- 01Application and domain
Product rules, permissions and data that remain the application's responsibility.
- 02Workflow and tools
Steps, checks, information retrieval and permitted actions.
- 03Model adapter
Provider-specific formats, instructions, structured outputs and tool calls.
- 04Model and execution environment
The external service or a model hosted on controlled resources.
This is a design proposal, not a universal requirement. Adapters reduce dependencies, but models differ in behaviour: migration often requires changes to prompts, tools and context handling.
Changing model: only with tests
Before replacing a model, I compare the old one and the new one on the business's real requests and on the difficult cases, with criteria set in advance: answers that are correct and faithful to the sources, the task carried through, time and cost per operation, behaviour when something goes wrong. The tests must be repeated after every change to the model, data or workflow. A public ranking helps shortlist candidates; it does not replace testing on the application.
What to ask whoever proposes a model to you
- Why this model and not a less expensive one? With what tests on our cases?
- What is the monthly cost of the whole service, not just the model? Measured in trials on our cases or estimated from the price list?
- What is the monthly spending cap, and who decides what happens when it is exceeded?
- Until when does the provider guarantee this model, and what happens afterwards?
- What would it cost to move to another model or provider?
- Who repeats the tests when the model changes?
Supporting sources
These sources support the facts and evaluation criteria. The architecture and the comparison between adoption and waiting are my methodological proposals, rather than findings demonstrated for every project.
- Epoch AI — open/closed comparison (opens in a new tab): an aggregate index and observation period, not a comparison of a particular use case.
- McKinsey — enterprise AI costs (opens in a new tab): survey and criteria for model selection and task routing.
- Anthropic — how model reasoning works (opens in a new tab): reasoning tokens are billed as output even when the text is not returned.
- OpenAI — reasoning models (opens in a new tab): reasoning tokens are not visible through the API, but take up context and are billed as output.
- Bai and colleagues — token consumption of coding agents (opens in a new tab): eight models on real tasks; same task, consumption differing by up to 30 times. It concerns programming, not every use of AI.
- 404 Media — Accenture limits employees' Claude spending to $29 a month (opens in a new tab): a report based on a non-public internal communication; a single case, without the company's reasons. Part of the article is for subscribers only.
- The Next Web — summary of the 404 Media report (opens in a new tab): details of the internal communication and employees' reactions; a secondary source, readable in full.
- OpenAI — API deprecations (opens in a new tab): announcements of service retirement and migration.
- Anthropic — model lifecycle (opens in a new tab): status and retirement of models available through APIs.
- Microsoft — RAG evaluations (opens in a new tab): examples of measures for information retrieval and answer quality.
- OpenAI — API data controls (opens in a new tab): data use, retention and available controls.
- OpenRouter — privacy policy (opens in a new tab): data handling and differences between model providers.
- Microsoft — model deployment types (opens in a new tab): processing location, performance and billing options.
- vLLM — request memory configuration (opens in a new tab): how GPU memory, request length and concurrency relate.