AT&T Slashes AI Costs by Adopting Model Routers and Open Source

AT&T Slashes AI Costs by Adopting Model Routers and Open Source

AT&T cut the costs of coding and some other advanced artificial intelligence tasks by as much as 56% by using tools that route employees’ queries to cheaper models when appropriate, The Information reported Thursday (Aug. 20).

When doing so, the quality of the performance of the AI declined by only 2%, according to the report, which cited an interview with Mark Austin, an AT&T vice president who oversees AI used by the telecommunication company’s employees.

The tools AT&T used in this instance were LiteLLM model routers, which determine the complexity of the task and then decide if it can be sent to a cheaper AI model, the report said.

AT&T aims to keep its employees spending on models from Anthropic and OpenAI flat by making greater use of open-source or open-weight models. The company aims to increase the share of employees’ queries that are powered by open-source models from the current 40% to between 60% and 70% in the coming years, per the report.

AT&T is using open-source or open-weight models such as Nvidia’s Nemotron, Meta’s Llama and Google’s Gemma. The company is not using open-source models from the Chinese firms DeepSeek and Moonshot but is evaluating the potential risks of using them, according to the report.

AT&T’s experience has been that the capabilities of open-source models have generally been six to 10 months behind those of frontier models, but the gap is narrowing and the open-source models are “just as good or better” than older models from Anthropic and OpenAI, Austin said, per the report.

It was reported in June that companies are looking to better manage their use of AI after seeing the costs of the technology rise.

The rising costs have been driven by the shift from chatbots to agents, which consume more computing power, as well as the AI labs’ move from flat subscriptions to token-based billing.

PYMNTS reported in July that new tools are being launched to address the issue of AI costs and that the era of “tokenmaxxing,” or pushing employees toward the biggest AI models and heaviest usage, is ending after two years of unchecked growth.