Jev, Laya or LLMs: cutting AI costs and latency

Jev, developed by TypeSafe AI, selects, ranks and evaluates. An LLM writes. Assigning each task to the right model can reduce an AI application's costs and speed up some responses.

In Askolia, our AI website search and chat tool, we identified two possible uses and tested Laya, an alternative you can host yourself.

When should you use Jev instead of an LLM?

“Which FAQ answers this question?” asks for a selection from existing answers. “Explain this answer to the customer” asks for new text. These tasks can use different models.

Jev returns structured decisions: a choice, a score or a yes/no estimate. It does not write replies.

An LLM can also make selections. Jev's value therefore depends on the cost, speed and reliability of that decision in your application.

Where do the savings come from?

Savings come from calls you avoid. If Jev finds an approved answer already available in the right language, the application can display it without asking an LLM to rewrite it.

At the rate checked on September 23, 2026, 100,000 Jev decisions with 1,000 input tokens each cost USD 4.20. This covers inference only.

A fictional example: 100,000 LLM calls at USD 0.001 cost USD 100. If Jev avoids 70% of those calls, the total falls to USD 34.20, including Jev decisions. Integration and operations still need funding.

If every request still reaches the same LLM with the same context, Jev adds cost and an extra step. At low volumes, the saving may also be too small to justify integration.

The same reasoning applies to speed: measure the time until the answer appears. A fast selection only helps the visitor if it actually shortens their wait.

Askolia: two concrete uses

Select an FAQ without generating text

In the Askolia FAQ workflow we studied, an LLM selects an answer when automatic matching fails. A second call then adapts the text to the visitor's language.

One opportunity is to let Jev handle selection. Another is to reuse stored translations. These changes can avoid different calls, so their savings need separate measurements.

When no FAQ fits, the system needs to keep searching or hand over to a person. Quickly displaying the wrong answer would create additional work for support.

Choose better sources for RAG

Retrieval-augmented generation, or RAG, finds documents before asking an LLM to write. Jev could rerank retrieved passages to pass more relevant evidence to the LLM.

In Askolia, this would be an additional step. Its initial value would be quality; savings remain possible if it reduces the text sent to the LLM or the number of answers needing correction.

For integration, we would start with the official TypeScript SDK. jev-reranker offers another path for Python experiments.

Laya: when should you host decisions yourself?

Laya also provides choices and scores, with open source code and downloadable models. It lets you run those decisions on your own infrastructure.

That may suit a team wanting control over where its data is processed. Hardware, availability and maintenance still cost money. At low volumes, an API can be cheaper.

What our Laya experiment taught us

On a small set of synthetic multilingual cases inspired by Askolia, Laya selected an FAQ in roughly 18 ms median, once the model was loaded. This local timing does not measure the complete response time.

But with the settings carried over from the Jev prototype, filtering blocked even questions that an FAQ could answer. Document source selection was not reliable enough either.

We therefore did not adopt Laya as a direct replacement with those settings. A model can decide quickly while leaving too many requests without a useful answer.

This experiment measures no gain over Jev or the current LLM. The detailed results and protocol remain available to examine the test's limitations.

Where should you start in your application?

Choose one frequent, specific decision: finding an FAQ, routing a ticket or selecting a source. Compare alternatives on your own requests, including ambiguous cases and your users' languages.

Track three outcomes: cost per correct answer, complete response time and requests handed to a person. A lower token price is insufficient if errors increase.

In Askolia, FAQ selection is our first opportunity to compare Jev with the existing workflow. RAG deserves a separate experiment focused on source quality.

To identify avoidable calls in your product, explore our AI integration service or tell us about your application.