Jev and Laya: when can you skip an LLM call?
AI search and chat for your website.
Explore AskoliaChoosing an approved answer takes different work from writing a new one. Jev, developed by TypeSafe AI, selects and evaluates. An LLM writes.
In Askolia, our AI search and chat tool for websites, we explored that distinction and tested Laya, an alternative you can host yourself.
An FAQ makes the difference clear
A visitor asks how to delete their account. Your FAQ already has an approved answer. Jev can choose that answer from the available options without rewriting it.
If you need to explain a specific situation or translate the answer, an LLM can write the text. An LLM can also select answers, so compare results before changing models.
In the Askolia workflow we studied, one LLM call selects an FAQ when automatic matching fails. A second adapts the language. Using Jev for selection and reusing a stored translation are separate opportunities that need separate measurements.
When no FAQ fits, the system must keep searching or hand over to a person.
Savings come from calls you avoid
Displaying an approved answer in the right language can avoid an LLM call. If that call is still needed, Jev adds a step and a cost.
Include integration and running costs in the comparison. At low volumes, savings may not justify the change. For the visitor, measure the time until the answer appears.
Jev can also rank retrieved documents before an LLM uses them. In Askolia, this opportunity concerns source quality first. We have not demonstrated savings from it.
Laya was fast in our test, but we did not adopt it
Laya runs choices and scores on your infrastructure. You control where processing happens, but pay for hardware and maintenance.
On a small synthetic multilingual sample inspired by Askolia, an FAQ decision took roughly 18 ms median, once the model was loaded.
This local timing is not an end-to-end benchmark. With settings carried over from the Jev prototype, filtering rejected even questions that an FAQ could answer. Source selection was also inadequate.
We therefore did not adopt Laya as a direct replacement with those settings. No gain over Jev or the current LLM was measured. The protocol and results explain those limitations.
What should you test in your application?
Start with a frequent decision, such as selecting an FAQ. Use your own requests, including ambiguous cases. Compare cost per correct answer and complete response time, then check which requests need human help.
For more, read our article on OpenAI's Decisions API and Jev.
Swiss Made Software
