You don't need to decode tokens when all you want is a decision from an LLM

The central idea here is simple and worth sitting with: you do not need to generate tokens to get a decision out of a language model. LLM2Jev extracts structured choices—yes/no, scored options, multi-candidate selection—purely from the logits produced during the prefill pass. The model reads the prompt and its candidate answers, and the probability mass over the candidate tokens at that single forward pass is enough to rank them. No sampling, no decoding loop, no beam search.

This matters because decoding is where latency accumulates. If your application only needs to know which of several options a model prefers, running the full autoregressive machinery is wasteful. Prefill-only scoring collapses that cost to a single forward pass per candidate, or fewer if you share prefixes.

The prefix-sharing design is the more interesting engineering detail. When multiple candidates share the same context—same state, same question instructions—the system stages submissions so that SGLang's Radix Cache is populated by the first candidate and reused by the rest. On cold requests, where no relevant cache exists yet, it scores a real candidate first specifically to seed the cache, then submits the remaining candidates against it. The MLX backend handles this differently, explicitly prefilling shared prefixes before scoring suffixes. Both paths converge on the same binary scoring interface, which is a reasonable design choice for portability.

The question types—Choice, Score, and Noul—map onto the Jev API's /v1/systemone endpoint, which means the project is positioning itself as a local drop-in for whatever Jev's hosted service offers. The README is explicit that this is an independent project with no affiliation. That disclaimer does real work: it signals the API is being reverse-engineered or mimicked, not licensed.

Multimodal support lands across all three backends—SGLang, Transformers, and MLX-VLM—letting you pass images alongside text in the state or instructions fields. The Snake game and MuJoCo pick-and-place demos are not just novelties; they illustrate the actual use case, which is model-driven control loops where a vision-language model picks actions from a discrete set on every frame or timestep. Prefill-only scoring is a natural fit there: low latency, no hallucination risk from open-ended generation, and order-independent candidate evaluation that avoids positional bias.

The benchmarks referenced in the README cover Qwen3-1.7B on an RTX 5090 and compare staged versus all-at-once submission across cold and warm cache states. The README is honest that gains depend heavily on input length, candidate count, and cache warmth—no single headline number is claimed.

Python 3.12 or higher is required. The project is a few days old by its own news log, which means the API surface will move.