Calibrated decisions on an ordinary CPU.
The edge tier of the Avrik family: a decision model with 4B effective parameters that answers on a laptop CPU in seconds, with no GPU. Built for offline and on-device use, and for high volumes of simple checks.
Send a state, like a ticket, a transaction, a policy and a case, and one or more typed questions. Get back a calibrated probability for every option, read straight from the model’s output in one forward pass.
Same model, same API, two ways to run it. Each build has its own repo and its own calibration, fitted on that build’s outputs.
cal-lite-q4-frozen-20261011-2Quantised with an importance matrix computed on decision prompts, so it keeps the weights that matter for letter readouts. On the same 148 calibration items, Q4_K_M got 111 right against bf16’s 115, and the top answer agreed on 142.
cal-lite-bf16-frozen-20261011-1GGUF Q4_K_M on a 6-thread laptop CPU (Ryzen 9 6900HX), one question
| State tokens | p50 | p95 |
|---|---|---|
| 256 | 5.02 s | 5.13 s |
| 300 | 6.02 s | 6.36 s |
| 1,000 | 21.19 s | 21.69 s |
All three Avrik models answer the same /v1/systemone API with the same question types. Pick a tier by accuracy, latency, and hardware, and switch without changing your code.
| Type | You send | You get |
|---|---|---|
Yes or no noul | A yes/no question | P(yes), from 0 to 1 |
Choice choice | A question and two or more named options | The top option and a probability for each |
Score score | A question and ordered levels, like low, medium, high | The expected level and a probability per level |
Multi-label multi_label | A question and a set of labels | P(yes) per label, and the labels above a threshold |
top, margin, confidence, and an abstain flag.abstain_option, the model can answer “Insufficient information to decide.”Run the model through avrik-server, not generate(). The readout and calibration are what turn raw weights into probabilities you can act on.
One fixed template per build, its SHA-256 in every response. Options become letters A, B, C.
One forward pass per question. Questions on the same state are batched. Nothing is generated.
Reads the probability of every form of each option letter, and refuses if the letters hold under 98% of the mass.
Per-type temperatures fitted for this exact build and bound to its hash.
Measured by Avrik AI on the bf16 build. Every accuracy carries a 95% Wilson interval, because a bare point estimate hides how few items sit behind it.
277/320 overall. 320 text items in 24 families. 100 labels are computed by code; 220 were drafted by a model and aren’t human-verified, so treat those families as indicative.
Agent governance
Core reasoning
Long context
Consistency
Scoring
The 231 public tasks of JevBench v1.4.2, run by Avrik AI. Not an official measurement, and not on the scale of the current leaderboard.
Pending. Avrik’s sealed evaluation set is written by people, kept encrypted, and never used for training or tuning. Results will be added to the model card in a later revision.
Expected calibration error, 10 bins, top option. Lower is better. When this model says 0.8, it should be right about 80% of the time. Calibration pool held out from dev; dev shown for comparison. Each build is fitted on its own outputs.
avrik pull verifies every file before anything loads. avrik serve exposes the API on 127.0.0.1:8080, loopback only.
On a CPU (Linux, macOS or Windows; no GPU needed). The first line installs llama.cpp’s prebuilt CPU wheel, so nothing has to compile.
pip install llama-cpp-python==0.3.35 --no-deps --only-binary=:all: \
--index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
pip install "avrik[cpu]"
avrik pull Avrik-Lite-4B # downloads avrik-ai/Avrik-Lite-4B-GGUF and verifies it
avrik ask Avrik-Lite-4B \
--state "Customer bought headphones 12 days ago, unopened, receipt attached. \
Policy: unopened items within 30 days get a full refund." \
--noul "Should the refund be approved?" \
--choice "Which action?=approve,deny,escalate"
avrik serve Avrik-Lite-4B # the API on http://127.0.0.1:8080On a GPU with vLLM (Linux, CUDA). avrik pull downloads the repo and verifies every file, vLLM serves the weights on loopback, and avrik serve puts the decision API in front.
pip install avrik "vllm==0.30.0"
avrik pull Avrik-Lite-4B --format bf16 # -> ~/.cache/avrik/Avrik-Lite-4B-bf16
HF_HUB_OFFLINE=1 VLLM_NO_USAGE_STATS=1 DO_NOT_TRACK=1 VLLM_BATCH_INVARIANT=1 \
vllm serve ~/.cache/avrik/Avrik-Lite-4B-bf16 --served-model-name avrik \
--dtype bfloat16 --max-model-len 70000 --logprobs-mode raw_logprobs \
--attention-backend TRITON_ATTN \
--host 127.0.0.1 --port 8000 &
avrik ask Avrik-Lite-4B --format bf16 --vllm-url http://127.0.0.1:8000 \
--state "Customer bought headphones 12 days ago, unopened, receipt attached. \
Policy: unopened items within 30 days get a full refund." \
--noul "Should the refund be approved?" --choice "Which action?=approve,deny,escalate"
avrik serve Avrik-Lite-4B --format bf16 --vllm-url http://127.0.0.1:8000 # API on :8080| Route | Purpose |
|---|---|
POST /v1/systemone | Decide: state and questions in, calibrated answers out |
GET /health | Readiness and the loaded build’s hashes |
GET /v1/models | The served model |
GET /openapi.json | The OpenAPI 3.1 schema |
# Typed Python client, sync and async
pip install avrik-sdkSet your own thresholds: approve above 0.95, send to a person between 0.2 and 0.95, and so on.
Weakest dev-set areas: trap_adversarial (4/10), score_quality (5/10), rule_precedence (9/15). The model judges only what’s in the state, so instructions planted there can sway it. Keep untrusted text clearly delimited.
Every file is listed with its SHA-256 in MANIFEST.sha256. Inference makes no network calls, sends no telemetry, and stores nothing.
| Base | google/gemma-4-E4B-it at ee0ef6023621cff504d758262d4e04895a5af4a2 |
|---|---|
| Recipe | Frozen base with fitted per-type temperatures. No fine-tuning. |
| Release gate | Frozen base. A fine-tuned Lite is in evaluation and replaces this build only if it beats it on the sealed, human-authored set. |
| GGUF Q4_K_M build | Model hash avrik-ai/Avrik-Lite-4B-GGUF@c6f10bb369fe162aPrompt template a0eb0c15ba82644a1c8cc1fa5193e9f80e64c4b63a88fe38a65588e11ab16ff3 |
| bf16 build | Model hash avrik-ai/Avrik-Lite-4B@8004525daedc7abdPrompt template faef1dd3a7300a8a3cbb5204a78a57c28d2b70ca8965f715920cb4a56a0f9aa7 |
| Release | v1.0.0, October 11, 2026 |
avrik package.auto_map, and symlinks are refused. Tokenizers load with trust_remote_code=False.NOTICE with any redistribution.Same API, same question types. Move up a tier for accuracy, or down for speed and cost.
The default Avrik model. About 90 ms per decision on one GPU.
The most accurate Avrik model, for the hardest calls.
Open weights under Apache 2.0, with calibration and hashes for every build. Questions or a use case to talk through? We’d like to hear it.