🎯 How to Use Laya — the practical guide

1 · The 60-second mental model

Laya is not a chatbot. It is a decision function: text in → typed answers out, one forward pass, no text generation. You hand it a compact state and a set of typed questions; it returns an answer + calibrated confidence per question in ~0.9 s on CPU:

Never ask Laya to write text, summarize, or reason in steps — that's the narrator/gemma path. Laya answers questions you define; the quality of those questions is the quality of the system.

2 · Where it lives

Loopback-only on both hosts. Callers must run on the same host (or tunnel). LAYA_API_KEY must be set before ever binding a non-loopback interface. systemd user unit laya-serve.service (Restart=always), 4.7 GB RAM, all 3 checkpoints preloaded.

3 · Quickstart (60 seconds)

curl -s http://127.0.0.1:8099/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Alert: repeated failed SSH logins from 10.0.0.5 targeting prod-01 root
            over 3 minutes. fail2ban banned the source IP. No successful logins.",
  "questions": {
    "alert_class": {"type": "choice",
      "instructions": "What kind of event is this?",
      "criteria": {
        "bruteforce": "repeated auth failures against a service",
        "malware":    "evidence of malicious code execution",
        "benign":     "routine ops noise, tests, expected maintenance",
        "other":      "anything else"}},
    "urgency":     {"type": "score",
      "instructions": "How urgent is triage for this alert?",
      "criteria": ["not urgent", "soon", "critical"]},
    "auto_close":  {"type": "noul",
      "instructions": "Is this alert safe to auto-close with no analyst action?"}
  }
}'

Round-trip on CPU: ~0.5 s for 3 questions. Response anatomy:

Read the confidence first — a choice at conf 0.02 is Laya saying "don't trust me"; treat that as "send to gemma", not as an answer. noul works opposite: confidence = 1 − noul is how sure it is about the "yes" reading — check the field, not the sign convention.

4 · Writing a question set (the part that matters)

Each question has 3 ingredients. Get them right and Laya behaves; skip them and you get zero-shot mush (see the real smoke: base-model "malware 0.71" for a textbook bruteforce — that's what criteria are for):

• instructions — one sentence, decision-oriented. "Is this a known, recurring event pattern?" beats "classify the event".
• criteria — for choice/score: the options with one-line definitions (score takes an ordered list like ["anytime","soon","critical"]). noul takes none — it's binary.
• a name that says the answer — the option is what you act on in code: answers["response"]["choice"] == "auto_remediate".

The live SOC sets (copy these patterns)

# evals/laya_questions.yaml — triage (v1)
severity:       type: choice  # informational|low|medium|high|critical, level-based criteria
known_pattern:  type: noul    # "Is this a known, recurring event pattern…?"
response:       type: choice  # note_only|digest_only|email|page|auto_remediate
escalate:       type: noul    # "Should this alert open an incident / reach a human now…"

# email (imap-watcher)
reply_worthy:     type: noul   # "Should the SOC send a substantive human reply…"
priority:         type: score  # anytime|soon|critical
automated_noise:  type: noul   # bounce/notification/bot vs human asking

# ioc (Phase 2 — needs a trigger path first)
reputation:        type: choice # malicious|suspicious|neutral|unknown
needs_enrichment:  type: noul

Semantics contract lives in soc-agents/soc-triage/IDENTITY.md; the YAML is the review surface. Keep them in sync — there's a self-check test in state_builder.

5 · State building — the 150-token discipline

Laya reads ≤512 tok (english) / 1024 (typed-decisions) / 8192 (multilingual+max_len). SOC turns are ~120k tok. The whole trick is building a small state — never feed raw turns:

# build_alert_state() — live code, edge host laya_shadow.py
"Wazuh alert rule 5710 severity level 12 on agent web-01."
"Rule: Multiple failed logins from 1.2.3.4."        # rule description ≤220 chars
"Decoder: sshd. Source IP: 1.2.3.4. Location: /var/log/auth.log."

• Put the fields the question needs in the state — decision inputs only (rule id/level/agent/srcip/description for triage; from/subject/body-head for email).
• Target ≤150 tokens per alert, ≤300 per email — check usage.input_tokens in the response.
• Nested JSON (rule.level etc.) must be flattened in the builder — learned the hard way (Option A).
• Long docs: HTTP API has no max_len in 0.3.20 → keep HTTP states compact; the SDK supports model="multilingual", max_len=8192 if ever needed.

6 · The gate — how a decision gets trusted

The whole point of calibrated confidence: conf ≥ τ → act on Laya; below → pay for gemma. Everything else is plumbing:

• choice: gate on answer_confidence; never act on a low-conf auto_remediate (live soak: 115 such rows at conf ≈0.03 — this is why response stays shadowed until finetune).
• noul: confidence = 1 − noul; a tight band (known_pattern sits at 0.68–0.74) means the gate threshold is doing the work.
• escalate also has an objective cross-check: the deterministic incident rule (sev high/critical). Live agreement 94.6% over 452 decisions.
• Deterministic facts (severity from rule level) never go to any model — a dict lookup is 1.00 accurate and free. Laya's value is the judgment questions around it.

7 · Adoption modes (how changes ship safely)

• off — default; gemma path untouched. LAYA_MODE=off in /etc/default/<service>.
• shadow — live now: every decision logged to /opt/soc-openclaw/data/laya-shadow.jsonl, caller ignores results. Nothing user-visible changes.
• gated — Laya primary at conf ≥ per-question threshold, gemma fallback below. Flip = set LAYA_MODE=gated + thresholds; rollback = the env var.
• primary — after 2 weeks gated-clean. LLM becomes the rare-path escalation only.

8 · Checkpoints — which one answers

Routing happens per request (routing.model in the response tells you who answered). LAYA_AUTO_TASK=0 — routing is explicit, not automatic. One calibration bin (choice:11+) ships uncalibrated — confidences from that bin are raw until the Phase 4 re-fit.

9 · SDK & CLI (for scripts and experiments)

# Python — same shapes as HTTP
from laya import Router
r = Router(preload=True)                    # 4.7 GB RAM, all 3 ckpts
out = r.predict(state, questions)
out = r.predict(long_doc, questions, model="multilingual", max_len=8192)

# CLI — quick sanity checks
laya "text" --preset triage --predict    # presets: triage, email, guard, moderation, router

10 · Ops: health, watchdog, limits

• Health: GET /health → {"status":"ok","loaded":[...],"device":"cpu"} — bypasses the inference queue, safe to poll.
• Watchdog already covers it: both hosts probed every 10 min + shadow-log staleness (>6 h = BROKEN).
• Limits: 64 questions max · 50,000 chars state · 2 MB body · single inference worker (requests queue; don't poll hot).
• Threads: LAYA_THREADS ≤ physical cores (8 on control host, 6 on edge host) — oversubscribing is a big regression.
• Restart: systemctl --user restart laya-serve (~10 s warm start, preloaded cache).

11 · Gotchas (all learned live)

• PyPI 0.3.20 ≠ GitHub main. Installed API = /v1/systemone + /health only. No /predict, no max_len over HTTP.
• laya-serve --help boots the server (5 GB RAM, binds nothing useful) — config is env-vars only.
• HF_HUB_OFFLINE=1 works because checkpoints are preloaded — preload any new checkpoint manually first.
• Install order matters: CPU-only torch wheel first, then laya[serve] (saves ~5 GB of CUDA).
• Zero-shot criteria judgments are mediocre on SOC specifics (documented in LAYA-SETUP.md §4) — structure is right, accuracy comes from good criteria + the Phase 4 finetune. Don't gate on accuracy before both land.

12 · What NOT to use it for

Anything needing generated text: narrations, replies, incident summaries — that's gemma's job (Laya can only gate whether narration runs). Also anything with an objective rule (severity from level = dict lookup) — spending a model on a deterministic mapping is wasted work you'll have to un-do later.

Sources: docs/LAYA-SETUP.md · evals/laya_questions.yaml · /opt/soc-openclaw/lib/laya_shadow.py (edge host) · docs/laya-baseline.md · 452-decision soak, 2026-09-27. Companion: REPORT-LAYA-300-POWER-20260927.html (same server).