Langgraph Decision Models

<overview> A **decision model** answers typed questions about state and returns probabilities instead of prose. It replaces the common pattern of prompting an LLM, parsing its text, and branching on the result.

• Noul(instructions=...) — binary question, returns a probability of yes
• Choice(instructions=..., criteria={...}) — picks one label, returns the full distribution plus confidence
• Score(instructions=..., criteria=[...]) — grades against an ordered rubric, returns an expected value plus confidence

TypeSafeClassifier is a LangChain Runnable[ClassifierRequest, ClassifierResponse], so it drops into a node like any other runnable. Up to 32 questions share one request and are answered independently — your code combines them.

Reach for one when a node generates text only so you can parse a decision out of it: routing, triage, filtering, guardrails, or per-item classification over a batch.

Do not reach for one when the node's output is the product (summaries, drafts, code) or when the judgment needs multi-step reasoning. A decision model classifies; it does not think. </overview>

---

Install and wire

langchain-typesafe is alpha (0.0.1a3) and TypeSafeClassifier is marked @beta — pin it and expect churn.

uv add langchain-typesafe

Three ways to reach a model. The classifier POSTs to {base_url}/v1/systemone with Authorization: Bearer {api_key}, so switching providers is constructor arguments only:

<ex-wiring> <python> ```python import os from langchain_typesafe import TypeSafeClassifier

1. TypeSafe directly (Jev). Reads TYPESAFE_API_KEY when api_key is omitted.

classifier = TypeSafeClassifier(model="jev-latest")

2. SemIf, hosted on the LangSmith Gateway. Note: LangSmith key, not a TypeSafe key.

classifier = TypeSafeClassifier( model="semif-qwen3.5-4b", api_key=os.environ["LANGSMITH_API_KEY"], base_url="https://gateway.smith.langchain.com", )

3. Jev through the Gateway (BYOK). The typesafe/ prefix routes to a

TYPESAFE_API_KEY stored in LangSmith workspace secrets. Without that secret

every typesafe/* id returns 424 Failed Dependency -- before the model name is

even validated, so a 424 does not confirm the id is real.

classifier = TypeSafeClassifier( model="typesafe/jev-1.13.0", api_key=os.environ["LANGSMITH_API_KEY"], base_url="https://gateway.smith.langchain.com", )

</python>
</ex-wiring>

---

## The conversion pattern

Ask every question about a page/item in **one** request, put the typed response in state, and let a plain function route on it. The router is ordinary Python — testable without touching a network.

<ex-classify-and-route>
<python>
```python
from typing import TypedDict
from langchain_typesafe import ClassifierResponse, Noul, Score, TypeSafeClassifier
from langgraph.graph import StateGraph, START, END

QUESTIONS = {
    "relevant": Score(
        instructions="How relevant is this ticket to a billing problem?",
        criteria=["Unrelated.", "Possibly related.", "Directly about billing."],
    ),
    "angry": Noul(instructions="Is the customer expressing anger?"),
}

class State(TypedDict):
    text: str
    answers: ClassifierResponse
    route: str

classifier = TypeSafeClassifier(model="jev-latest")

def classify(state: State) -> dict:
    # One request, every question. They are answered independently.
    return {"answers": classifier.invoke(
        {"state": state["text"], "questions": QUESTIONS}
    )}

def route(state: State) -> str:
    a = state["answers"]
    if a.nouls["angry"].noul > 0.7:
        return "escalate"
    if a.scores["relevant"].score < 0.5:
        return "close"
    return "handle"

</python> </ex-classify-and-route>

Reading answers: response.nouls[id].noul, response.choices[id].choice, response.scores[id].score. Each view is keyed by your question id; response.answers holds them all.

---

Three traps when reading answers

These cause silent misrouting, not exceptions.

1. Score.score is an expected value, not a level. It is a probability-weighted average over the rubric and is routinely fractional. score == 0 almost never fires — a "not responsive" item lands at 0.07, not 0. Always compare against a band.

if a.scores["relevant"].score < 0.5:   # correct
if a.scores["relevant"].score == 0:    # WRONG -- nearly never true

2. Confidence measures distribution shape, not correctness. On a Score, confidence reports how concentrated the rubric distribution is. An item sitting cleanly between two levels scores low confidence even when the model is entirely clear about it. A blanket confidence < X -> escalate rule therefore escalates items the model already decided. Gate on confidence only inside the ambiguous middle:

if score < NOT_RELEVANT:                          # decisive -- trust it
    return "close"
if score < RELEVANT or confidence < MIN_CONF:     # ambiguous -- escalate
    return "human_review"
return "handle"

3. Thresholds do not transfer between models. Calibration is part of the model. The same policy over the same items routes differently on Jev vs SemIf vs an LLM adapter. Re-tune thresholds whenever you change models, and pin the model id.

---

Question wording dominates accuracy

A loose question produces confident wrong answers, and no threshold fixes it. Use criteria to say what each outcome means, including what should not count.

In a measured case, "Is this a confidential communication with a lawyer?" scored a routine finance memo at 0.798. Rewriting it to name the actual test — written by or to a lawyer, with an explicit carve-out for finance and accounting content — moved the same page to 0.005 while a genuinely privileged page held at 0.991.

Noul(
    instructions=(
        "Was this written by or to a lawyer, or does it convey a lawyer's legal "
        "advice? Answer no for ordinary business or accounting discussion, even "
        "when the subject is litigation-sensitive."
    ),
    criteria=NoulCriteria(
        true="A named attorney is author or recipient, or it relays legal advice.",
        false="Business or accounting content with no attorney involved.",
    ),
)

Before blaming the model, rewrite the question and re-measure.

---

Auditing an existing agent

To find where a decision model fits, look for these in the codebase — see references/conversion-playbook.md for the full walkthrough.

The last two matter most: cheap semantic judgments change what you can build, not just the bill. If evaluating every item became affordable, what would you stop sampling?

---

Expectations

Measured on a 24-item batch, identical LangGraph graph and routing policy, only the classifier swapped:

Routing agreed on 4–5 of 6 items across engines; disagreements clustered on genuinely borderline items. Treat these as shape, not benchmarks — measure on your own workload.

If you compare against an LLM baseline, use method="json_schema" so the comparison is fair. LangChain's with_structured_output defaults to method="function_calling", which injects a tool schema into every request — 556/35 tokens versus 228/12 for the native output_config.format path on the same one-field probe.

---

Batching with Send

Classification is per-item and independent, so fan out with Send and let each item route on its own.

<ex-fan-out> <python> ```python from langgraph.types import Send

def fan_out(state): return [Send("classify_item", {"text": t}) for t in state["items"]]

builder.add_conditional_edges(START, fan_out, ["classify_item"])

</python>
</ex-fan-out>

Fan-out hides latency, so it flatters slow classifiers most: in the run above, Sonnet gained 8x from concurrency and Jev only 1.4x — yet Jev still finished first. Compare throughput, not the speedup multiple.

---

## Related skills

- **langgraph-fundamentals** — StateGraph, `Send`, `Command`, conditional edges
- **langgraph-human-in-the-loop** — `interrupt()` for the escalation branch above
- **langchain-middleware** — structured output when you need an LLM, not a classifier