Ship TALON in two lines

Every request passes a Tier 1 regex/schema heuristic (<1ms) and a Tier 2 quantized INT8 ONNX classifier (3–8ms). Low-reasoning prompts resolve on a local SLM; only genuinely hard prompts reach a frontier model. A deterministic circuit breaker escalates automatically when an SLM returns malformed output.

resp = client.chat.completions.create(
    model="talon/auto",
    messages=messages,
    talon={
        "cascade": "aggressive",  # "conservative" | "balanced" | "aggressive"
        "min_tier": "slm",
        "max_tier": "frontier",
        "fallback": "gpt-4o",
        "schema_retry": 1,
    },
)

Strip-mining hashes stable system prompts so they are billed once and replayed at zero cost, then removes duplicated retrieval chunks and dead context before egress. Output fidelity is preserved — pruning is lossless with respect to the answer surface.

resp = client.chat.completions.create(
    model="talon/auto",
    messages=messages,
    talon={"strip_mining": True, "dedupe_chunks": True, "max_context": 8000},
)

print(f"Tokens Saved: {resp.talon.tokens_saved}")  # Output: 4812
print(f"Prompt Hash: {resp.talon.prompt_hash}")    # Output: sha256:1f9c (cached)

Both clients mirror the OpenAI surface: chat.completions.create, streaming, tool calls, and embeddings. Every response carries a talon block with the chosen route, overhead, and savings.

1import talon
2 
3client = talon.Client(api_key="tl_live_...")  # Drop-in replacement for OpenAI
4 
5resp = client.chat.completions.create(
6    model="talon/auto",  # Cognitive routing decides the optimal tier
7    messages=[{"role": "user", "content": "Extract the invoice total."}],
8    talon={"max_tier": "frontier", "strip_mining": True}
9)
10 
11print(resp.choices[0].message.content)
12 
13# Response metadata:
14# talon.route -> "slm:llama-3.1-8b" | Latency: 42ms | Saved: $0.0089

Drop-in wrappers keep long agent loops cheap: the system prompt is hashed once and replayed across every step.

from talon.langchain import ChatTalon

llm = ChatTalon(model="talon/auto", strip_mining=True)
chain = prompt | llm

from talon.crewai import TalonRouter

router = TalonRouter(policy="cost_first", fallback="claude-3-5-sonnet")
researcher = Agent(role="Researcher", llm=router.llm())

from autogen import AssistantAgent

config_list = [{
    "model": "talon/auto",
    "api_key": os.environ["TALON_API_KEY"],
    "base_url": "https://api.talon.ai/v1",
}]

assistant = AssistantAgent("planner", llm_config={"config_list": config_list})

Run KEEL Core inside your own VPC for zero-data-retention, SOC2, and HIPAA workloads. The Rust data plane is a single static binary; telemetry ships asynchronously.

services:
  keel:
    image: ghcr.io/talon-ai/keel-core:v1.0
    ports:
      - "4000:4000"
    environment:
      TALON_API_KEY: "${TALON_API_KEY}"
      TALON_ZDR: "true"

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keel-core
spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: keel
          image: ghcr.io/talon-ai/keel-core:v1.0
          ports:
            - containerPort: 4000

helm repo add talon https://charts.talon.ai

helm install keel talon/keel-core \
  --set zdr.enabled=true \
  --set autoscaling.minReplicas=3