Under the Hood: KEEL Core Engine

A Dual-Plane Architecture Built for Millisecond Precision.

Decoupling the ultra-fast Rust Data Plane from the asynchronous ClickHouse Telemetry Control Plane.

Data PlaneRust · Tokio · ONNX Runtime

Every request path is synchronous, lock-free, and allocation-bounded. KEEL Core adds sub-10ms P99 overhead and never blocks on telemetry, billing, or logging.

Ingress & Token Count→Sub-10ms Classifier→Context Strip-Miner→Model Cascade→Fallback Circuit Breaker

Control PlaneClickHouse · Kafka · async workers

Routing decisions, token deltas, and schema violations stream out of band into ClickHouse. Policy updates propagate back to the data plane without a redeploy.

Aggregate→Score→Re-policy→Push

Technical Features

Four primitives that make cheap tokens safe

Every optimization is deterministic, auditable, and bounded by a hard latency budget.

Sub-10ms Pre-Inference Classification

A dual-tier router decides reasoning depth before a single billable token leaves your VPC.

Tier 1 — Regex & JSON-schema heuristics resolve deterministic traffic in under 1ms.
Tier 2 — A quantized INT8 ONNX micro-model scores reasoning depth in 3–8ms on CPU.
No extra network hops, no auxiliary LLM call, no tokens billed for the routing decision.

Token Strip-Mining & Prompt Deduplication

TALON content-hashes system prompts and prunes redundant middle-turn context to eliminate token waste.

System prompts are content-hashed once, then referenced instead of retransmitted.
Conversation history is pruned by recency and salience, preserving head and tail instructions.
Duplicate RAG chunks are collapsed deterministically — same input, same output, every time.

Deterministic Schema Fallback Circuit Breaker

Cheap tiers only ship when outputs are provably correct. Otherwise TALON escalates silently.

SLM output is validated against your JSON schema before the stream is committed.
On malformed output, TALON escalates mid-stream to GPT-4o with the original context.
Per-provider breakers trip on latency or error-rate thresholds and reset on rolling health probes.

Zero-Data-Retention & Enterprise VPC

Deploy KEEL Core inside your own cloud so prompts never cross your network boundary.

Docker images and Helm charts for EKS, GKE, and AKS.
ZDR mode ensures no prompt or completion bodies are persisted to disk — only numeric telemetry.
SOC 2 Type II controls, HIPAA BAAs, and per-tenant residency pinning.

SDK Integration

Wire-compatible with the stack you already ship

Native clients for Python and TypeScript, plus first-party wrappers for LangChain, LlamaIndex, and CrewAI.

main.py·pip install talon-python

1import talon
2 
3client = talon.Client(api_key="tl_live_...")  # Drop-in replacement for OpenAI
4 
5resp = client.chat.completions.create(
6    model="talon/auto",  # Cognitive routing decides the optimal tier
7    messages=[{"role": "user", "content": "Extract the invoice total."}],
8    talon={"max_tier": "frontier", "strip_mining": True}
9)
10 
11print(resp.choices[0].message.content)
12 
13# Response metadata:
14# talon.route -> "slm:llama-3.1-8b" | Latency: 42ms | Saved: $0.0089

Live Router

Test KEEL Cognitive Routing Live

Pick a sample prompt and watch the classifier pick the cheapest tier that can still answer it correctly.

Awaiting prompt — select one above to route it.

Stop Paying Top-Tier Prices for Basic String Utilities.

Route intelligently, prune aggressively, and reserve frontier reasoning for the requests that actually need it. Two lines of code, sub-10ms overhead.

Get API Key (Free 100k Tokens) Schedule YC / Grant Demo