Latency & SLA

01. What is fast, what is LLM-bound

• Fast (retrieval path). Search, raw writes, and reads combine retrieval, fusion, filters, and optional ranking stages. The fast-search release objective is p95 at or below 750ms; it is an engineering target, not a per-request or contractual guarantee.
• LLM-bound (write/reasoning path). Fact extraction, conflict resolution, and dialectic reasoning call a reasoning model. These are seconds, not milliseconds, and they are exactly the operations you should run in the background.

02. Per-endpoint guidance

Typical observed latency on warm infrastructure. These are guidance targets for capacity planning, not contractual guarantees (see SLA below). Cold starts and large batches run higher.

Note: these are end-to-end numbers, the full hybrid retrieval + rerank (and, for chat, LLM generation), not an isolated vector-lookup figure. Measure against your own clients using Server-Timing, X-Process-Time, X-Hebbrix-TTFT-Target-Ms, and X-Hebbrix-Degraded-Stages response headers.

03. Production pattern: async extraction

async_dispatch defaults to true on /v1/memories/process. You get a 202 with a job_id; poll /v1/memories/jobs/{job_id} for completion. Your user already has their answer.

Raw ingestion has a separate durability/readiness contract. With wait_for_index: false (the default), the response acknowledges the atomic PostgreSQL and indexing-outbox commit; inspect searchable and poll status_url. With wait_for_index: true, Hebbrix waits up to 250ms for searchability, within a 1.5-second response objective. It returns 201 if indexing completes or 202 with Retry-After, Location, and outbox_event_id when the durable write is still converging.

import os, time, requests

BASE = "https://api.hebbrix.com/v1"
H = {"Authorization": f"Bearer {os.environ['HEBBRIX_API_KEY']}"}

# 1) Inline: fetch context for THIS turn (fast, hot path)
ctx = requests.post(f"{BASE}/search",
    headers=H, json={"query": user_message, "collection_id": "coll_123", "limit": 5}).json()

# 2) Respond to the user with YOUR LLM (using ctx) ... already done here ...

# 3) Off the hot path: learn from the turn ASYNCHRONOUSLY
job = requests.post(f"{BASE}/memories/process", headers=H, json={
    "messages": [
        {"role": "user", "content": user_message},
        {"role": "assistant", "content": assistant_reply},
    ],
    "collection_id": "coll_123",
    "async_dispatch": True,        # explicit; true is currently the default
}).json()

# 4) (Optional) poll for completion in a background worker
job_id = job["job_id"]
while True:
    s = requests.get(f"{BASE}/memories/jobs/{job_id}", headers=H).json()
    if s["status"] in ("completed", "failed"):
        break
    time.sleep(1)

04. SLA & status

• What we publish. GET /v1/slo is the canonical public contract. It currently describes best-effort service with p95/p99 and availability engineering objectives, not a generic contractual SLA. Signed plan or enterprise terms control.
• Observability. Every response carries X-Process-Time (server ms) and X-Request-ID. Chat additionally reports stage timings and any degraded stages. Measure real latency from your own clients and include the request id and deployed build header when reporting a slow call.