Legacy Policy Administration
Inspect and administer offline policy snapshots when legacy RL training is enabled for your deployment. New learning integrations should use Outcome Memory.
- See which version is currently running
- Compare different versions to see which performs better
- Promote a better version to production
- Rollback to a previous version if needed
01. Operator-Controlled Deployment
A policy rollout should use three explicit, separately validated stages:
1. Shadow Mode
Evaluate a candidate without applying its decisions. Shadow behavior requires deployment configuration and representative outcome collection.
2. Canary Release
Route an operator-chosen traffic fraction only after offline checks. Rollout percentages, guardrails, and rollback criteria belong to your deployment configuration.
3. Production
Promote through the authenticated admin endpoint only after validating causal evidence, safety constraints, latency, and tenant isolation.
02. When Do You Need This?
Testing New Improvements
When an offline candidate exists, evaluate it against held-out and randomized evidence before exposing it to user traffic.
A/B Testing
Compare policy versions with predeclared metrics and propensities. Outcome Memory provides the decision and delayed-result evidence.
Quick Rollbacks
If a new version isn't working as expected, roll back to the previous stable version through an explicit admin action. Do not assume automatic rollback unless your deployment has separately configured and tested it.
Per-Request Control
Search accepts explicit ranking controls, but production deployments may disable legacy RL entirely. Inspect response policy and fallback metadata.
03. Available Endpoints
04. Tier Access
05. Code Examples
Check Policy Status
import os
import requests
BASE = "https://api.hebbrix.com/v1"
H = {"Authorization": f"Bearer {os.environ['HEBBRIX_API_KEY']}"}
# GET /v1/policies/status: current production policy
r = requests.get(f"{BASE}/policies/status", headers=H)
status = r.json()
prod = status.get("production")
if prod:
print(f"Current production version: {prod['version']}")
print(f"Promoted at: {prod.get('promoted_at')}")
print(f"Metrics: {prod.get('metrics', {})}")
print(f"Total versions: {status['total_versions']}")
# GET /v1/policies: paginated list of trained policy versions
r = requests.get(f"{BASE}/policies", headers=H)
page = r.json()
for policy in page["items"]:
tag = "ACTIVE" if policy["status"] == "production" else policy["status"].upper()
print(f"Version {policy['version']} ({tag})")
if page["has_more"]:
# Fetch next page with ?cursor=page["next_cursor"]
passPromote a New Policy
# POST /v1/policies/compare: side-by-side eval of baseline vs candidate
r = requests.post(
f"{BASE}/policies/compare",
headers=H,
json={
"baseline_version": "v1.2.4",
"candidate_version": "v1.2.5",
"agent_type": "memory_manager",
},
)
comparison = r.json()
print(f"Improvement: {comparison['is_improvement']}")
print(f"Recommendation: {comparison['recommendation']}")
# If the candidate wins, promote it (POST /v1/policies/promote)
if comparison["is_improvement"]:
r = requests.post(
f"{BASE}/policies/promote",
headers=H,
json={"version": "v1.2.5", "agent_type": "memory_manager"},
)
print("Promoted v1.2.5 to production!")
else:
print("New version doesn't perform better. Keeping current.")Emergency Rollback
# POST /v1/policies/rollback: restore a previous version to production
# Requires admin role + Pro tier. Both version and agent_type are required.
r = requests.post(
f"{BASE}/policies/rollback",
headers=H,
json={
"version": "v1.2.4",
"agent_type": "memory_manager",
"reason": "regression in retrieval quality", # optional, logged for audit
},
)
result = r.json()
print(f"Rolled back to version: {result['version']}")
print(f"Promoted at: {result.get('promoted_at')}")06. Best Practices
Always Compare First
Never promote a new policy without comparing it to the current one.
Monitor After Promotion
Watch for changes in error rates or performance degradation after promoting.
Keep Previous Versions
Don't delete old versions immediately. Keep at least 2-3 stable versions for rollback.
Have a Rollback Plan
Know how to rollback quickly. Save the rollback command somewhere accessible.