OmegaEngine · Calibration

Risk scores that actually mean what they say.

OmegaEngine recalibrates risk scoring from real graded outcomes. As operators grade decisions, per-domain feedback patterns nudge risk and regret scores up or down — automatically, and only once enough signal exists.

How Calibration Works

Calibration aligns risk scores with observed outcomes. OmegaEngine continuously recalibrates using operator feedback:

adjustedRisk(domain) = baseRisk + Δ(badRatio, goodRatio)

A domain with more BAD than GOOD feedback is nudged up (up to +20 points); consistently GOOD domains ease down (up to −10). Domains with fewer than 5 graded events are left unchanged.

Each graded decision (POST /api/feedback) feeds the calibration loop
Risk and regret scores are adjusted per domain using recent feedback patterns
The current snapshot is readable at GET /api/calibration/summary

Calibration Metrics

The calibration snapshot tracks these per-domain signals:

🎯

Average Risk Score

Mean risk score of recent decisions in the domain — the baseline that feedback adjusts.

📏

Average Regret Score

Mean regret score of recent decisions — how costly a wrong call in this domain tends to be.

📊

Feedback Ratios

GOOD / NEUTRAL / BAD ratios per domain. A high BAD ratio nudges risk and regret scores upward.

📉

Event Counts

Decisions and feedback events per domain, so you can see how much signal backs each adjustment.

Risk Bands

Risk scores run 0–100 and map to a risk level; the judge's recommendedAction follows from the level, your risk tolerance, and whether human review was flagged:

Risk Score	Risk Level	Typical Recommended Action
0 – 25	LOW	PROCEED (or PROCEED_WITH_GUARDRAILS)
26 – 60	MEDIUM	PROCEED_WITH_GUARDRAILS
61 – 85	HIGH	PROCEED_WITH_GUARDRAILS — RECONSIDER at LOW risk tolerance
86 – 100	CRITICAL	RECONSIDER
any	—	ESCALATE whenever the decision is flagged for human review

API Reference

Calibration is fed by graded feedback on real decisions; the current snapshot is available to operators:

calibration.sh

# Feed the calibration loop: grade a real decision
curl -X POST https://omegaengine.ai/api/feedback \
  -H "Content-Type: application/json" \
  -H "x-api-key: $OMEGA_API_KEY" \
  -d '{
    "decisionId": "1c2a7e0e-5b3f-4c9d-9a41-8f2e6d0c7b1a",
    "rating": "BAD",
    "comment": "False positive - vendor was already approved in Q4 review",
    "labels": ["false_positive"]
  }'

# Read the current calibration snapshot (operator/admin key)
curl -H "x-api-key: $OMEGA_ADMIN_KEY" \
  https://omegaengine.ai/api/calibration/summary
# → { success, snapshot: { ...current calibration state... } }

Feedback & RLHF →Risk Scoring