RAG-013RAG · Enough to answerlayaeval.AbstentionCorrectness
Abstention Correctness
When sources are empty, did it honestly say “I don't know”?
Simulation state:
When it passes
Honest “I don't know”.
Threshold satisfied: evaluation score >= 0.50.
When it fails
Made up an answer.
Violation detected: execution flagged for human review or fallback routing.
SDK Specification
| Parameter | Value | Description |
|---|---|---|
| Class | AbstentionCorrectness | Python evaluator class in layaeval |
| Required Fields | "query", "retrieved_context", "response" | Arguments expected in the call signature |
| Default Threshold | >= 0.50 | Passes when quality score is at or above this value |
| Category | RAG Evaluation | Enough to answer (Is there enough in the pages to answer at all?) |
Python Example
abstention-correctness.py
1from layaeval import AbstentionCorrectness23metric = AbstentionCorrectness(threshold=0.50) # passes at or above 0.5045res = metric(6 query="What is the capital of France?",7 retrieved_context="Paris is the capital of France, with 2.1M residents.",8 response="Paris is the capital of France.",9)1011print(res.passed, res.value, res.confidence)
Related RAG Checks