Document RelevanceRetrieval GuardrailsGranular Chunk Audit3 min read

One Bad Chunk Poisons the Well: Auditing Document Relevance in RAG

You pulled 5 chunks from your knowledge base. 4 are gold, but 1 is an outdated 2021 policy. Guess which one your LLM quotes? Here is how to filter chunks one by one.

Published October 26, 2026·LayaEval Engineering

View interactive Document Relevance scene

The Reality Check

Vector databases return chunks in bundles, but LLMs weight contradictory chunks unexpectedly. One outdated sentence can overturn four paragraphs of accurate data.

01 · Business Problem

The 'Rotten Apple' Syndrome in Multi-Document Retrieval

When aggregating documents from Confluence, Notion, Google Drive, or customer ticket archives, outdated or tangential paragraphs inevitably get swept into the top-k results. The LLM treats every chunk provided as absolute gospel.

Real-World Scenario

"In a medical diagnostic assistant, 3 chunks describe the current 2024 treatment protocol. 1 chunk is an archived study from 2018 with deprecated dosage limits. The LLM merges both and suggests a conflicting dosage to a practitioner."

The Hidden Costs of Leaving This Unchecked:

Dangerous compliance and legal liabilities from hallucinated or deprecated guidance.
Hours of engineering triage trying to tweak prompt instructions like 'Only focus on modern advice'.
Erosion of user trust when users notice conflicting facts in the same response.

02 · What We Solve

Prune Every Single Chunk Independently

Unlike aggregate relevance checks, Document Relevance assesses each retrieved chunk on its own merits. Irrelevant chunks are discarded before string concatenation.

Prevented

Inclusion of legal boilerplate, navigation menus, and disclaimers in prompts.

Prevented

Contradictory context clusters poisoning generation.

Prevented

Wasted prompt window capacity on low-information paragraphs.

03 · The Solution

Iterative Chunk Filtering with DocumentRelevance

Pass each candidate chunk through DocumentRelevance. Discard chunks below threshold and assemble only verified documents into the final prompt context.

Rule of Thumb: If a chunk doesn't stand on its own feet for the user query, it has no business being in your prompt.

04 · Value Proposition

The LayaEval Advantage

Advantage 01

Parallelizable & Fast

Filter 10 retrieved chunks concurrently in under 15ms total.

Advantage 02

Per-Chunk Explainability

Know exactly which document failed and why, instead of guessing what went wrong in a monolithic prompt.

Advantage 03

Clean API Integration

Drop into LangChain document transformers, LlamaIndex node postprocessors, or raw Python lists.

05 · Python Implementation

$pip install layaeval

Run This Check in 4 Lines of Python

You just stripped out newsletter links and copyright boilerplate, saving tokens and shielding the model from distraction.

filter-toxic-chunks-document-relevance.py

1from layaeval import DocumentRelevance
2
3evaluator = DocumentRelevance(threshold=3.0)
4query = "What are the health benefits covered under Plan Silver?"
5
6raw_chunks = [
7    "Plan Silver provides 100% preventive dental coverage and $20 copays.",
8    "Click here to subscribe to our company newsletter for monthly health tips.",
9    "Plan Silver includes in-network prescription medications at $10 flat tier.",
10    "Copyright 2024 Acme Health Corp. All rights reserved across all states."
11]
12
13# Filter down to only high-relevance chunks
14clean_context = []
15for chunk in raw_chunks:
16    res = evaluator(query=query, document=chunk)
17    if res.passed:
18        clean_context.append(chunk)
19
20print(f"Kept {len(clean_context)} of {len(raw_chunks)} chunks.")
21# -> Kept 2 of 4 chunks! Garbage chunks safely discarded.

06 · REST API & Microservice Pattern

API Payload & Evaluation Response

Send single chunks to your evaluation layer for real-time document ranking:

Request (cURL / POST)

request.sh

1curl -X POST https://api.yourdomain.com/v1/eval/document-relevance \
2  -H "Content-Type: application/json" \
3  -d '{
4    "query": "What are the health benefits covered under Plan Silver?",
5    "document": "Click here to subscribe to our company newsletter..."
6  }'

Instant Sub-10ms JSON Response

response.json

1{
2  "metric": "DocumentRelevance",
3  "passed": false,
4  "score": 1.15,
5  "threshold": 3.0,
6  "confidence": 0.98,
7  "action": "prune"
8}

07 · Actionable Implementation Checklist

1
Hook DocumentRelevance into your retriever's re-ranking step.
2
Prune chunks scoring below 3.0 before building your system prompt.
3
Track pruned chunk ratios to tune your embedding model and chunking size.

Interactive Playground

Test Document Relevance in our visual demo

Tweak query and context sliders to see live pass/fail scoring in real time.

Open Interactive Scene

One Bad Chunk Poisons the Well: Auditing Document Relevance in RAG

The 'Rotten Apple' Syndrome in Multi-Document Retrieval

Prune Every Single Chunk Independently

Iterative Chunk Filtering with DocumentRelevance

The LayaEval Advantage

Parallelizable & Fast

Per-Chunk Explainability

Clean API Integration

Run This Check in 4 Lines of Python

API Payload & Evaluation Response

Test Document Relevance in our visual demo

More in the RAG Cookbook

How to Stop RAG Context Pollution with Retrieval Relevance