AG-003Agent · Understanding the joblayaeval.TaskAdherence
Task Adherence
Did the agent follow the instructions and limits?
Simulation state:
When it passes
Stayed within the lines.
Threshold satisfied: evaluation score >= 3.0.
When it fails
Went off script.
Violation detected: execution flagged for human review or fallback routing.
SDK Specification
| Parameter | Value | Description |
|---|---|---|
| Class | TaskAdherence | Python evaluator class in layaeval |
| Required Fields | "instructions", "query", "agent_trace", "response" | Arguments expected in the call signature |
| Default Threshold | >= 3.0 | Passes when quality score is at or above this value |
| Category | Agent Evaluation | Understanding the job (Did the agent get what you wanted, and finish it?) |
Python Example
task-adherence.py
1from layaeval import TaskAdherence23metric = TaskAdherence(threshold=3.0) # passes at or above 3.045res = metric(6 instructions="Only book hotels under $200 per night.",7 query="What is the capital of France?",8 agent_trace="Step 1: search_hotels(city='Paris') -> 3 results\nStep 2: book(id=12) -> confirmed",9 response="Paris is the capital of France.",10)1112print(res.passed, res.value, res.confidence)
Related Agent Checks