AG-012Agent · Taking the pathlayaeval.ToolSequenceCorrectness
Tool Sequence
Were the tool steps done in a sensible order?
Simulation state:
When it passes
Right order.
Threshold satisfied: evaluation score >= 3.0.
When it fails
Booked before choosing.
Violation detected: execution flagged for human review or fallback routing.
SDK Specification
| Parameter | Value | Description |
|---|---|---|
| Class | ToolSequenceCorrectness | Python evaluator class in layaeval |
| Required Fields | "task", "executed_sequence", "expected_sequence" | Arguments expected in the call signature |
| Default Threshold | >= 3.0 | Passes when quality score is at or above this value |
| Category | Agent Evaluation | Taking the path (Sensible order, no repeats, no going in circles.) |
Python Example
tool-sequence.py
1from layaeval import ToolSequenceCorrectness23metric = ToolSequenceCorrectness(threshold=3.0) # passes at or above 3.045res = metric(6 task="Book a hotel in Paris for 2 guests on Mar 4.",7 executed_sequence=["search_hotels", "choose", "book"],8 expected_sequence=["search_hotels", "choose", "book"],9)1011print(res.passed, res.value, res.confidence)
Related Agent Checks