Latest run
gemini:gemini-3.5-flash-lite:v4 · held-out test set
82%
Wrong approvals0
money at risk
Needless asks2
annoyance
Wrong blocks1
task fails
Correct14/17
accuracy on cases never used for tuning
Spending rules for your shopping agent
67 purchase requests, each labelled approve / ask / block against one fixed set of rules. 17 are held back as a test set and never used for tuning.
gemini:gemini-3.5-flash-lite:v4 · held-out test set
82%
money at risk
annoyance
task fails
accuracy on cases never used for tuning
| Checker | Acc. | WA | NA | WB |
|---|---|---|---|---|
| v4-repeatgemini:gemini-3.5-flash-lite:v4 | 82% | 0 | 2 | 1 |
| v4gemini:gemini-3.5-flash-lite:v4 | 82% | 0 | 2 | 1 |
| v3-repeatgemini:gemini-3.5-flash-lite:v3 | 82% | 0 | 2 | 1 |
| v3gemini:gemini-3.5-flash-lite:v3 | 76% | 1 | 2 | 1 |
| baseline-67keyword-baseline | 53% | 5 | 3 | 0 |
| v2gemini:gemini-3.5-flash-lite:v2 | 82% | 0 | 2 | 1 |
| flash-lite-v1gemini:gemini-3.5-flash-lite:v1 | 82% | 1 | 1 | 1 |
| baselinekeyword-baseline | 53% | 5 | 3 | 0 |
WA = wrong approvals · NA = needless asks · WB = wrong blocks