The Payer

Spending rules for your shopping agent

Agent on

How well does the checker decide?

67 purchase requests, each labelled approve / ask / block against one fixed set of rules. 17 are held back as a test set and never used for tuning.

Latest run

gemini:gemini-3.5-flash-lite:v4 · held-out test set

82%

Wrong approvals0

money at risk

Needless asks2

annoyance

Wrong blocks1

task fails

Correct14/17

accuracy on cases never used for tuning

All runs (test set)

CheckerAcc.WANAWB
v4-repeatgemini:gemini-3.5-flash-lite:v482%021
v4gemini:gemini-3.5-flash-lite:v482%021
v3-repeatgemini:gemini-3.5-flash-lite:v382%021
v3gemini:gemini-3.5-flash-lite:v376%121
baseline-67keyword-baseline53%530
v2gemini:gemini-3.5-flash-lite:v282%021
flash-lite-v1gemini:gemini-3.5-flash-lite:v182%111
baselinekeyword-baseline53%530

WA = wrong approvals · NA = needless asks · WB = wrong blocks