There’s a demo that kills more agent products than any competitor ever will. The agent selects 143 customers, drafts 143 emails, and gets 129 of them right. The founder calls that 90% and celebrates. The business owner watching sees fourteen wrong emails about to go to real customers — and quietly decides never to hand over the list.
The maths of almost-right
The value of an agent is not the work it does. It’s the checking you no longer have to do. The moment any row might be wrong, you have to read every row — 143 checks, exactly what you were doing before, now with an extra layer of anxiety on top. A 90% reliable agent doesn’t save you 90% of the work. It saves you none of it.
An agent is useful at 100% or it is useful at 0%. There is no market for almost.
That sounds like an impossible bar, because models make mistakes and always will. It stops being impossible when you notice the bar isn’t on the model. It’s on the architecture around it.
The model is not the authority
The model’s job in Datify is to understand the instruction and draft the plan. What it never does is touch the database freehand. “Everyone who hasn’t bought in 180 days” is not a paragraph the model writes — it’s a query deterministic code resolves, the same way every time. Then the policy layer strips anyone your rules protect, before a single message exists.
A model mistake in this shape produces a wrong plan — which you read, priced and itemised, before anything runs. It cannot produce a wrong send, because the send never came from the model.
A retry can never double-send
Distributed systems fail in boring ways: a timeout, a dropped connection, a worker that dies mid-batch. The naive fix — run it again — is how a customer gets the same email twice, which is worse than the timeout ever was. So every write in an execution carries its own key. Run the batch again after a crash and the 118 that already went are recognised and skipped; only the remaining 25 go. Boring engineering, invisible in a demo, and the entire difference between a tool and a liability.
Done means read back, not written
The last step of every run is the one most products skip: go and look. Did the provider accept it? Did it bounce? Did the record actually update? The run report is written from what came back, not from what was attempted:
The ledger never rounds up
Two bounces are two failures, and the report says so — no green tick for work that didn’t happen. Completed, skipped or failed is the whole vocabulary, and failures don’t cost a credit. This is also why we can charge on success: the same verification that makes the report honest is what makes the invoice honest.
None of this makes headlines. All of it is why, three weeks in, a business owner stops opening the report to check and starts opening it to see what got done. That transition — checking to reading — is the product.
Never ask “how accurate is the model?”. Ask what happens when it’s wrong. If the answer is “a bad email reached your customer”, the architecture failed, not the model.
Writes about what actually happens when you put an AI in charge of real business operations — including the parts that go wrong.