Four questions for any AI tutorial before you copy it

A tutorial is not evidence. It is a recording of one happy path, and the only thing it proves is that the path can be recorded.

Before you copy an AI workflow into your own sales team, ask four questions:

1. Who stands between the client and the system? If a person has to operate every run, that is not automation.
2. Is the saving shown, before and after? Time, touches, cost — measured, not asserted.
3. What data did it run on — five rows, or your real thousand?
4. How is it maintained and rolled out? Where it lives, who else can open it, what happens when the price list changes.

Our position, stated as ours: a model earns its place in a process only when it removes a step. If it adds one, you have bought a demo.

This is the second piece in a series. The first was for the person buying from a contractor; this one is for the founder watching a tutorial and thinking of running it on Monday.

The claim, in its strongest form

The claim we are answering is a promise made in a tutorial: a client sends a voice message, the assistant reads it together with an attached text price list, works out material volumes and delivery tariffs, and returns a finished commercial quote in two minutes. We do not name the author, we do not quote them, and we archive the page before publishing. The subject is the claim, not the person.

Steelman it first, because the strong version is partly right. Independent testing of a flagship model on applied tasks found a clean boundary: templated automation and first-pass transcription hold up, while directing and manual editing do not. Turning a spoken order into structured parameters is genuinely inside the model's competence.

So the boundary is not how smart the model is. It runs along two other lines: who is responsible for the number, and who is left standing between the client and the system. Where the answer is money, the same check says, a model does not calculate — it guesses a formula. With multi-factor pricing (weight, dimensions, delivery zones, stepped discounts) it makes arithmetic errors and interpolates conditions that were never given. In production the model may only extract order parameters; the calculation has to come from a deterministic rules engine.

Watch the steps, not the output

In our experience the output in a tutorial is a clean document, because the genre is a rehearsed run on tidy data where nothing unusual turns up and nothing fails. The steps are where the claim lives or dies.

What follows is our reading of what the video shows, not a transcript. That the operator never leaves the loop is in the recording; the breakdown into steps is our inference.

what the run appears to contain
  1. client sends a message to the manager
  2. manager re-dictates the order into the model's chat
  3. model asks clarifying questions
     (material length, distance to the site — the manager's own script)
  4. manager calls the client back for the answers
  5. manager dictates the answers into the chat
  6. model returns a quote
  7. manager sanity-checks the load volume against experience

steps removed by the model: 0
steps added by the model: 2   (re-dictation in, re-check out)

The person never left the loop. They changed jobs: from salesperson to operator of a chat window. And the half that would make this automation — the model talking to the client directly — is deferred, in the author's own description, to a future video. The missing part is the whole claim.

Question 1: who is between the client and the system

A published account of production pilots names the mechanism plainly: a pilot runs inside a cushion of human support that production will not have — the odd request somebody sorts out by hand, the quiet fixes nobody logs. The readiness test offered there is blunt: take that remediation away at any real scale, and things break.

A tutorial is that cushion at its thickest. One operator, one order, full attention, and every ambiguity resolved by a phone call the video does not count as work.

Question 2: is the saving shown

In the demo there is no before and no after. No baseline, no stopwatch, no count of touches. "Two minutes" is the length of the clip, not a measurement of the process.

The version with measurements exists. An engineering lead published an account of a team that drew a repetitive workflow on a board: twelve major steps, nine of them deterministic, two simple classification, one genuinely ambiguous text. The question that settled the architecture was which of those decisions needed a model at all. They built both versions. The agent one worked, and it demoed better — it was the one people liked to watch. Measured, it was worse on latency, higher on cost, and its failures were harder to reproduce. The deterministic build won on speed, on cost, and on how easily it could be tested, explained and run day to day.

Note what is not on that list. It was not less capable. It was less impressive.

Question 3: five rows, or the real thousand

A demo runs on a clean file — in our experience that is part of what makes it a demo. Ours did not, and this one is our own mistake rather than someone else's.

inventory file, as delivered
  key = item type + size + region     e.g. "widget | size M | region"

  35 distinct items shared one key
  -> collapsed into a single row, named after whichever arrived first
  -> repricing plan reported   79 positions
  -> real pool                 46 positions

on five demo rows: invisible — every key is unique

We found it in our own audit and fixed it. The point is not that a key can be weak; it is that five rows cannot show you that yours is. Whatever breaks your catalog breaks it at the size you actually have, which is the size no tutorial runs at.

Question 4: maintenance and rollout

Ask where the data lives, who else can open it, and what happens when the price list changes on Monday. A CSV on one laptop answers none of these. Neither does a chat history.

What we would build instead

For this job, three sheets in one spreadsheet — and a model only at the edge, if at all.

sheet 1  inventory        item, unit, price, stock
sheet 2  extra services   delivery zones, loading, stepped discounts
sheet 3  quote template   pulls rows from 1 and 2, formulas do the arithmetic

access   one link, the whole team
change   edit sheet 1, every future quote follows

where a model belongs, if the inbound message is messy:
  message    -> model: extract order parameters (fields, never numbers)
  parameters -> rules engine: volumes, tariffs, discounts, total
  total      -> template: the quote document
  document   -> person: sends it, and owns it

That is not a rejection of AI. It is a placement decision. The same shape holds one level up, and pipelines routinely get it backwards: classification, filtering and routing are dumped onto a heavy generative model when a specialized discriminative model does that job hundreds of times faster and cheaper, cutting the noise before the expensive call. Cheap and deterministic first, generation last, and never in the place where the answer is a number somebody pays.

FAQ

Does this mean an LLM has no place in quoting? It has a precise place: reading an unstructured request into fields. The arithmetic belongs to a rules engine, and the sent document belongs to a person.

What would change our mind? A tutorial that measures the same process before and after, on a real catalog, with no operator between the client and the system, and shows what happens when the price list changes. If that lands, we publish the correction in the same channel, at the same length.

Our catalog really is nine positions. Is a spreadsheet enough? Probably, yes — and that is the honest answer even though it sells nothing. The question is never how modern the tool is. It is which step disappears.

A related case

The pattern we keep meeting is the reverse of the one people expect. The clever component is usually fine; the simple thing standing next to it is wrong.

In one of our audits, a pricing engine was checked against a 332-line methodology reference by two independent reviewers with fresh context, plus a separate fact-check of every statement. The engine came out as best practice on every axis checked. The plain metrics layer beside it produced one critical finding and six important ones. The critical one was the headline KPI: what was labeled sell-through counted stock the seller had deliberately held back as sold, and it fed a threshold signal that fires above 0.8.

Check the arithmetic next to the clever part. That is where the money is wrong.

Cases and process at iloblique.com.