Enriched in Advance

Manufacturing support runs on context the agent does not have in front of them. A customer calls about a delivery delay. Whether that call should end in an apology, a credit, a warranty claim or an upsell depends on things scattered across systems: what the customer is worth, what they have already complained about this quarter, whether the unit is still in warranty, and whether the last three touchpoints were about the same unresolved issue or three unrelated ones.

Assembling that picture manually takes longer than the customer will wait. So it does not get assembled. The agent answers the question in front of them, the commercial opportunity goes unnoticed, and the fourth call about the same freight classification problem gets treated as if it were the first.

The enrichment turns a delivery delay in March and an invoice dispute in May into one narrative rather than two unrelated records, which turns satisfaction from a survey question into a trajectory you can see moving.

Why this ordering matters. Enriching in advance is what makes the recommendation possible at all. Classifying a customer segment or reconstructing an issue thread is a reasoning task measured in seconds; doing it while somebody waits on the line is not an option. Moving it upstream converts an impossible latency budget into a solved one, and has the useful side effect of making the enriched profile available to every other system that wants it.

A single prompt cannot do this job. Five narrow ones, each with its own tool, can — but only after they have been measured honestly. Each agent has a single responsibility and a single mechanism. Narrow agents are easier to evaluate, easier to fix, and fail in ways you can locate.

The recommendation weighs whether this customer's segment and lifetime value justify a credit, whether the unit is in warranty, whether this is the fourth contact about one unresolved freight classification issue, and whether the sentiment across that thread is recovering or deteriorating.

The 33% is the most useful number in this case study, because it is roughly where a great many multi-agent systems ship.

A ground-truth set of 50 question-and-answer pairs was generated to evaluate against. Then two distinct rounds of judging: first LLM-as-judge, which is fast and cheap and catches the structural failures — wrong retrieval, missing warranty check, malformed response. That loop took accuracy to 56%.

The second loop swapped in subject-matter experts as judges. SMEs catch what an LLM judge cannot: answers that are well-formed and plausible and wrong in a way only someone who knows the product line would notice. That loop took the system to 84%.

The lesson worth stealing. The agent architecture did not change much between 33% and 84%. The evaluation harness did. Most of the distance was covered by generating a ground-truth set, judging against it cheaply, then judging against it expensively — in that order. An LLM judge on its own would have stalled near the middle; SMEs from the start would have burned expert time on errors a model could have caught for pennies.