Why Is Your AI Quoting Assistant Getting Worse Over Time? The Key Is Feedback

Over the past month or two, I've visited several small and medium-sized print shops. Their owners all asked me the same thing: The AI quoting assistants and automated LINE customer service bots they set up last year were impressive at first, but now they feel stuck, and sometimes they make even worse mistakes

A recent paper titled 'Scaling Laws for Agent Harnesses via Effective Feedback Compute' explains this clearly. The authors are Xuanliang Zhang and others, and I read the Chinese summary by Wisely Chen

It directly quantifies something counterintuitive: you think giving the AI more compute, more tools, and more runs will make it stronger. It doesn't

The paper uses raw tokens and tool calls to explain task success rates. The correlation coefficient R² is only:

・0.33 to

・0.42

In plain print-shop terms: if you expand the AI customer support conversation logs to the highest detail, increase quote recalculations from one to three times, and connect two more databases, these 'I did a lot' actions only explain about 30% to 40% of the results. The remaining 60% has nothing to do with how many resources you burn

I compare this to training an apprentice. If a master makes an apprentice print 200 practice sheets a day but never points out mistakes or explains why colors are misaligned, that apprentice will stay at the same level even after printing 10,000 sheets. They aren't more skilled. They are just more tired

The core concept of the paper is Effective Feedback Compute, or EFC. It means that not all interactions count, only 'effective feedback' makes the AI actually improve

It defines effective feedback as satisfying four conditions at the same time. Let's look at them through a printing lens:

・Informative: Feedback must bring new information. If a customer complains that a quote is too expensive but doesn't specify if it is the paper or the post-press finishing, that feedback is useless

・Valid: Feedback must be reliable, not noise or guesswork. If a sales rep casually notes 'this customer doesn't care about price' but actually got it completely backwards, feeding this error in is worse than giving no feedback at all

・Non-redundant: Don't repeat what is already known. Recording 'customer wants 100 lb gloss coated paper' a hundred times adds no new information

・Retained: This is the most demanding rule. Did the feedback actually make it into the next decision? If a sales rep shares a correct insight in a group chat but no one updates the quoting logic, it is as if it was never said

Here is the key number: the paper details a control experiment where, without changing the compute budget at all, they only improved the quality of feedback. The task success rate jumped from 27% to 90%

Without spending an extra dime, just by making feedback effective, the success rate tripled. Recalculating the numbers, the explanatory power R² shot up from:

・0.33 all the way to

・0.94 to

・0.99

This is exactly what learning science has called 'deliberate practice' for decades: feedback must be specific, correct, and applied to the next attempt. Practicing without reflecting, or reflecting without changing, is useless. AI works the same way

Now that we understand the theory, the question is how to actually connect this loop in print workflows. Here are a few actions you can start taking this week

First, create a 'ground truth' reference table. Look at the 20 or 30 most common items you quoted over the past six months, such as saddle-stitched catalogs, perfect-bound books, stickers, and paper boxes. Organize the correct item numbers, paper types, post-press finishing, and reasonable price ranges into a master reference list. If the AI's quote doesn't match this list, you have a clear error signal to correct. Otherwise, you won't even know when it goes off track

Second, record every AI error and trace it to the root cause. Don't just write down 'quoted incorrectly.' Write down 'it calculated 250 lb cardstock as 200 lb' or 'it forgot to include the cost of varnish.' This matches the Informative rule: keep it specific enough to act on

Third, feed failure cases back into the system regularly. Spend one hour each month taking the quotes that went off track or the customer support questions the AI got wrong, and use them to refine its prompts or rules. This step is Retained, the key to closing the feedback loop. Logged conversations do not count unless they are organized and used to improve the rules

Fourth, run every new feature through EFC's fourth rule. Before adding another tool or setting up another auto-reply, ask yourself: will this actually change the AI's next decision? If not, adding it will only waste money and increase your maintenance burden

The same goes for design. If you use AI to help generate images, revise drafts, or write proposals, each round of client feedback is your training signal. Note exactly why the client rejected a draft and avoid those mistakes in the next proposal to raise your success rate. Leaving rejected drafts in a folder without analyzing the cause will keep you stuck in place, even after a hundred revisions

Some vendors promote features like 'the AI will remember your company's preferences.' It sounds great, but the paper offers a warning that I strongly agree with

Memory architectures solve the fourth and hardest rule, 'retain,' but they only solve storage. They do not filter whether the information is correct, informative, or non-redundant

In other words, if you dump wrong, repetitive, and noisy feedback into the memory, these bad memories will be recalled over and over. The toxicity is worse than having no memory at all, turning a one-time mistake into a permanent error loop

So before introducing any memory feature, you must set up a write gate: is this information informative, reliable, and non-redundant? Save it only if it passes. For print shops, this means preventing unchecked customer preferences jotted down by sales reps from automatically becoming facts in the system

To be honest, this paper is not a magic cure. That:

・0.94 to

・0.99 limit relies on perfect, after-the-fact information (what the paper calls Oracle-EFC). Real systems cannot achieve this, so it is a theoretical ceiling, not a number you can hit tomorrow. Also, judging whether feedback actually changes a decision is hard in itself. Even with these caveats, I still buy into the core direction

The future of AI tools will not be about who has the most features or the longest chat history. It will be about who can make sure every piece of feedback actually gets used. A good AI assistant is not one that does more work, but one that, like a good apprentice under a master, learns from every step it takes

・Giving AI more compute and tools only explains 30% to 40% of the results (R²:

・0.33

・0

・42), while the remaining 60% depends on the quality of feedback

・With compute unchanged, simply making feedback effective can boost success rates from 27% to 90%. The difference lies in practicing right, not practicing more

・Effective feedback must be informative, correct, non-redundant, and retained. Missing the fourth condition makes practice useless

・AI memory features only solve storage and do not filter errors. Without a write gate, wrong memories are more toxic than no memory at all

・Feeding failed AI quoting and design draft cases back into the system once a month is the key to making it increasingly accurate

For print shops and design studios, the real takeaway is not whether to adopt AI, but whether you have designed a review system after setting it up. Most people get stuck at the first step, treating integration as the finish line. I suggest starting small: pick a high-frequency scenario, like catalog quoting or sticker proof inquiries, and build a 30-item ground truth list first. Then, set aside an hour each month to refine rules based on where the AI made mistakes. Once this loop runs smoothly, you can think about adding memory features or expanding the scope. For integration service providers, this is also a way to build long-term client retention. If you help clients design their feedback loops, the system will become more tailored to their needs over time, rather than being discarded as inaccurate after six months

・Agents Need Real-Time Feedback Too: Effective Feedback Compute and Agent Deliberate Practice