Why Is Your AI Quoting Assistant Getting Off Track over Time? The Key Lies in Feedback

Over the past month or two visiting clients, several owners of small and medium print shops asked me the same thing: The AI quoting assistant and automated LINE customer service bot introduced last year were amazing during initial trials. Why does performance feel stalled now, and sometimes even get worse?

This phenomenon is explained thoroughly in a recent paper titled Scaling Laws for Agent Harnesses via Effective Feedback Compute by Xuanliang Zhang et al. I read the Chinese summary by Wisely Chen

It directly quantifies a counterintuitive fact: You think that giving AI more compute, attaching more tools, and running it more times will make it stronger. It doesn't

The paper uses raw tokens and tool calls to explain task success rate, with correlation coefficients R² of only 0.33 to 0.42

In plain terms for the print shop floor: Turning AI customer service chat logs to maximum detail, increasing quoting recalculations from one to three times, and connecting two more databases, these actions where you feel like you did a lot only explain 30 to 40 percent of the results. The remaining 60 percent has nothing to do with how many resources you burn

I compare this to training an apprentice. A master printer lets an apprentice print 200 test sheets a day, but after printing, never points out flaws or explains where color registration went off. That apprentice could print 10,000 sheets and still stay at the exact same level. He isn't getting better, he's just getting more tired

The core concept of the paper is Effective Feedback Compute, abbreviated as EFC. It means not all interactions count; only effective feedback makes AI actually improve

It defines effective feedback as meeting four conditions simultaneously. Let me match them one by one against printing scenarios:

・Informative: Feedback brings new information. When a client complains that a quote is expensive but doesn't say whether it's paper or post-press finishing, that is useless feedback

・Valid: Feedback must be credible, not noise or wild guesses. If a sales rep casually notes 'this client doesn't care about price' when it's actually the exact opposite, feeding in this wrong feedback is worse than giving none at all

・Non-redundant: Don't repeat what is already known. A system recording 'client wants 100 gsm coated paper' a hundred times contains no new information

・Retained: This one is the toughest. Did the feedback actually feed into the next decision? If a sales rep shares a correct judgment in a chat group but no one integrates it into the quoting logic, it's as if nothing was said

The most critical number is right here: The paper ran a controlled experiment where, under a completely unchanged compute budget, simply improving feedback quality boosted the task success rate from 27% to 90%

Without spending a single extra cent, just making feedback effective caused success rates to jump more than threefold. After recalculation, explanatory power R² went from 0.33 jumping all the way to 0.94 to 0.99

This logic is essentially what learning science has called deliberate practice for decades: Feedback must be specific, correct, and incorporated into the next practice session. Practicing without review, or reviewing without changing, equals no practice at all. AI works the exact same way as humans

Once you understand the principle, the question becomes: How do you actually connect this loop inside print operations? Here are a few actionable steps you can start this week

First, build a standard answer benchmark table. Pick 20 to 30 of the most frequently quoted items over the past six months, saddle-stitched catalogs, perfect-bound books, stickers, paper boxes, and organize exact item numbers, paper types, post-press finishing, and reasonable quote ranges into a ground truth file. Only when the AI's quote fails to match this benchmark do you have a right-or-wrong signal to calibrate against; otherwise, you won't even know when its quotes drift

Second, log every single AI error, down to the root cause. Don't just write 'quoted wrong', write 'it miscalculated 250 gsm paperboard as 200 gsm' or 'it forgot the coating fee.' This corresponds to the Informative condition: specific enough to take action

Third, regularly feed back failure cases. Spend an hour each month taking cases where AI quotes drifted or customer service answered incorrectly, and use them to refine prompts or rules. This step fulfills the Retained condition; whether the feedback loop actually closes depends right here. Floating chat logs don't count, only when they are analyzed and converted into improved rules do they count

Fourth, every time you add a feature, pass it through EFC condition four first. Want to connect one more tool or enable another automated reply? Ask yourself first: Will it actually change the AI's next judgment? If not, adding it just burns cash and bloats maintenance burden

The same goes for design teams. If you use AI to assist image generation, revisions, or proposals, every client revision request is your feedback signal. Note down specifically why the client rejected a version, and avoid those issues in the next proposal, that's how your hit rate goes up. If you just toss rejected files aside without analyzing the causes, revising a hundred drafts still leaves you stuck in place

Some vendors promote memory features like 'AI will remember your company's preferences,' which sounds great. But the paper offers a warning I strongly agree with

A memory architecture solves the fourth condition, retain, the hardest of the four. But it only solves remembering; it won't filter whether the first three conditions are correct or non-redundant

In other words, if you blindly store incorrect, repetitive, and noisy feedback, these bad memories will be recalled and reused repeatedly, making them far more toxic than having no memory at all. It scales 'getting worse over time' from a one-off mistake into something permanent

Therefore, implementing any memory feature must come with a write gatekeeper: Is this piece of information informative enough, credible enough, and non-redundant? Store it only after it passes. For print shops, this means not letting unverified client preferences casually typed by sales reps automatically become system 'facts.'

To be honest, this paper isn't a silver bullet either. That 0.94 to 0.99 upper limit relies on ideal information known only after the fact (the paper calls it Oracle-EFC). Real-world systems can't achieve this, so that is a theoretical ceiling, not a number you can grab tomorrow. Evaluating whether feedback actually changed decisions is also hard to judge in practice. Even with these caveats, I fully buy into the core direction

The future competition among AI tools won't be about who attaches more features or who has longer context windows. It will be about who ensures every piece of feedback is actually used. A good AI assistant isn't about making it do more work; it's like a good master craftsman, ensuring that every step taken actually teaches it something

・Giving AI more compute and tools only explains 30 to 40 percent of performance (R²: 0.33 to 0.42); the remaining 60 percent relies on feedback quality

・Keeping compute unchanged and simply making feedback effective can jump success rates from 27% to 90%, the difference lies in practicing correctly, not practicing more

・Effective feedback must simultaneously be informative, valid, non-redundant, and retained; missing the fourth condition means practicing for nothing

・AI memory features only solve remembering, not filtering out errors. Without a write gatekeeper, bad memory is more toxic than no memory

・Feeding back failure cases from AI quoting and design revisions once a month is the key action to make it sharper over time

For print shops and design studios, the real takeaway isn't whether to adopt AI, but whether you designed a review mechanism after adoption. Most people get stuck at step one, treating tool setup as the finish line. I suggest starting small: Pick a high-frequency scenario, such as catalog quoting or sticker proofing inquiries. First build a 30-item benchmark table, then set aside an hour each month specifically to fix rules using cases where the AI got answers wrong. Once this loop runs smoothly, consider adding memory features or expanding scope. For integrated service providers, this is also an entry point to build long-term retention with clients: If you help clients design their feedback loop properly, the system gets more tailored to their needs over time instead of getting tossed out as inaccurate after six months

・Agents Need Timely Feedback Too: Effective Feedback Compute and Agent Deliberate Practice