"Does it learn?" is the question that ends nearly every industrial AI evaluation, and the answer is almost always yes. It is also almost always a different yes than the one the buyer heard. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating costs and unclear business value, and a good share of that gap is the distance between what a vendor means by learning and what an engineering team assumed it meant.
Why does every industrial AI evaluation end with "does it learn?"
Because the buyer has already been through a pilot that made the same mistake twice. Industrial work is repetitive in structure and unique in content: every RFQ is a new document, but the corrections are the same corrections. The same customer part number has to be resolved to the same internal item code. The same clause about witness testing has to be read the same way. The same supplier appears under four spellings. A team that has to re-teach a tool those things on every cycle has not bought a system, it has bought a very fast intern with no memory.
The MIT NANDA study The GenAI Divide: State of AI in Business 2025, widely reported in August 2025 for finding that roughly 95% of enterprise generative AI pilots produced no measurable revenue impact, put the cause in the same place. As Fortune reported, the study's authors located the failure in a "learning gap" rather than in model quality, with lead author Aditya Challapally noting that general-purpose chatbots stall in enterprise use because they do not learn from or adapt to the workflow around them.
What do vendors mean when they say the system learns?
At least five different mechanisms get sold under the same word, and only one of them involves the model changing. Knowing which one you are being offered is most of the evaluation.
Retraining or fine-tuning a model. The weights change. This is what buyers usually picture, and it is the rarest answer in production. It is expensive, it is almost never done per customer, and it produces a system whose behaviour on any single answer cannot be traced to a specific input.
Retrieval over your documents. The index grows, the model does not. Adding last quarter's tenders to a vector store is often described as the system learning your business. It is a bigger haystack. If similarity ranking was returning a superseded revision before, it will return it faster now.
A curated answer library. Response platforms such as Loopio, Responsive and AutoRFP.ai improve as a librarian approves content. That is real and useful, and it is human curation with a search box on top. The system learned nothing; a person did the work and filed it.
Authored rules. Configurators such as Tacton and Configit get better because an engineer encodes a constraint. Deterministic, auditable, and entirely dependent on somebody writing the rule.
Assistant personalisation. Microsoft 365 Copilot and its peers adapt to tone, recent files and calendar context. None of that touches whether a flange rating in a datasheet maps to the right internal item.
What does a real feedback loop look like?
It looks like a correction travelling a short, inspectable path from the person who made it to the next answer, with the model held still the whole way. Four properties separate a loop that works from a thumbs-up button.
- The correction is the edit, not a rating. A thumbs-down records dissatisfaction and nothing else. The useful signal is the specific substitution: this part number resolved to that item, this clause means witness testing rather than notification, this document was the wrong revision. Capture it where the work happens, inside the review, not in a feedback form nobody fills in.
- It is written to a layer you own. Aliases, identifier mappings, unit conventions, clause interpretations, supplier identity. These belong in storage a person can open, read, export and delete. The hard part of industrial document work was never reading the text, it is resolving an external reference to an internal identifier, and that mapping is exactly the kind of knowledge that should accumulate as rows rather than as weights.
- It changes ranking, not just recall. Most retrieval failures in engineering content are authority failures rather than relevance failures: the system found a document that reads correctly and is two revisions out of date. Corrections should teach the system which source wins, which is the same problem as stale revisions in document control approached from the retrieval side.
- The effect is measured, not assumed. Every correction is also a test case. A loop without a growing regression set cannot distinguish improvement from drift, and drift in this setting means a system that is quietly getting worse at cases it used to handle. If a vendor cannot show you how a change was evaluated, the loop is a hope.
The useful mental model is not a model that studies you. It is a mapping layer that accumulates your decisions while the reasoning engine underneath stays fixed and replaceable.
What do regulators and standards say about systems that keep learning?
They treat continuous learning after deployment as a risk to be contained, not as a selling point. That is worth knowing before a vendor offers it as a differentiator.
Article 15 of the EU AI Act addresses it directly. High-risk systems that continue to learn after being placed on the market must, under Article 15(4), be developed so that the risk of biased outputs influencing input for future operations is eliminated or reduced as far as possible, with those feedback loops duly addressed by mitigation measures. The provision exists because a loop that feeds its own output back in is how a system becomes confidently wrong in a consistent direction.
Regulated manufacturing goes further. The EU and PIC/S draft package published for consultation on 7 July 2025, which includes the new Annex 22 on artificial intelligence, scopes critical GMP applications to static, deterministic models: trained rather than explicitly programmed, but not adapting during use, and returning the same output for the same input. Models that keep learning in live operation are outside that boundary. The annex has not been adopted and its treatment of generative models is still unsettled, but the direction is clear enough to plan around.
ISO/IEC 42001:2023, the AI management system standard, points the same way from the governance side: improvement is a documented, monitored management activity, not something a system does to itself between releases. Taken together, the posture across all three is that a change in behaviour needs a record. A loop built around readable rows produces that record as a by-product. A loop built around silent retraining has to manufacture it.
A system that remembers your corrections, in rows you can read
See what cited, revision-aware comprehension looks like when the mapping layer is yours and the model stays fixed.
Where this is going
The agentic wave is about to make this question load-bearing. Gartner's June 2025 forecast came with a second finding that matters more for evaluations: of the thousands of vendors marketing agentic capability, the firm counted roughly 130 it considered genuine, a pattern it named agent washing. An agent that acts rather than drafts compounds its own errors, so the gap between a system that records what it learned and one that merely claims to improve stops being an architectural preference and becomes an operational exposure.
Expect the question to get sharper on the buyer side too. "Does it learn?" is already migrating from a verbal question in a demo into a written one in vendor questionnaires, sitting next to data residency and model hosting, and the answer that survives procurement review will be the one that names an artefact. The useful version of the question is not whether the system learns. It is whether you can read what it learned, reproduce it, and switch it off.
How should you evaluate a vendor's claim that the system learns?
Treat the claim as testable and test it in the demo, not in the pilot. Six questions separate a loop from a feature slide:
- What physically changes when a user corrects an answer: the weights, an index, or a stored rule?
- Where is that change stored, and can you open it, export it and delete a single entry?
- Does a correction take effect on the next query, or at the next release?
- When a correction turns out to be wrong, what is the exact procedure to reverse it, and who can run it?
- How is improvement measured, and can you see the regression set that proves a change did not break earlier cases?
- If the vendor disappeared tomorrow, would the accumulated knowledge still be yours in a form another system could read?
The last question is the one that is asked least and matters most. Knowledge that only exists as an improvement in a vendor's model is not an asset you own. It is an asset you rent.
Key Takeaways
- "Does it learn?" has at least five different answers in the market, and only retraining involves the model itself changing, which is the rarest of the five in production.
- The MIT NANDA study reported in August 2025 attributed widespread enterprise generative AI pilot failure to a learning gap rather than to model quality.
- A working feedback loop captures the specific edit rather than a rating, writes it to a layer a person can read and reverse, applies it to retrieval ranking, and measures the result against a growing regression set.
- Most retrieval failures in engineering content are authority failures rather than relevance failures, so corrections should teach the system which source wins, not just which words match.
- Article 15 of the EU AI Act and the draft EU and PIC/S Annex 22 both treat post-deployment learning as a risk to contain, which favours systems whose changes leave a record.
- Knowledge that exists only as an improvement inside a vendor's model is rented, not owned, and that is the question to ask before the pilot rather than after it.
Self-improving AI in industrial workflows is a real thing, but it is a storage and ranking story far more often than a model story, and the vendors worth shortlisting are the ones willing to show you the rows. For the related question of how an answer proves where it came from in the first place, see citing every answer back to source, or read how Ranger approaches engineered scope in precision manufacturing.
