Blog

Why every step
needs a source.

A general AI writes text. A maintenance procedure needs proof.
How traceability turns an AI draft into something
an expert can check.

Published
Updated
Reading time
5 minutes
An exploded drawing of a pump with numbered parts under a desk lamp, a tracing sheet on top with one detail marked up in indigo and orange

In short

  • A language model writes fluent text, but fluency says nothing about whether a step, a value or a warning matches the supplier’s manual. NIST lists this as confabulation, a main risk of generative AI.
  • Tracium links every step it drafts to its source: a page of a manual, a table, a row of a parts list. Reviewers verify against the source instead of searching for it.
  • Where sources disagree or a value is missing, Tracium flags the step for a person instead of guessing. An expert approves every module.
  • Traceability checks the content. Validation against S1000D and the project’s business rules checks the form. A module needs both.

Why is a plausible draft not good enough?

Language models write fluent text. Ask one to turn a supplier manual into a removal procedure and the result will read well. Whether each step, value and warning matches the manual is another question, and the text alone will not tell you.

The failure has a name. NIST’s profile for generative AI lists confabulation, “the production of confidently stated but erroneous or false content”, among the main risks of the technology. Research surveys of hallucination in language generation describe the same thing in detail.

Looking things up first does not make the problem go away. A Stanford study of AI tools for legal research, which retrieve sources before they answer, found that each of them still hallucinated in between 17 and 33 percent of the questions tested (Magesh et al., Journal of Empirical Legal Studies, 2025). Their providers had claimed the opposite.

In a maintenance procedure, a plausible step in the wrong order or a torque value from an old revision is worse than no draft at all. Someone has to check every line against the source. Without a link to that source, the check means searching the manual all over again.

Traceability means that every statement in the draft carries the place it came from. Tracium links every step it drafts to its source: page 37 of a scanned manual, table 6 of a PDF, row 14 of a parts list. The reviewer follows the link and checks the step against its source instead of looking for it.

Take one step from the removal of a seawater pump: “Loosen the four bolts of the motor bracket.” In Tracium it links to the passage in the supplier’s manual it was drawn from. If the step and the page say the same thing, the reviewer confirms it. If not, they correct it there and then.

That changes what review is. The expert no longer rewrites. They verify.

What happens when the sources disagree?

Sources disagree. A value is missing. A step only makes sense with a drawing that is not in the documents. Tracium does not fill gaps like these with a guess. It flags the step, and a person decides.

Nothing is published unchecked. An expert approves every module.

Traceability and validation: what is the difference?

Traceability answers one question: where does this come from? Validation answers another: is this valid S1000D? The links check the content. The rules check the form. Every module needs both.

Validation in S1000D means two checks. The module has to conform to the schema of its type, and it has to follow the project’s business rules, which travel as a business rules exchange module, the BREX. Tracium runs both checks on every draft, so the review can concentrate on what only a person can judge.

How do the links pay off after publication?

At an audit, every instruction leads back to the supplier’s own documentation. Nobody has to reconstruct where a value came from.

When a supplier revises a manual, the same links show which modules rest on the pages that changed. Tracium finds every module a revision touches and drafts the update. Because you know which steps rest on which pages, you know what to look at again, and nothing else needs to be touched.

What a reviewer needs to see

A traceable draft puts everything the check needs in one place. For every step, the reviewer should see:

  1. The step as drafted.
  2. The passage it was drawn from, with the manual, its revision and the page.
  3. A flag where the sources disagree or something is missing.
  4. The result of the checks against the schema and the business rules.

With these four, a review is a comparison. Without the second, it is research.

How to test an AI documentation tool

A demo on the vendor’s own examples proves little. These four tests, run on your own manuals, show whether a tool can be trusted with procedures:

  1. Give it a real supplier manual and ask for one procedure. Check whether every step links to a page you can open.
  2. Give it two manuals that disagree on a value. See whether it picks one silently or flags the conflict.
  3. Leave out the page with a key value. See whether it guesses.
  4. Replace the manual with a newer revision. See whether it can tell you which modules are affected.

Questions and answers

Can AI write S1000D data modules?

It can draft them. A language model can read a supplier manual and propose the steps of a procedure in S1000D structure. Whether the draft is right still has to be checked against the source and the rules, so the useful question is how quickly an expert can check it. Links from every step to its source page and automatic checks against S1000D and the business rules make that check fast.

What is an AI hallucination in technical documentation?

A step, value or warning that reads plausibly but is not supported by the source, for example a torque value from an older revision or a step the manual does not contain. NIST calls this confabulation. In a maintenance procedure it is dangerous precisely because it looks right.

How do you check an AI-drafted procedure?

Follow each step to the passage it was drawn from and compare the two. With traceable drafts this is a matter of opening a link. Without them it means searching the manual again, which takes almost as long as writing the procedure.

Does traceability replace the expert review?

No. It makes the review faster and more reliable. The expert still decides, and nothing should be published without their approval.

More from the blog

Contact

Let’s talk.

See Tracium write a data module
from your own manuals.