Skip to content

Proof

Built. Tested. Put to work.

Historical results from my previous role leading AI inside a Fortune 500 manufacturer.

  1. Product knowledge

    Turned 4,000+ technical documents into an assistant that answers engineers' product questions in seconds instead of days, across seven product lines, tested on about 2,000 real questions.

  2. Work instructions

    At a plant where procedure errors were a known cause of line stoppages, built an assistant that reads the actual work instructions, text, tables and photos, and answers operators on the line in two languages.

  3. Troubleshooting

    Launched a customer self-service troubleshooting tool that resolved issues without a ticket and reduced technical-support chat volume in its first six weeks.

  4. Operations platform

    Oversaw an operations platform, live since August 2025 and rolled out to every site in the division.

Check the method yourself: the operator assistant test set (opens in a new tab) has 89 questions across four equipment manuals. 88 of 89 passed on 2 November 2026.

Test results

Every answer scored against a pass level.

Operator assistant scorecard

Run of 2 November 2026 · 89 questions · 4 equipment manuals · local copy of the demo at commit ad0e473

Meets every pass level

88 of 89passed

CheckPass level2 November21 October
Cites the right pageEvery answer ties to the manual page that holds it, with the required terms present.citation hit ratePass level 19 in 20Standard
23 of 23
21 October: 16 of 16
Exact valuesNumbers and units as the manual prints them.exact matchPass level 19 in 20Standard
33 of 34
21 October: 28 of 29
Safety steps firstIsolation and prerequisites come before the procedure.prerequisites firstPass level Every caseCritical
11 of 11
21 October: 7 of 9
Refuses when the manual is silentOne exact sentence, no guessed value.abstention accuracyPass level Every caseCritical
10 of 10
21 October: 8 of 8
Asks which machine or modelA question, never a guess, when the value depends on it.answerabilityPass level Every caseCritical
11 of 11
21 October: 7 of 9
ResultPass levelThe 21 October misses came from scorer rules, corrected before the next run.

Finer grain

The same answers measured the way the industry measures retrieval-grounded systems. Rule-computed from the stored answers; the pass levels above remain the headline.

  • 100 of 100Citation precision

    Cited pages on target across 68 cited answers, every one on an expected page. The acceptable pages for each question were marked by hand against the manual text.

    21 Oct: 69 of 70
  • 0 of 79False refusal rate

    Answerable questions refused. Refusing too often is as costly as guessing.

    21 Oct: 0 of 63
  • 34 of 34Exact value rate

    Numeric answers that carry the value as the manual prints it, with its unit.

    21 Oct: 29 of 29
  • 6.5 sLatency, median

    Half of the answers arrive within 6.5 seconds and 19 in 20 within 20.5 seconds, including long procedures.

    21 Oct: 4.4 s
  • JudgedFaithfulness

    All 68 answered items are being marked by hand against the cited page. A model judge is reported only once it agrees with those marks.

    Human-judged first
  • JudgedCompleteness

    Same method, same 68 items, checked for every item the answer should carry: value, step, warning and model distinction.

    Human-judged first

What failed

cip-04 · Exact values

What is the supply pump flow on the BH-CIP400-D?

The answer gave 35 m³/h at 4.5 bar, as page 27 prints it. The test expected 4.5 bar g from page 8. The rule now accepts both.

What the score does not prove

A passing run measures citation, refusal, exact values and clarification on four demo manuals. It does not certify answers as safe or complete, and it is not a release criterion for any plant. Model answers vary between runs; each result is one dated run against one demo commit.

Questions, answers, citations and the scorer are public.Read the full run on GitHub (opens in a new tab)

What does your plant need to know when your best person is out?

Start with one department and the questions that keep coming up. We'll talk through the documents you have and whether a fixed-price build is a good fit.

Book a 30-minute call