NAFEMS reproduction (Morrison blood pump)

Looking for the interactive tool?

This page is the CLI reproduction of the NAFEMS demo. For the browser-based version where you upload your own evidence, see the interactive Gap-Finder.

This page reproduces the rule-engine portion of the live demo presented at NAFEMS Americas 2026 on May 27, 2026. Same evidence packages, same rule engine, same numbers as the slide-16 terminal output.

What you will see

Morrison et al. (2019) is the FDA-co-authored worked example for ASME V&V 40 credibility assessment of a centrifugal blood pump. The same model, mesh, and validation data are evaluated under two different intended uses:

COU	Intended use	Device class	Model Risk Level	Decision
COU 1	Cardiopulmonary bypass support (short duration)	Class II	MRL 2	Accepted
COU 2	Ventricular assist device support (long duration)	Class III	MRL 5	Not accepted

Both COUs are bundled as signed JSON-LD UofA packages. Running the rule engine against them produces the slide-16 terminal output. COU 1 fires 11 weakeners across 5 patterns with no compound risk. COU 2 fires 18 weakeners across 6 patterns including 2 firings of COMPOUND-01 (Critical and High coexistence). The compound inference fires only on the higher-risk COU, even though the underlying model and dataset are shared.

What COU 1’s eleven are made of

The counts above reproduce, but 9 of COU 1’s 11 are vacuous, and a reader should know which before drawing a conclusion from the comparison.

COU 1 references its three validation results as bare IRIs; COU 2 inlines its three as full objects. W-AL-01, W-AR-05 and W-EP-02 each test noValue(?result, …), so against a bare IRI — a node carrying no properties at all — every one of them succeeds for want of anything to read, and fires on every result. That is 3 × 3 = 9 of COU 1’s 11. Inline the same three results, changing nothing else, and those nine disappear.

COU 2’s 18 are substantive; it has none of these. So the COU1-vs-COU2 contrast on the page is partly a difference in serialization, not only in evidence adequacy. The COMPOUND-01 point stands — compound firings are unaffected — but “COU 1 is cleaner than COU 2” is not a reading these numbers support.

Measured 2026-08-16 at catalog v0.5.15.1; both packages verify, and both are profile-conformant before assessment. The rule behaviour is reported, not patched: v0.5.15.1 is frozen, and a guard for these three (mirroring the one W-EP-01 already carries) is a v0.6 candidate.

Reproduce in 3 minutes

Prerequisites: Python 3.10+, Java 17+. If Java is unavailable, use --skip-rules on uofa check and you will still get SHACL and integrity validation.

1. Clone and install

git clone https://github.com/cloudronin/uofa
cd uofa
pip install -e '.[excel]'

The bundled JSON-LD UofA packages and the rule engine ship with the repo. No extra setup.

2. Run the rule engine, COU 1 (CPB use, MRL 2)

uofa rules packs/vv40/examples/morrison/cou1/uofa-morrison-cou1.jsonld --build

Expected: 11 weakeners detected across 5 patterns. Severity breakdown: 0 Critical / 10 High / 1 Medium. Nine of the eleven are the vacuous noValue firings described above — W-AL-01, W-AR-05 and W-EP-02, three each, on this package’s bare-IRI validation results.

Pattern	Severity	Hits
W-AR-05	High	3
W-EP-02	High	3
W-AL-01	High	3
W-ON-02	High	1
W-CON-04	Medium	1

Zero compound inferences. The CFD-side UQ omission is methodologically defensible at MRL 2 (Morrison’s rationale is that bench testing carries the safety determination), but the rule engine still flags it as a potential reuse blocker at higher MRLs.

3. Run the rule engine, COU 2 (VAD use, MRL 5)

uofa rules packs/vv40/examples/morrison/cou2/uofa-morrison-cou2.jsonld

Expected: 18 weakeners detected across 6 patterns. Severity breakdown: 9 Critical / 7 High / 2 Medium.

Pattern	Severity	Hits
W-PROV-01	Critical	7
⚡ COMPOUND-01	Critical	2
W-EP-04	High	6
W-ON-02	High	1
W-AL-02	Medium	1
W-CON-04	Medium	1

COMPOUND-01 fires when Critical and High weakeners coexist on the same UofA. It cannot be expressed as a single SPARQL query because it reasons about the output of the Level-1 rules. This compound inference is the central differentiator over query-based credibility checking.

4. Diff the two COUs

uofa diff packs/vv40/examples/morrison/cou1/uofa-morrison-cou1.jsonld \
          packs/vv40/examples/morrison/cou2/uofa-morrison-cou2.jsonld

Same model, same data, different intended use produces a measurably different credibility evidence profile. The diff surfaces every divergent pattern automatically.

That is the rule-engine reproduction. The next two sections describe how the bundled UofA packages were authored, and an optional pipeline round-trip if you want to exercise uofa extract and uofa import against a worked example.

How the bundled evidence was authored

The two Morrison UofA packages bundled in packs/vv40/examples/morrison/cou{1,2}/ were authored by hand from Morrison et al. (2019). A V&V 40 expert read the paper, mapped the prose evidence into the 13 V&V 40 credibility factors, and serialized that mapping as JSON-LD against the UofA schema.

This is the honest path for a published worked example. The Morrison paper is prose with embedded figures and tables. It does not ship as a structured evidence folder. Going from prose to UofA requires either AI-assisted extraction followed by human review (the future-state on-ramp) or hand authoring (what was done here).

(Optional) Round-trip the extract pipeline

If you want to see the uofa extract and uofa import pipeline mechanics, the repo also ships a synthetic evidence folder at packs/vv40/examples/morrison/source/. The folder contents are not from the Morrison paper. They were generated from the hand-authored UofA, structured as the kind of artifact you might find in a real engineering team’s evidence directory.

ls packs/vv40/examples/morrison/source/
# EVIDENCE_MANIFEST.txt
# ansys_cfx_solver_settings.txt
# boundary_conditions_specification.txt
# cou_definitions.docx
# credibility_assessment_narrative.docx
# decision_rationale_cou1.pdf
# decision_rationale_cou2.pdf
# hemolysis_mih_comparison.csv
# mesh_convergence_study.csv
# monte_carlo_uq_cou2.csv
# piv_velocity_validation.csv
# risk_assessment_memo.pdf

What this demonstrates and what it does not

Running uofa extract against this folder and comparing the result to the bundled UofA is a round-trip sanity check on the pipeline. It demonstrates that the extract, review, import sequence executes cleanly and produces a UofA equivalent to the hand-authored ground truth. It does not measure extraction performance against real-world prose. The synthetic folder is structured against the V&V 40 schema; real engineering documentation is messier. Phase 3 of the praxis runs extraction against prose-only PDFs to produce a real extraction-performance number.

To exercise the round-trip:

uofa extract packs/vv40/examples/morrison/source/ \
  --pack vv40 \
  --model ollama/qwen3.5:4b \
  -o morrison-roundtrip.xlsx

# Open morrison-roundtrip.xlsx, verify factor levels and decision text

mkdir -p keys
uofa keygen keys/morrison
uofa import morrison-roundtrip.xlsx \
  --pack vv40 \
  --sign --key keys/morrison.key \
  --check

The default model is ollama/qwen3.5:4b, which runs locally. To use a hosted provider, substitute any LiteLLM-compatible model identifier (gpt-4.1, claude-sonnet-4-7, gemini-2.5-pro).

(Withdrawn) The NASA-STD-7009B HPT-blade step

This step used to appear here as a cross-domain reproduction. It was not one, and it has been withdrawn.

The two files it pointed at — uofa-aero-cou{1,2}-nasa7009b.jsonld — are weakener reports, not UofA packages. Their @graph holds only stored WeakenerAnnotation nodes (17 and 20 respectively) with no UnitOfAssurance, and 24 of the catalog’s 29 rules open on (?uofa rdf:type uofa:UnitOfAssurance). So uofa rules against them printed the 17 and 20 annotations already serialized in the files. Nothing was inferred: strip the stored annotations and the engine reports 0 weakener(s) detected.

Placed directly after the Morrison section, where the same command does perform live detection, the step invited exactly the wrong reading. That is why it is withdrawn rather than relabelled.

No encoded HPT package exists in this repository. The source evidence is present as packs/nasa-7009b/examples/aerospace/aero-evidence-cou{1,2}.zip, and building a genuine cross-domain package from it is future work, not a demo step.

Source-data summary

Artifact	Path	Origin
Morrison COU 1 JSON-LD	packs/vv40/examples/morrison/cou1/uofa-morrison-cou1.jsonld	Hand-authored from Morrison et al. (2019)
Morrison COU 2 JSON-LD	packs/vv40/examples/morrison/cou2/uofa-morrison-cou2.jsonld	Hand-authored from Morrison et al. (2019)
Morrison source folder	packs/vv40/examples/morrison/source/	Synthetic; generated from the hand-authored UofA for pipeline-mechanics demonstration
HPT blade weakener reports	packs/nasa-7009b/examples/aerospace/uofa-aero-cou{1,2}-nasa7009b.jsonld	Stored WeakenerAnnotation records, not UofA packages — they carry no UnitOfAssurance, so the rule engine cannot assess them. See the withdrawn step above
HPT blade evidence zips	packs/nasa-7009b/examples/aerospace/aero-evidence-cou{1,2}.zip	Synthetic; same pattern as the Morrison source folder
Original Morrison paper	DOI: 10.1007/s13239-019-00420-7	Morrison et al. (2019), Cardiovascular Engineering and Technology

Run uofa on your own evidence

The future-state on-ramp for a practitioner with their own simulation evidence:

1. Drop your source documents into a folder
2. Run uofa extract <folder> to produce a structured workbook
3. Review the workbook (this is where the human catches what the LLM missed)
4. Run uofa import <workbook> --sign --check to produce a signed UofA
5. Run uofa rules <uofa.jsonld> to surface credibility weakeners

No data leaves your environment. The CLI is fully local. The optional uofa extract step supports local LLMs (Ollama qwen3.5:4b is the default).

See Get Started for installation details, and the Excel on-ramp for a fuller walkthrough of the import-side workflow if you would rather author the workbook yourself.

Feedback

If you scanned this from the NAFEMS room, tell me what would make UofA useful for your pipeline. Even a one-sentence note is useful.