model="gpt-4-32k" is not an answer to that question. Neither is
a _test.go file naming twelve providers to prove a detector fires.
Left unhandled, this is not a cosmetic problem — it is most of the document.
Scanning AIROM’s own repository produced 185 components, of which 180 existed
only in test scaffolding. The published bill of materials read as if a Go
scanner that calls no models at all depended on fifty hosted LLMs.
What counts as test scope
A path is test scaffolding when it matches an unambiguous, language-standard convention: | Directories |testdata/, tests/, test/, spec/, __tests__/, __mocks__/, __fixtures__/, fixtures/, e2e/, integration-tests/, testutil/, testutils/ |
| Go | *_test.go |
| Python | test_*.py, *_test.py, conftest.py |
| JS / TS | *.test.ts, *.spec.tsx, *.test.js, … |
| Ruby | *_spec.rb, *_test.rb |
| Java | src/test/…, ChatTest.java, ChatTests.java (case-sensitive — Latest.java is not a test) |
| Rust | *_test.rs |
Two rules keep this from doing damage:
Matched relative to the scan root
Point AIROM at a fixture directory —
airom fs ./testdata/rag-app — and it
reports everything normally. You asked about that tree; nothing inside it is
“the tests” of anything. Only paths that sit under a test directory of the
scanned project are scoped out.spec/, fixtures/, and e2e/ are the three names most likely to hold real
code in a project that does not follow the convention they come from. If yours
does, point AIROM at the subtree directly (airom fs ./e2e) — matching is
root-relative, so everything inside is reported normally.Never a substring match
src/testimonials.py, models/latest/, src/contest/, and
internal/attestation/ are all production code. Only exact, conventional
names count — a false positive here would hide real AI from the default view,
which is the more expensive mistake.Reached from production code? Then it is production AI
The rule is all, not any. A component is test-scoped only when every occurrence is test scaffolding.app.py
tests/test_app.py
gpt-4o-mini is production AI that happens to be tested, so it stays in the
report. Scoping it out because a test also touched it would suppress exactly what
the document exists to record.
Recorded everywhere, shown selectively
Nothing is ever silently dropped. What changes is which surface counts it:
For the table and SARIF the filtering is per occurrence, not just per
component: a real dependency that also appears in fixtures keeps its place in the
report but stops planting alerts inside
testdata/.
Including them
Not the same as data-about-AI
Test scope is decided automatically. The neighbouring problem — a file whose AI names are data rather than dependencies, like a provider routing table or a docs page listing supported models — is not decidable by static analysis: the identical table in a gateway that routes to those providers is a true finding. That judgement belongs in.airomignore, with the people who
know the answer.
AIROM’s own rule packs and lifecycle catalogs are the one
exception, because they have an unambiguous structure: they are recognised by
shape and never inventoried as prompts, in your repository as well as ours.
Test-scoped components are also skipped when querying OSV.dev, so the
CVE overlay does not spend network round-trips on advisories
for packages nobody will be shown. The scan says so in its warnings rather
than leaving an empty
vulnerabilities list to be read as an all-clear.