BootUI 2.0 validation report

This report checks whether Runtime Insights is worth reading on applications that were not written for BootUI (v2 plan §2.2 and §2.4). It is filled in from an early-adopter build and rerun before 2.0.0.

The validation harness (validation/), the rerun's raw evidence (docs/validation/), and the adjudication file were removed from the repository after the rerun; they remain in Git history and in the maintainer's archive. Paths to them below are historical.

Draft, first run (2026-10-03), for the maintainer's review. Built from v2 at fb4cc07e5 (version 1.19.0). Every application ran with its own Maven repository seeded with that build, and each run proved it used it: the runtime-insights endpoint answered, and the resolved bootui-engine jar's SHA-256 matched the local build. Two reviewers on different models judged every observation independently, with the application's source at hand; the operator who ran each application gathered facts and code pointers but judged nothing.

Rerun (2026-10-05), scored. M4-20's rerun ran under the registered protocol (m4-20-protocol-2) on one build of 5bd7cb76e; two reviewers judged it, the maintainer adjudicated it, recall was marked under his rule with his approval, and the registered scorer computed the result. See Rerun results and the adjudication file. The first run's sections below are kept as they were.

Release gates, first run

The external-validity result also trips §2.3's gate after M3: fewer than 50 % of external-application observations were judged useful.

Supplementary implementation audit, 2026-10-09 to 2026-10-10

This later pass examines implementation defects, missing regression coverage, and inconsistencies before release. It does not rerun or rescore the registered external-application study below. Its starting source, 32f3a614289f0afdb43249c985212c3a7c17468d, passed the Java 17 baseline and JDK compatibility run. Green workflows did not establish observation usefulness, a universal overhead bound, or release authorization.

The audit followed shared engine behavior through the adapters and browser, MCP, and CLI consumers. Independent reviews covered the Java agent, engine/adapters, and MCP/CLI; direct review covered UI state, documentation, release guards, and actual executed and skipped CI steps. This was a bounded, evidence-driven pass, not a claim that every changed line received independent review. Its 19 confirmed implementation findings were resolved in the eight focused PRs below; the remaining evidence qualifications and human release decision are separate.

Integration evidence and its limits

Some reviewed PRs became conflicting only because another reviewed fix had landed. Their mechanical conflict merges used [skip ci] under the established branch policy, with unchanged owned source, independently validated inherited fixes, and a targeted check of the resolved composition. These mechanical heads have no replacement-head full CI. The acceptance evidence is explicitly the original tested source plus byte equivalence and resolved checks, not an invented workflow on the later head.

The separate pristine-head checks for #1415 and #1416 encountered database-image-fetch timeouts before affected test bodies ran. Each terminal workflow received one specifically authorized failed-job-only retry. The retries passed, including previously blocked live database and downstream conformance tests; already successful jobs were not rerun. The original bootstrap failures remain recorded rather than being described as successful test execution.

The agent's local forked tests used explicitly capped child JVMs, including its JDI launch. Those validation-only helper changes were restored byte-for-byte before delivery; the production agent jar was unchanged. The 53-case local result therefore does not replace the pristine-head Java 17 CI result.

Starting-source measurements

The starting source 32f3a614289f0afdb43249c985212c3a7c17468d also supplied these completed measurements in run 37970180730. They are supplementary source-pinned results, not amendments of the registered study or measurements of the later combined fixes.

Final combined-source evidence

The SQL follow-up landed as f957c930899b2eb634575afc55dc2634e726b806. Its tree, 6cbab38148ecf3c1d7cdde9c2af61931d9dd7524, is exactly the tree CI actually checked out at 377a252d894c7c7b5b59dde06bd7c57691049748, combining f8769f158e7634d5e8fa3a19707f5551ac510661 and the reviewed SQL follow-up. The following evidence therefore covers that landed implementation tree, not just the PR's head SHA. It does not include this later documentation-only report edit or designate an approved release candidate.

The Java 17 baseline completed 19 successful jobs and one configured placeholder skip. The JDK compatibility run completed six successful jobs, including Java 27, and the documentation run passed. Across the PR's 27 contexts, 26 succeeded and one was skipped. These job/context totals do not turn skipped test bodies into executed tests.

Combined-source agent measurements. The baseline run above measured source 377a252d894c7c7b5b59dde06bd7c57691049748 on Java 17.0.20.1 with four processors, 16 clients, 10-second warm-up and 15-second samples. Both benchmark verdicts use gate=median and enforced=false.

No statistically established improvement over the earlier measurements is claimed. The historical I/O miss of 12.0% and starting-source journal miss of 10.1% remain Not met on their stated sources. The combined-source JournalOverheadBenchmarkIT was skipped: journal throughput was not remeasured on this tree.

Read-only release rehearsal. The coordinator's rehearsal on actual CI source 377a252d894c7c7b5b59dde06bd7c57691049748, against main at 56a114, reported 35 passing checks, two pending checks and one informational check. The temporary candidate was unreferenced, cleanup completed, and no release or repository state was changed. Human release approval and 1.x Maven Central publishing permission remain pending; the Pages policy remains informational. This was not a live rehearsal or release authorization. The preserved raw log's SHA-256 is 761c63030c7a2f35255dfb9913242f128f5d0607068ed72672e9fc4c61e65c68.

Remaining qualifications

• Java 27 early warning. On source def556cab31eacdeb34fe94281f5caef9c98db90, job 114068068098 genuinely failed ResourcesBehaviorsIT.everyResourcesBehaviorPassesBeforeOpenTelemetry: the pooled JDK HttpClient connection's required library-owned LEFT_OPEN record was absent. The child exited normally and resource self-tests passed. This historical failure remains unexplained, not labelled infrastructure or a confirmed flake. Java 27 is informational under the unchanged compatibility policy; workflow success does not make this failed job pass. The exercised files/network/resources claim does not enable the application-method transformer changed by #1419. On the combined source 377a252d894c7c7b5b59dde06bd7c57691049748, Java 27's ResourcesBehaviorsIT ran all five cases without failures/errors/skips. That later pass does not retroactively pass or explain the failure.
• Workload-specific overhead miss. Source a966d64258b012ddbff4d772c0d566c83cf1d510, run 37996802939, measured the I/O route with 16 clients, nine alternating pairs, 10-second warm-up and 15-second samples on Java 17.0.20.1 with four processors. Median paired throughput was 88.0% of the no-agent run: 12.0% overhead against a 10% budget, not met, with a 96.1% distribution-free interval of 6.0-18.5%. Median p99 was 40.59 ms with the agent and 34.62 ms without it. This nonblocking benchmark failure (enforced=false) remains a failure despite a successful workflow; it is not a universal estimate or a measurement of the final combined candidate.
• SQL first-head failures remain recorded. #1420's first head, 1f9398dc94aff57dc18b0d525fbc8f9e35fea76f, genuinely failed the preparation fixture in whole-engine runs on Java 17, 21, 25, and 27. Two Quarkus browser assertions also failed, exposing preparation-only profiles' healthy-zero SQL copy. The 610 targeted tests and one genuine codec VERSION 14 binary fixture (1,515 bytes, from source 89d50, hash prefix c2ca7edc) are narrower evidence, not substitutes for corrected-head whole-engine, conformance, browser, and consumer CI.
• Release authorization remains pending. The combined implementation's executed evidence is recorded above, separately from earlier sources, skipped measurements, and the unchanged registered study. Human approval, candidate designation, and the remaining release prerequisites are not supplied by green workflows or this audit. The Release sign-off fields below remain PENDING/TODO.

Fresh three-model pass and verified corrections, 2026-10-10

GPT-6.1 Sol, Claude Sonnet 5.5, and Grok 4.7 independently reviewed the entire product at e3c10a468d675a9b5a89fbaee63e52b30bb68e3e, tree 7e0f12f31fefe9f10018bb80a82d68248af80b36. Their source findings were adjudicated against actual producers, documented contracts, and deterministic counterexamples; disagreement was not resolved by majority vote. This pass neither reruns nor rescores the registered external-application study, and does not supply release approval.

The initial reports raised eight candidate groups. Two were withdrawn after their premises were checked: Spring's SQL tracing producers record executions only, so the alleged preparation-only Spring profile is not reachable; and the journal clear acknowledgement deliberately counts retained events, not queued offers. Neither claim justified a production change. Six retained correctness groups were repaired:

These are correctness/workflow priorities, not CVSS scores. Native-scope and owner-slot variants are not counted as extra roots. The fixes do not add public DTO fields, change supported adapter capabilities, enable sensors, retry mutations automatically, or alter performance targets.

Reproduction and validation

The tracker counterexamples ran against the exact reviewed bridge sources with two positive same-generation controls. Both stale-generation assertions failed before the repair, and the unchanged probes passed after it. The owner-slot reader's stale stop was independently reproduced on the same starting source. A real Micrometer ObservationRegistry reproduced both ambient-request and initially-unowned worker leaks; the corrected test also covers two workers sharing one observation context.

The malformed-baseline reproduction used an isolated, processor-capped JVM with a 64 MB heap. The invalid count produced an immediate VM array-size-limit rejection, not a deliberate large allocation. The same 45-byte file is now ignored with a reason. Collection and text bounds, overflowed varints, histogram indexes, and genuine legacy summaries are covered separately. Codec version 15 and the genuine earlier-version fixtures remain unchanged.

The shutdown regression uses the real periodic drainer and transport ring. It orders publication while the claim is armed, observes disarm, and verifies one final delivery, accurate counts, idempotent closure, and refusal to drain a replacement owner's records. An earlier unstarted-worker fixture was a harness failure, not product RED.

The final targeted Java classes executed 199 cases with no failures, errors, or skips, each through its own actual jvmRun1 on JDK 26. The missing ThreadActivityTests selector was recorded and not suppressed; existing thread-tracker, side-effect, resources and code-paths classes exercise that integration.

The UI regressions ran against the eight owned production originals: 65 failures, 11 passing controls, and eight unhandled errors caused by the original malformed-report rendering and missing catches. Those errors were not counted as additional defects or silently labelled harness noise. The corrected selection passed 167 tests across nine files with no unhandled errors, and its typed build and owned formatting checks passed. The final assertions are the formatting-normalized equivalent of the RED assertions.

The first published head, 573c316513eb9aa128dd37be1a76c8a473e1ca8f, genuinely failed the bridge bytecode guard on Java 17, 21, 25, and 27: its initial synchronization introduced forbidden JVM monitors. The correction uses an immutable generation/reader snapshot published by compare-and-set, retaining the stale-generation checks without changing or weakening the architecture guard. A bounded review found no blocking defect in that repair. The subsequent 13 selected bridge classes executed 238 cases without failures, errors, or skips, including the unchanged bytecode guard and the additional earlier-claim regression; these are narrower than whole-source CI.

The next head, 00fa39850aaa681f7ed7ec4308b733a40a4bd504, passed those bridge rules but failed Spring's thread-local architecture guard on Java 21, 25, and 27: the new scheduled scope holder used a raw ThreadLocal. It now uses the existing BootUiThreadLocal marker so the agent does not report BootUI's own correlation bookkeeping as application thread-local state. The guard remains unchanged; nested restoration and removal of empty holders retain their behavior. The eight scheduled-scope cases and both Spring architecture checks then executed without failures, errors, or skips on the corrected source, each through its own jvmRun1.

The first head's browser runs also exposed two fixture errors: unavailable capture reports correctly offer a disabled Resume, not Pause, and WebSockets show the reason both in their static availability notice and their single action alert. The scenarios now assert the actual native state and target the semantic action alert, without changing production UI or weakening refusal/state assertions. The Java 21 agent browser additionally recorded one request-value sink assertion failure before its configured retry passed; that original failure remains recorded, not labelled infrastructure or explained by these fixture corrections.

The shared browser scenario adds 19 cases per runtime. Actual Playwright discovery registered 28 cases on each of MVC, WebFlux, and Quarkus, including the nine previously delivered cases. Discovery is not execution: native capability guards preserve WebFlux's absent Hibernate integration, metadata-only WebSockets, Quarkus's unavailable transactions, and its read-only configuration. Every test mutation is intercepted rather than sent to the app. Discovery is not counted as execution; the final source's actual browser results are recorded below.

The documentation checks passed: 11 workshop cases, the 66-page production build, installer publication and shellcheck, three fragment/link cases, and seven changelog cases. The historical release sign-off and registered study below remain byte-identical; neither the implementation corrections nor these checks amend their results.

The required main-into-v2 synchronization used a merge commit, not a rebase. Its two inherited changes were the Windows skill-documentation path test and Docker-free Quarkus demo configuration; the plans, dependency versions, and reviewed runtime code were unchanged by that synchronization.

Final corrected-source evidence

The corrected implementation is 93f1abcc4b8b04fdd1c94e5b4c40b8e3f50f66e8, tree a6d4ae4fa1dce1ebcaf986ebf9d595540c1c5a81. Its Java 17 baseline completed all 37 jobs successfully, and its JDK compatibility run completed all six jobs successfully. The evidence below comes from their actual source checkouts, logs, and JUnit artifacts, not from an earlier head's green status. This later report-only documentation update is not part of that tested tree.

The source-pinned CI artifacts and original failed-head logs are preserved separately from the local RED/GREEN controls. The first-head monitor violation, the later raw-thread-local violation, and the original browser fixture failures remain failures on their stated sources; this later success does not retroactively pass them.

Remaining acceptance qualifications

The reviewed integrated source's baseline run genuinely failed the journal-insight read-time assertion: 2,289 ms exceeded the test's 2,000 ms limit. This is a performance failure, not infrastructure or a passed gate; the user excluded performance optimization from this correctness task, and no workload or threshold was relaxed. Earlier source-specific throughput misses and the registered study's results remain on their original sources.

The final source's automatic report-only benchmarks also retained failures: journal overhead was 9.7% against 5%; default-agent I/O 13.7% against 10%; resources cumulative 10.1% against 10%; request-value matching cumulative 13.1% against 10%; and thread-activity cumulative 15.4% against 10%. The checks and request-value A/B legs each reported 3.1% against 3%, and thread-activity A/B reported 13.7% against 10%. These are failures under their individual recorded gates, not budget acceptance supplied by the successful workflow. No statistically established improvement, performance remediation, changed default, or relaxed workload/threshold is claimed.

The three reviews and executed regressions, browser scenarios, and native CI do not exhaust every advisor, database, browser, or framework path. The correctness task is complete on the recorded source; performance acceptance, human sign-off, release-candidate designation, main-merge authorization, and administrative prerequisites remain separate and unchanged.

Fourth independent three-model pass, 2026-10-10

At the maintainer's request, GPT-6.1 Sol, Claude Sonnet 5.5, and Grok 4.7 performed a new independent source review of 84e3c1fca7675baaf41a06b7c84d5632fcef03d1, tree 0e5eb8c8553c9cf70a897b83297a8e5f28e0c54f. Live main at bd46bf819e9e75282b849184e058090d9c769219 was fetched and already included; the explicit merge reported up to date. No redundant merge or plan/version change was needed.

Sol reported four candidates, Sonnet four after a bounded clarification, and Grok no additional high-confidence finding. Their reports describe their source-only coverage and limits; none independently ran the application. The parent adjudicated candidates against native producers and controlled reproductions, not majority vote. Two claims were rejected: MCP/CLI action markers match the documented panel-only contract, and the proposed dynamic cache issue lacked a verified live-state-change counterexample. Sonnet also withdrew an Exceptions triage race, but the parent's native-acknowledgement regression demonstrated that race with auto-refresh off.

These are correctness priorities, not CVSS scores. Shared task-count wording and unique rendering keys belong to the native-handoff root, not extra defects. No public DTO fields, transport schemas, sensor defaults, dependency versions, performance thresholds, or release permissions changed.

Controlled reproduction and local evidence

The real handoff producer's capture/reopen/close path yielded four SQL timeline rows from two retained SQL events before correction. The CPU collector-latched closure yielded attribution 200 for CPU 100. Two distinct value-equal exceptions yielded one occurrence. Native Quarkus close handling produced no exchange when one was required. Blank-source tests failed in both the shared engine and the real Spring binder. Each of those six original Java class runs had one assertion failure, no errors, and no skips.

An initial fixture incorrectly tried to access a package-private journal constructor and dispatcher from another test package; those compile failures are harness failures, not product RED. The corrected fixture uses the public journal and its bounded drain wait. Actual failed XML and logs are retained separately.

The three owned UI views had 39 failing assertions, 35 passing controls, and one genuine unhandled rendering error on original production bytes. Identical final assertions passed all 74 cases after correction, with no unhandled errors. Two additional parent regressions for triage read ordering and shared handoff wording failed with 17 positive controls before correction. The composed five-file selection then passed all 93 cases. A backend control batch stopped before its next command when the one-minute system load exceeded 50; no test failure is claimed from that safety stop.

The final formatted local selection executed 231 Java cases across 12 individual class runs with no failures, errors, or skips, each through its own jvmRun1. The composed UI selection passed all 93 cases, its typed check and build passed, and root Spotless plus all three frontend/browser formatting checks passed. Native Spring and SmallRye blank-property tests preserve explicit source lists. Playwright discovery registered 44 cases on each of the three default stacks, including 16 additions per stack; discovery does not establish their execution.

The first published fourth-pass head, f18158d140b2ba81c160de538328d0333edd60ad, exposed browser-fixture errors in real WebFlux execution: migration reports carry flywayPresent or liquibasePresent, not an available field, and the paused-clock navigation fixture did not advance the hash-router transition before checking unmount. The fixture now checks the native DTO discriminator and advances that transition, explicitly asserting that the resource profile is absent before releasing its old response. Production code and the no-retry/lifecycle assertions remain unchanged; the original failing run is retained, not labelled infrastructure or product RED.

The first fixture follow-up, 59d5ebca81d3faf0618b15762f747610cb06e9b6, fixed migration registration but still failed the three unmount fixtures: the trace showed Health assets successfully loaded while the paused clock left the shell's out-in page transition unfinished. The fixture tried native sidebar navigation while resuming the clock until Health renders and the profile is absent, then pauses again before releasing the pending response. This validates component disposal rather than a frozen transition; no production navigation code changed. The subsequent 0f3945025 and 840b93dd8 fixtures still waited for a nonexistent exact accessible name: the sidebar's native aria-label can include availability details. The native sidebar link was also hidden in a collapsed group on e160a6d58. The fixture now changes the hash route with the clock already resumed, independently of sidebar expansion, and asserts Health plus component disposal before releasing the old response. These fixture failures are not product correctness findings.

Before publishing that final fixture correction, the current WebFlux sample was packaged using the already-built verified UI, explicitly activated on an isolated loopback port, and tested locally: all three unmount cases passed, then all 44 diagnostic scenarios completed with 39 passes and five native capability skips. The first local launch left BootUI inactive and returned 404 before test bodies; it is a harness activation failure, not product RED. Both local sample processes were stopped after the bounded check. The f3e832014 agent browser exposed one additional selector ambiguity: the optional caught-in-code table also contained the same request path. The triage fixture now selects the row with the native Change status controls, not unrelated caught evidence. Its status/read-order assertions and production code remain unchanged.

Final fourth-pass acceptance evidence

The final tested source is 33eadcee6bb4631d02778eca46d8e0b02a93fb0e, tree 67a86d444e859304e3a01858150c6c7ad4bf5999. Its Java 17 baseline completed all 37 jobs successfully; the JDK compatibility run completed all six jobs successfully. Actual checkouts, logs, and XML were inspected. This later report-only update is not part of that tested tree, and the earlier 93f1abcc4 green result is not used as a proxy.

The Java 21 agent browser's request-value sink fixture initially observed one matching sink where two were required; its configured retry passed. That original assertion remains recorded and unexplained, not labelled infrastructure or erased by workflow success. No diagnostic acknowledgement case failed on the final source.

The automatic benchmark artifacts retain budget misses on this source: default-agent 10.7% against 10%, journal 10.7% against 5%, resources cumulative 10.9% against 10%, thread-activity cumulative 14.6% against 10%, and thread-activity A/B 22.5% against 10%. Environment A/B recorded 3.2% against 3% under its interval gate. Files A/B recorded 3.7% against 3%, with an interval of -0.5% to 6.1% and enforced=true; its actual benchmark test passed under the unchanged 30% fail-above ceiling, which does not pass the 3% budget. A green workflow is not budget acceptance, and no performance threshold, workload, sensor default, or budget was relaxed by these fixes.

The nine confirmed correctness roots are repaired and their applicable current-source gates have executed. Original product RED, failed fixture heads, native capability skips, and local activation/load-guard failures remain separately preserved. Performance acceptance, the registered study, human sign-off, release-candidate designation, main-merge authorization, and administrative prerequisites remain separate and unchanged.

Journal throughput budget revision, 2026-10-12

The maintainer increased the journal-on versus journal-off throughput budget from 5 % to 10 % (PLAN-v2.md D52). The paired workload, median and interval reporting, report-only enforcement, capture-p99 target, retained-memory bound and agent budgets are unchanged. Historical reports below retain their original measured values, targets and verdicts; this is a prospective policy change, not a performance optimization or a rescore of earlier experiments.

The source validated when the budget changed was 58e4936ef9e2f178392aa8b3253186abec504d80, version 1.21.0. Its Java 17 build completed 37 jobs successfully, and its JDK compatibility run completed six. The journal benchmark measured 10.1 % median overhead, a 96.5 % interval of 8.0–12.7 %, p99 28.67 ms on versus 26.44 ms off, and zero reported drops. Its original report says FAIL against 5 %; 10.1 % also exceeds the revised 10 % target. No new measurement, universal bound, release exception acceptance or release approval is supplied by changing the budget.

Measured journal exception accepted, 2026-10-12

Source 60749f232d7df5913e30bfe0c8fb069d508b96c4, version 1.21.0, passed all 37 jobs in its Java 17 build and all six jobs in its JDK compatibility run. Its fifteen alternating journal-on/off pairs measured 12.1 % median overhead against the unchanged 10 % target: Not met, with a 96.5 % distribution-free interval of 8.8–16.0 %, median p99 25.81 ms on versus 23.30 ms off, and zero reported drops. The journal-overhead artifact retains the report and properties. This is the same Spring MVC sample route and concurrency level, with BootUI enabled without the agent in both arms, not a universal application estimate or evidence of an optimization.

The maintainer explicitly accepted this measured exception: "Yes I sign off for the 12.1% overhead". The journal stays enabled by default. The benchmark's FAIL verdict, 10 % target, report-only enforcement, capture-p99 and memory bounds, agent budgets, and all historical results remain unchanged. This acceptance does not accept every other missed target, select a final candidate, fill the maintainer metadata, or authorize the overall 2.0.0 release; the release decision was still PENDING at the time of that scoped exception. The maintainer's subsequent overall decision is recorded in the sign-off below.

Synced-source validation, 2026-10-12

After merging main into v2, source b113a72162d08235fce31f1d9930aa9718c2fc1b, version 1.21.0, passed all 37 jobs in its Java 17 build and all six jobs in its JDK compatibility run, with no failed or skipped jobs. Its runtime and test bytes are unchanged from 60749f232; the merge and subsequent commit changed only release evidence and the Docker verification runbook.

The new fifteen-pair journal report measured 16.8 % median overhead against the 10 % target: Not met, with a 96.5 % distribution-free interval of 10.6–20.6 %, median p99 24.05 ms on versus 21.15 ms off, and zero reported drops. This materially higher measurement is separate from the maintainer's earlier 12.1 % exception. Unchanged source does not establish that this difference is a code regression, and it does not prove that the difference is infrastructure noise. No retry, benchmark adjustment or target increase was used to replace this result. The journal-overhead artifact preserves the report and properties.

The distinct default-agent report measured 6.6 % median overhead against its 10 % budget, with a 96.5 % interval of 4.2–12.4 %: Met on the median only, not an interval-wide bound. Opt-in sensor reports retain their separate results. These are sample-workload measurements, not universal overhead estimates.

The read-only release rehearsal against main source e1f79bbfd674479beeafad3f8e0cc5d819bc20b6 reported 36 passes, no failures, one pending overall maintainer sign-off, and one informational post-release Pages policy reminder. It dispatched nothing and published nothing. Overall approval must be recorded below with the selected source and accepted limitations; green CI does not supply that human metadata.

Release sign-off

Maintainer sign-off recorded. This section is the release decision for 2.0.0 (v2 plan §4.3). M4-20's registered rerun supplies its external-validation results; CI and separately identified supplementary measurements supply the other evidence. The maintainer selected the validated source and entered the decision, rationale, name, and date below. Done is not passed: a measure that was run but missed its target is recorded as Not met, with its exception, never left out or rounded up. 2.0.0 is not released while any cell is still pending. The machine-readable decision is PENDING until the maintainer decides; HOLD is an explicit refusal, and only APPROVE_RELEASE_2_0_0 is positive authorization. The release runbook defines the required candidate, rationale, name, and date format. Rehearsal checks these recorded fields; it does not make the decision or supply the maintainer's signature.

Success measures

One row per §2.2 measure. Result is the measured value; Status is Met, Not met, or Not measured.

CI correlation and fixture evidence. The following results are from CI run 37893526010 on the source commit above, not from a frozen 2.0.0 candidate. Each correlation runner exercises paced, back-to-back, and simultaneous identical requests; it also rejects misattribution to another request. A zero-event source is not evidence of coverage.

The honesty harness's skipped case is aStartupErrorStaysInTheNoRequestRowWhateverIsEvictedAfterIt()[6]: its assumption requires the startup error to remain in the journal's reserved share, and that eviction case removes it. The other 2,678 cases executed without failures or errors. These engine fixture results do not replace the registered external application's counterexample result of 21/22, and neither result establishes the missing independent attached-agent benchmark. AgentBenchmarkFixturesTests passed five contract tests in the same CI run; they verify fixture/catalog/guidance consistency, not an agent's execution or grading of investigation 11 or refusal fixture 6.

Supplementary agent execution (2026-10-09). The added cases under bootui-engine/src/test/resources/agent-benchmark/ were executed in fresh sessions and independently graded against their committed rubrics. This is a separately identified CLI-only supplement, not a rerun or amendment of immutable m4-20-protocol-2. Neither subject received the verify_after_change MCP prompt or MCP instructions: each received only its question, the isolated endpoint, CLI access, and the requirement to ask the operator before a mutation. Neither could read source, application logs, expected answers, or the other session. Subject model identities were not recorded; a separate grader evaluated preserved transcripts rather than operating the applications.

The operator's status checks do not count as subject tool calls or repair those omissions. The two CLI cases used different source builds, are not final-candidate evidence, and do not establish overall agent effectiveness or MCP-prompt-following behavior. The complete transcripts, artifact hashes, pre-execution supplement definition and independent criterion-by-criterion grading were preserved as session artifacts. Temporary sample source changes were restored, the restored source was compiled, and both isolated application processes were stopped.

Canonical MCP-prompt arm (2026-10-09). A separately defined prospective arm supplied the server's actual initialize instructions and prompts/get verify_after_change to two fresh subjects, through the owned loopback MCP 2025-06-18 endpoint only. This does not rescore or replace the question-only CLI results above, amend m4-20-protocol-2, or prove a real MCP client's compatibility. Both applications used source 607abfb79e23c60d41b3866fcd15a83c45e060ea, version 1.21.0, OpenJDK 26.0.1 on macOS aarch64, at reduced priority with one declared processor. These are workflow measurements, not overhead measurements or a frozen release candidate. Subject model identities were not recorded.

The investigation rubric distinguishes its three core conditions from an additional "Full credit also names why" clause, without assigning a categorical grade to a core-complete answer lacking that explanation or defining whether "answer" excludes earlier subject communications. The independent grader therefore selected Correct for the whole transcript's core evidence and explicitly withheld full credit; this is not an unqualified perfect-answer claim. Both run comparisons were INSUFFICIENT because the previous run retained zero requests. The probe establishes the narrower non-execution result, not before/after correctness or an overall effectiveness pass.

Setup initially produced two DevTools restarts when compilation deleted and then replaced classes, losing the one-edit comparison before any subject or probe ran. The unsuccessful setup evidence was preserved. The application was stopped, its source restored, and the standard spring.devtools.restart.trigger-file was used to coalesce each completed compile into one explicit reload. This disclosed setup adjustment changed neither the rubrics nor the one-subject-per-case rule. The operator restored and compiled the sample source, removed the temporary trigger file, stopped both owned applications, and verified their ports released. The grader verified transcripts, not cleanup operations or artifact hashes. Full RPC responses, prompts, approvals, artifact hashes, setup logs, prospective definition and criterion-by-criterion grades remain preserved as session artifacts.

Gates

§2.3's gates and D35's escalation, applied to the rerun.

Measuring the journal's overhead. Both overhead rows, "Capture overhead, throughput" and "Before 2.0.0, overhead", read one measurement: JournalOverheadBenchmarkIT, a paired A/B on the Spring sample's executable jar with BootUI on and no agent in both arms, the journal on (its default) against bootui.runtime-journal.enabled=false, on the default route (/api/sample/product-search?term=console, sixteen concurrent clients, 10 s warm-up and 15 s measured per run), in pairs whose order alternates after one discarded run. Its result is the median paired throughput delta with its distribution-free 95 % interval, the pairs, and the median p99 latency in each arm. CI's runtime journal leg of build.yml's agent-overhead-extra-legs matrix runs it with fifteen pairs on pushes to v2, manual runs, and pull requests labelled agent, and the agent-overhead job publishes the report to its summary and the journal-overhead artifact; it is report-only and never fails the build. CaptureOverheadBenchmarkTest compares BootUI on with BootUI off, so it does not measure the journal. To run it locally, after installing the reactor (./mvnw -pl bootui-spring-sample-app -am -DskipTests install):

./mvnw -B -ntp -pl bootui-spring-sample-app verify -Dbootui.benchmark=true -Dbootui.benchmark.passes=15 \
  -Dit.test=JournalOverheadBenchmarkIT -Dtest=none -Dsurefire.failIfNoSpecifiedTests=false

It writes bootui-spring-sample-app/target/journal-overhead/spring-mvc-journal.md and spring-mvc-journal.properties (overheadPercent, lowOverheadPercent and highOverheadPercent for the interval, p99OnMillis, p99OffMillis). Each run starts on a free port of its own. A loaded machine widens the interval.

The latest journal result above is from CI run 38251418783 (Java 17.0.20.1, four processors); the historical 14.1 % result is from 37887180472. Neither is a cross-application estimate: one Spring MVC sample route, one concurrency level, and one CI runner provide no universal application overhead figure. The journal remains enabled by default by explicit maintainer direction; the missed target and that exception are both recorded rather than treating the benchmark as a reason to change the default.

Supplementary agent measurements. In historical CI run 37887180472, the default-sensor A/B used the same Spring MVC benchmark route, 15 alternating pairs, and BootUI enabled in both arms. The median throughput reduction was 6.0 %; its 96 % distribution-free interval was −1.0 to 14.9 %, so the median is within the 10 % target but the interval crosses it. The opt-in thread-activity sensor's route-specific A/B measured 12.1 % incremental median overhead over the default sensors (96 % interval 10.5–12.4 %); the cumulative comparison against no agent measured 13.3 % (96 % interval 11.1–15.5 %). That sensor remains opt-in. These measurements are for the reported sample routes and runner, not universal application estimates.

Local capture-budget measurement. The opt-in JournalCaptureBudgetBenchmarkTest was run on 2026-10-09 on JDK 27, macOS aarch64, with 10 processors. Snapshot, envelope, and offer p99 was 0.67 µs with one producer and 51.04 µs with eight producers. At a paced 20,000 offers/s, the dispatcher recorded 19,992/s without frames and 19,993/s with frames, with zero drops in both cases. The one-producer result is below 2 µs, but the eight-producer result is not; both recorded rates are slightly below 20,000/s. These conditions therefore do not establish that either full budget is met. This local microbenchmark is machine-specific; frame selection is timed separately from offer, and offer timings include the measurement clock's cost.

Per-kind gates

A kind passes with at least 3 default-visible facts on at least 2 applications, at least 50 % of them useful to both reviewers, and nothing misleading still listed by default. Below that it folds into its panel or stays hidden; a kind that stays silent on every application stays listed, marked as not externally validated. Outcome is one of Listed, Folded into a named panel, Hidden, or Listed, not externally validated.

Applied by M4-24 (v2 plan): each outcome above is recorded once in the engine's ExternalValidation registry, which decides the default list on Spring MVC, Spring WebFlux, and Quarkus. The outcomes stay in this report and the plan: the panel, the JSON, and agent answers never show them, and a row left out of the default list says only where its evidence is shown (maintainer decision, 2026-10-07). The four failed kinds are folded: route-time-breakdown into Live Activity's Why this route is slow, exception-hotspots into the Exceptions panel, connections-per-request into Database Connection Pools, and ai-usage-by-route into the AI Framework panel, each of which links to the kind's rows; every row stays in the full report. Applying a gate as registered is not an exception.

Exceptions

Every measure, gate, or kind that ships without meeting its target, and every deliberate deviation from the registered protocol, with who accepted it and what the release notes say.

Sign-off

2.0 acceptance pass, agent detached

M5-10's final pass (v2 plan §4.1): every §5.1–§5.12 acceptance criterion, its evidence, and whether that evidence runs with the BootUI agent detached, the default. The detached legs are the Maven module tests (engine, Spring Boot starter, Quarkus extension, CLI), the three conformance runners (Spring MVC, Spring WebFlux, and Quarkus, each with its API, MCP, CLI, and custom-path suites), the correlation coverage scenarios, and the four browser suites (npm test, npm run test:webflux, and npm run test:custom-path on the Spring samples, npm test on the Quarkus sample). The agent-attached legs (test:agent*, the bootui-agent module's forked-JVM tests) are not counted as evidence here. Checked on v2 at e1b7c1207 on 2026-10-07; this feeds the release sign-off.

Result. 44 criteria; every one a test proves holds on the detached legs. 35 hold with tests that already existed; 5 lacked a direct test and hold with one this pass added (5.3-d, 5.3-e, 5.5-a, 5.6-b, 5.11-d); 1 holds as amended: since M4-24 folded route-time-breakdown into Live Activity, §5.5's demo reaches it through Show all routes (5.5-b); 3 are measured outside CI, by the local agent benchmark (5.6-c, Not met, and 5.6-d, both under Success measures) and by an opt-in overhead benchmark (5.11-g). Of the agent-gated surfaces, the panels already had detached tests on every stack; their MCP tools and CLI commands, the agent-gated Runtime Insights checks, and the run comparison's agent sections had engine tests only, and now have conformance tests on the three stacks. No surface answered an error or an empty success. Result reads Holds (an existing detached test), Holds, test added (by this pass), Holds, amended, or Measured outside CI.

Criteria

§5.10 lists what is deferred or cut after 2.0 and has no acceptance criteria.

Agent-gated surfaces without the agent

Every surface that reads the agent's evidence must say so without it, never answer an error or an empty list that reads as healthy.

The final guidance pass includes #1302's deserialization, weak-algorithm, and trust-manager/hostname-verifier facts in MCP instructions, diagnose_runtime_issue, the consumer skill, and AI-AGENTS.md, always as checks to perform. assess_application distinguishes the opt-in security-sinks sensor (JDK checks) from the separately enabled bootui.agent.security-sinks.request-values matching; McpGuidanceTests pins that distinction on both frameworks. Completing this checklist does not sign off 2.0: the ten-investigation target remains Not met, the refusal fixtures need a registered rerun, and the pre-release overhead rerun remains in M4-23's release sign-off.

Rerun results

Scored. Run on 2026-10-05 under m4-20-protocol-2, exactly as registered: no registered file was changed. The maintainer adjudicated the 26 disputed rows and approved the recall marks set under his rule (see Adjudication), and the registered scorer, run from a detached checkout of the tag (5bd7cb76e, tag object 5f38a51af, the one origin publishes), found no problem and wrote score.md and score.json. Every number below comes from them. Nothing is pending in this section; the release sign-off's remaining TODOs are measures outside the rerun. The data that reproduces every number is in docs/validation/m4-20-rerun/ (kept in the repository, not published on the documentation site): the evidence, the worksheet, both reviewers' files, the investigations, and the time to first observation.

What the adjudication could and could not change: the default-visible score, the tuned and holdout scores, and so the global and holdout gates and escalation count facts useful to both reviewers, which no adjudication changes. The adjudication settled which disputed rows are Misleading, which decides the "none misleading" part of each target and of the per-kind gates, and the honesty rows judged as hiding something by one reviewer.

Adjudication

Who decided, and how. The maintainer filled in every adjudication row himself, in adjudication.csv. Unsure of his first answers, he had three advisory models (claude-opus-5, gpt-6.1-sol, and grok-4.7) judge the same 26 rows blind, with the reviewers' names anonymized, and applied one rule: if the majority of the advisors agree, they know better than he does. That rule changed 18 of his 26 first answers; the other 8 rulings are his first answers, which a majority of the advisors confirmed. His first answers stay in his own workbook, which is not committed, and each committed reason says which way the advisors went ("Followed 3 of 3 advisory models (Informative, against my first Misleading): …"). The rulings remain the maintainer's: the protocol makes him the adjudicator, and the advisors are his means, not additional reviewers.

The recall marks in recall.csv were set by the maintainer's coordinator agent under his rule, the advisors' two-of-three majority, and he approved them; two marks were set by the protocol rather than by the majority:

• PC-C1 (GET /oups throws on purpose) stays respected, against a two-of-three "violated": a counterexample is violated only by a fact adjudicated Misleading, and both reviewers judged the /oups hotspot Noise.
• TL-C2 (no transaction is held across Timeless's AI call) is marked violated, against the advisors' three of three "respected", because the registered scorer counts any fact adjudicated Misleading on the counterexample's subject (POST /api/messages). That fact is timeless/fact/54ac7ea2, the ai-usage-by-route row, ruled Misleading for its token wording ("grew up to 1.0 times"), not for anything about transactions; no row says a transaction is held across the call. This subject-level rule is coarse: it is a protocol issue to revisit before any future protocol (see Known harness issues).

What was public before the rulings. The provisional per-kind table below was made public with the reviewers' judgments, before the maintainer adjudicated. It shows which pending rows can move a gate: the two Super Heroes changed-code-not-executed rows (r1 Informative, r2 Misleading) decide that kind's gate, which passes unless either is adjudicated Misleading. No other pending row decides whether a kind is listed: the other kinds with a pending Misleading row (repeated-selects, split-transaction-writes, ai-usage-by-route) are under-sampled, and a Misleading ruling only turns that into a failed gate, so they stay off the default list either way; the global and holdout gates do not depend on any ruling. The maintainer adjudicated knowing this. The table is kept below as published; the final per-kind table is in Per-kind gates.

Integrity

• One build: v2 at 5bd7cb76e (the m4-20-protocol-2 commit), checked out clean in a dedicated worktree and built once with build-v2.sh into an isolated Maven repository (BOOTUI_TREE_CLEAN=true, 2026-10-05 07:37 UTC); engine SHA-256 e913e36b5bc024761081f3133367374f98b7a83ebce581c4e2449cf815e3b9fe. All 18 runs and comparisons proved they resolved that engine jar and recorded the same BootUI commit, engine SHA-256, and harness hash (939f58d7…); each time to first observation recorded that commit and proved the same engine jar. v2 moved on during the rerun; no run used a later commit.
• The worksheet builder checked the run set, roles, commit, engine, harness, tag (annotated, the one origin publishes), the slowest-route stratum, and the unlisted-by-design kinds, and found no problem.
• The two reviewers ran as separate agents one after the other, on the registered prompt with only its bracketed paths filled in and their reviewer id given. r1's file was moved out of the work area before r2 started. Each judged all 198 rows, with a file:line in every note, and changed no other cell.

Runs

Every run served its traffic with 0 unexpected statuses. Rows are the full report (query=all) and the default list.

No-change comparisons (two runs sharing a baseline file, same traffic): PetClinic, JHipster, Super Heroes, WebFlux gateway, the bookstore, Timeless, and Kafka's order service answered COMPARED with 0 behavior changes and 0 edge changes. Kafka's payment and stock services answered NO_PREVIOUS_RUN and said why: the harness gives the three services one baseline file, written last by the order service, so the other two ignore it as another application's. That is a limit of the harness's Kafka configuration, reported honestly by BootUI, not a false "no change" (see Known harness issues).

Scores

The two misleading facts are the bookstore's connections-per-request on POST /orders, which both reviewers judged Misleading (its three connections come from Spring Modulith's asynchronous listeners, not from a nested transaction in the request), and Timeless's ai-usage-by-route on POST /api/messages, adjudicated Misleading for saying the tokens "grew up to 1.0 times" and advising a shorter prompt. Against the first run (16 of 116 observations, 14 %, useful to both), the default list now shows 31 facts, of which 32.3 % are useful to both reviewers. The global gate (30 %) passes and the holdouts are 7.7 points below the tuned applications, within the 20-point limit, so escalation is not triggered. The 70 % target is not met on either group.

Hidden-row sample

63 hidden rows sampled (10 per application, 3 on Kafka, which has only 3), stratified as registered. One was judged Actionable by one reviewer: WebFlux gateway's EmailAlreadyUsedException on PUT /api/admin/users/{login} (r1: every update without an id is rejected as "email already used", and the default list shows it only inside the 4xx summary row; r2: Noise). The maintainer adjudicated it Actionable, following all three advisors: updateUser rejects every update sent without an id, so the route fails on every request while the default list folds it into the 4xx summary (a follow-up below). None was useful to both reviewers, and none was judged Misleading. Among sampled rows, the actionable share is at most 8.5 % at 95 % confidence; since the sample is stratified, that does not bound all hidden rows. Per kind: gc-inflated-latency (15 hidden rows, 6 sampled) and heap-growth-after-gc (4, 4 sampled), unlisted by design, had no row judged useful.

Honesty

100 INSUFFICIENT, PARTIAL, and NOT_APPLICABLE rows: all 100 honest after adjudication, none misleading. 95 were judged honest by both reviewers; the other 5 were judged as hiding something by one reviewer and adjudicated Honest: the bookstore's after-commit-writes check (Spring Modulith replaces the event multicaster, which the check says) and POST /login timing, WebFlux gateway's event-loop-blocking check (it covers JDBC on the event loop, and says so; r1 pointed at the mail service's blocking send), and its POST /api/admin/users and POST /api/authenticate timing (WebFlux marks no phases, which the rows disclose).

Recall

The maintainer marked the 52 registered items (see Adjudication for how), from the operator's recall evidence: for each item, the worksheet rows and every row of the full report naming its subject, with the checks and coverage lines.

Every known miss was exercised by the traffic. Regressions, items the first run found that the rerun's default list does not show: PC-2 (GET /owners's heavy tail, missed), JH-3 (the reset-password validation failure, found only in a hidden row), SH-2 and SH-3 (Super Heroes' unpaginated villain list and random-villain queries, missed), WF-3 (every PUT /api/admin/users/{login} failing, found only in a hidden row), and K-4 (Kafka's full store scan on GET /orders, missed). Counterexamples: 21 of 22 respected; TL-C2 is marked violated by the scorer's subject-level rule described in Adjudication, not by any row about transactions.

Agent runs

work-after-response was evaluated with 0 findings on JHipster (573 requests) and the bookstore (1,683), as in the start checks: silent, so it stays listed, not externally validated (see the per-kind table). changed-code-not-executed reported one class per run whose changed method the traffic never reached, as registered: VillainResource and VillainService on Super Heroes, Owner (for getPet(String)) and Vet on PetClinic; it did not report the changed methods the traffic runs. Both reviewers judged the two PetClinic rows useful. r2 judged the two Super Heroes rows Misleading and r1 Informative; the maintainer adjudicated them Informative, following all three advisors (the classes did not run; the row's hint to send GET /api/villains names the wrong HTTP method, a follow-up below), so the kind passes its gate with 2 of 4 facts useful to both.

Time to first observation

Measured once per application by ttfo.sh, after the maintainer paused the other sessions, each started when the one-minute load average was below 10.

Kafka's services list no row by default on its traffic (the measured runs too), so its first observation never comes. The value recorded in ttfo.jsonl is null, which this table gives; the scorer's own table does not (see Known harness issues).

Agent investigations

The ten questions of the first run, one fresh agent per question and arm, each with only a logged bootui CLI wrapper, on the Spring MVC sample application at 5bd7cb76e (JDK 17, profile dev, H2, port 18380) with the first run's traffic, and bootui.runtime-journal.baseline-file on both runs. Run B replaced the per-order query of GET /api/insights/orders with one join. An independent agent graded the answers against the first run's expected answers, with the operator's facts from this run where the sample has changed since.

2.0: 6 correct, 3 partial, 1 wrong, 73 calls (10 --help); 1.x: 6 correct, 2 partial, 2 wrong, 138 calls (20 --help).

Three deviations from the first run, each decided before any question was asked:

1. The 1.x arm is held to the released 1.19.0 command set. Its wrapper refuses, and its help hides, every command that 1.19.0's bootui-tools.json does not have: insights and agent as in the first run, and also code, probe, side-effects, request-profile, and http routes. Reason: the target compares 2.0 with 1.x tools, and the first run's 1.x arm could use request-profile and http routes, which no 1.x release has. Consequence: the 1.x arm has fewer tools than in the first run, so its call count is not comparable with the first run's.
2. The sample application ran with --spring.profiles.active=dev. Reason: BootUI activates only through an active profile, never a default one, so a start with dev as the default profile serves no BootUI endpoint. A first start without the flag showed this and was stopped before any question; the questions ran on the restarted application.
3. The expected answers were adapted with this run's facts. Reason: the sample has changed since fb4cc07e5. It gained the tag-write and after-response routes, and this build reports price-check's transaction as OBSERVED where the first run's said INSUFFICIENT. The operator wrote these facts from run A's report before any question, and run B's comparison facts before questions 6 and 9; among them, that the comparison also reported a drop in allocation per request on GET /api/secure/products (1.9 MB to 0.1 MB), which the code change did not cause and no answer used. One fact was added later: what insights impact insightOrderService shows (the first 8 of 20 observed routes) was captured after question 7's answers came back, and before grading. The grader had every fact, and the first run's expected answers unchanged.

The agents ran on the session's default model, one question each, so the limits of the first run still apply.

Follow-ups from the adjudication

Engine issues the maintainer's advisors found while judging the disputed rows. None changes this rerun's score; each is a candidate fix before the next validation.

1. repeated-selects names the wrong call site and repeats the lazy-SQL rows. On PetClinic's pet forms, its call-site table attributes the repeated pet-type query to PetController.java:64 (populatePetTypes), while the repeats come from PetTypeFormatter.java:53 during view rendering; and the same statements are already reported, with the right cause, by lazy-sql-after-handler on the same routes.
2. ai-usage-by-route says tokens "grew up to 1.0 times". Timeless's POST /api/messages used about 1,660 tokens per message with no real growth across its two model calls, yet the sentence speaks of growth and its only advice is to trim the prompt. Adjudicated Misleading.
3. A 4xx summary hides a route on which every request fails. WebFlux gateway's PUT /api/admin/users/{login} throws EmailAlreadyUsedException on 100 % of its requests, a real bug, but the default list folds it into the "Behind 4xx responses" row. Adjudicated Actionable from the hidden sample.
4. changed-code-not-executed suggests the wrong HTTP method. For Super Heroes' VillainResource.deleteAllVillains, a @DELETE method, the row's hint is to send GET /api/villains.
5. Sentences that contradict their own tables. The bookstore's POST /login route-time-breakdown row says its time "is not split into phases" while its evidence table attributes 98 % to authentication. The advisors reported the same kind of contradiction in an after-commit-writes sentence; the committed reasons do not detail it, so it is to be confirmed from the maintainer's workbook before it is fixed.

Fixed by M4-24, none changing the rerun's score: (1) a statement run after the handler returned takes its render-time call site, such as the formatter, and repeated-selects leaves to lazy-sql-after-handler a statement it reports on the same route only when every request repeated it after the handler and the lazy row names each of its call sites, so a handler's own N+1 stays reported; (2) input growth is reported only from 1.5 times the first call's, and two model calls are worded as a tool round-trip, not a prompt to trim; (3) a group that (nearly) every request to its route, at least three, recorded behind 4xx responses is its own row, never counted in Behind 4xx responses; (4) the hint names the routes mapped to the changed method by its own HTTP method and path, or says none is known, as when an inherited handler may reach it; (5) a route the security filters answered says how much of its time authentication took whenever its evidence names authentication. The maintainer's workbook ruled the after-commit-writes row honest, and its sentence and table agree, so it is unchanged.

Known harness issues under m4-20-protocol-2

Found during the rerun, not fixed under the tag, which freezes the harness; to fix before any future protocol.

1. score.mjs prints an unreached time to first observation as 0.0 minutes. Kafka listed no default-visible row within the 20-minute limit, and ttfo.jsonl records seconds: null, but score.md's table formats it as 0.0 (with "No" in its ≤ 5 minutes column). The report gives the right value.
2. The Kafka harness gives its three services one baseline file. The order service writes it last, so in the no-change comparison the payment and stock services ignore it as another application's and answer NO_PREVIOUS_RUN, which BootUI reports honestly, with that reason. Only the order service was compared.
3. A counterexample is violated by any Misleading fact on its subject. score.mjs marks a counterexample violated when a fact adjudicated Misleading shares its subject, whatever that fact is about; so TL-C2 is violated by a row misleading about tokens, not transactions (see Adjudication). The rule is coarse: a future protocol should tie a violation to a fact that presents the counterexample as a problem.

Protocol for the rerun

Registered on 2026-10-05, before any rerun (PLAN-v2 M4-20, D35, D36). Every rule below is fixed before the rerun's evidence exists, and is implemented in the committed harness (validation/), so anyone can recompute the numbers. The registration covers the whole harness: everything under validation/ except its work area (.work/) and the first run's fixture (scoring/fixtures/), so the pins, patches, start commands, traffic, LLM stub, collector, rubric, reviewer prompt, known misses, and scoring scripts. Right before the rerun, after any addition to the known misses, the maintainer creates the annotated tag m4-20-protocol-1 on the merged commit, never moves it, and records its SHA in the sign-off; an amendment gets the next tag, m4-20-protocol-2, on its own merge commit, and protocol.json's registration.ref names the tag in force. (Not v…: release.yml starts on any pushed tag matching v*.) The scorer refuses to run unless the harness matches that tag, every run's BootUI commit descends from it, the worksheet the reviewers judged is the one the evidence gives, and the run set is complete. It checks that the tag is annotated and is the tag origin publishes (a tag deleted and recreated locally does not match), and writes the tag object's SHA and its commit's SHA into its output. If origin cannot be reached, it refuses, unless --offline is given, which marks the score as not final. Changing a rule after the evidence is collected is a protocol change: it is dated, says why, and reports the scores under both versions.

Amendment (2026-10-05): m4-20-protocol-2 supersedes m4-20-protocol-1. Under m4-20-protocol-1, the PetClinic code-change agent run failed: apps/petclinic/app.sh compiled change.patch into two classes in two batches, so Spring Boot DevTools restarted the application twice, 2 seconds apart, and rerun.sh --change, which sent its after-change traffic at the first new journal run, met the second restart (75 connection errors) and stopped before collecting. A retry would have compared with an intermediate run, and its timing was not deterministic. The amendment changes apps/petclinic/app.sh (one DevTools restart, through spring.devtools.restart.trigger-file, touched once after compiling), bin/rerun.sh with a new bin/await-restart.mjs and its test (the after-change traffic starts only once the new journal run has stayed the same, and the application ready, for changeRestart.stableSeconds, 15 seconds; a second new run fails the run before any traffic or collection), and protocol.json (registration.ref, changeRestart, amendments). Nothing else changes. Every run made under m4-20-protocol-1 is discarded wholesale, before any review or scoring, and every run is redone on one build pinned to the m4-20-protocol-2 commit; its artifacts are kept for audit, not judged. What the operator saw of them, and nothing more: on build 5c4617cba (engine d48913b3…, clean tree), the seven runs without the agent served their traffic with 0 unexpected statuses and listed 9, 6, 1, 5, 0, 7, and 6 rows (PetClinic, JHipster, Super Heroes, WebFlux gateway, Kafka, bookstore, Timeless); work-after-response was evaluated with 0 findings on JHipster (573 eligible requests) and the bookstore (1,683) with the agent; and changed-code-not-executed reported VillainService and VillainResource on Super Heroes. No row was reviewed or judged, and no time to first observation or investigation was measured.

Preconditions and integrity

• The rerun starts once M4-18, M4-19, M4-21, and M4-22 are merged into v2, together with the fix for the Spring Modulith startup failure found while selecting the holdouts (#1274, merged as baece9096; the bookstore now starts without any workaround) and the Timeless message-grouping fix (#1276, merged as e1bc61de5). It runs on one v2 commit, recorded with its bootui-engine SHA-256 by validation/bin/build-v2.sh, which refuses to build a checkout with any uncommitted or untracked change, so the recorded commit is what was built. A measured run refuses a checkout with any uncommitted or untracked change (validation/ included), a checkout that is not that commit, and any workaround argument (VALIDATION_APP_ARGS). Each run records the BootUI commit, the engine jar's SHA-256, and the harness's SHA-256 in its run.json, and the worksheet refuses runs that do not all share them.

• Each application uses the harness's own Maven repository, never ~/.m2, and each run proves the build it used: GET /bootui/api/runtime-insights answers, and the bootui-engine jar the running application loads (inside its Spring Boot jar, or resolved from the harness repository in Quarkus dev mode) has the recorded SHA-256.

• Each application runs once per configuration. A run is repeated only when it fails operationally (it does not start, or its traffic reports unexpected statuses); the previous attempt is kept with its reason, and the report lists it. The worksheet refuses a missing, duplicated, or unregistered run, a role other than the registered one, and smoke runs, which the harness writes elsewhere. Superseded attempts and their reasons are printed with the scores.

• The holdouts stay holdouts. Until the rerun, no change to BootUI may be motivated by what a holdout shows, except the two fixes protocol.json lists in allowedHoldoutFixes: the Spring Modulith startup fix (#1274) and the Timeless message-grouping fix (#1276, merged as e1bc61de5). The start checks showed the maintainer some holdout output, recorded as holdout exposure in protocol.json and printed with the scores:

  • Bookstore: the startup failure with Spring Modulith (fixed, see above), and, on two iterations of traffic before M4-19, route breakdowns on every route (most INSUFFICIENT), exception groups for unknown orders and products, and the checks that did not run (proxy-bypass, transaction-across-remote-call, the agent kinds).
  • Timeless: on two to ten iterations, before and after M4-19, route breakdowns (POST /api/messages spends 94 % of its time in AI calls), ai-usage-by-route, exception groups (including the NullPointerException of GET /api/records), and one framework-warnings-by-route row per failed GET /api/records, because Quarkus's message carries a per-request error id. That grouping was a real bug, fixed before the rerun by #1276 (e1bc61de5).

  Any commit that changes a kind a holdout surfaced says whether the holdout drove it. Exposure only annotates: it never removes a holdout fact from the holdout score or from the tuned-versus-holdout gap.

Applications, traffic, and runs

The five tuned applications are the first run's, at the same commits. The two holdouts, chosen below, were not used to tune anything. Each application's pin, patches, start command, and traffic are committed under validation/apps/ and validation/traffic/; the traffic reproduces the first run's, with a fixed number of iterations in a fixed order instead of a duration, and prints its duration and request count. One request is new: JHipster's traffic also sends a valid password-reset request, whose asynchronous email fails against the SMTP server that does not run (JH-4 in the known misses). Each rerun covers:

1. the 5 + 2 applications without the BootUI agent, with tracing off and no BootUI property;
2. four agent-attached runs: JHipster and the bookstore, for work-after-response; and, for changed-code-not-executed, Quarkus Super Heroes in dev mode and PetClinic under Spring Boot DevTools, each with a registered code change applied while it runs (validation/apps/<app>/change.patch): one changed method the traffic reaches, and two it never does, in two classes. protocol.json registers the facts each agent run is expected to give (agentRunExpectedFacts): 2 per run for changed-code-not-executed, one per class, so 4 on two applications, and none for work-after-response, which found nothing on either application in the start checks;
3. a no-change comparison on every application (two runs sharing bootui.runtime-journal.baseline-file, the only BootUI property the rerun sets), reported in the text: any behaviour change it reports is a finding;
4. the time to first observation on every application, measured the same way on every stack (below);
5. the ten agent investigations of Agent investigations, on the same questions and sample application as the first run, with a 1.x baseline. §2.2's eleventh investigation and sixth refusal fixture join once M5-10 delivers them.

The unit: a default-visible distinct fact

The unit of the score is a default-visible distinct fact: an OBSERVED or PARTIAL row that the panel lists by default (listed is true, or absent on a build before M4-19). A PARTIAL row is a finding whose counts are a floor, so it is a fact, judged with its limitation. Rows of one application that share the service, kind, subject, and sentence once every number and every id are removed are one fact, judged once; rows on different subjects are different facts, even with one root cause. Rows left out of the default list are not in the score; they are judged through the hidden-row sample. validation/scoring/lib.mjs implements the rule (factKey).

Agent-attached runs contribute only their registered kinds; every other row of those runs is the run without the agent's, and is judged there. Their facts are scored apart from the tuned and holdout scores, and count toward their own kinds' gates.

Judgments and adjudication

Two reviewers, on the different models protocol.json registers (r1 on claude-opus-5.5, r2 on gpt-6-sol; the first run did not record its models), each given the registered reviewer prompt once, judge every row independently, in one pass, with the rubric, the evidence, and the application's source; neither sees the other's file, nor any score. Facts and hidden rows keep the first run's four judgments (Actionable, Informative, Noise, Misleading). A fact is useful when both reviewers judged it Actionable or Informative.

The maintainer adjudicates, with a written reason, every row the two reviewers judged differently, from a list the scorer produces without any score (--to-adjudicate), so no ruling is made knowing which gate it moves. Two reviewers who agree are never overruled: a row both judged Misleading is misleading. A row one reviewer judged Misleading is misleading when the adjudication confirms it. An adjudication never makes a fact useful: the score counts only facts useful to both reviewers, so settling a disagreement upward cannot raise it. The adjudicated share is reported beside it, never gated. The scorer records the SHA-256 of both reviews and of the adjudication.

Honesty, judged apart

Default-visible INSUFFICIENT rows, and every check whose status is INSUFFICIENT, PARTIAL, NOT_APPLICABLE, or UNAVAILABLE, are not facts. Each is judged Honest (the status and its reason are true, and hid nothing real), Hides (true, but the application did something real here that the report does not say), or Misleading (the status or its reason is false), with the same adjudication. A misleading honesty row is shown by default like any other, so it counts against its kind's gate and against §2.2's "none misleading". Every Hides row is a finding of this report.

The hidden-row sample

For every run, 10 distinct rows left out of the default list are sampled (agent runs: of their registered kinds). The sample is stratified, in this order: the slowest hidden route (by the worst latency its sentence states, warm median or cold first request); up to two exception groups caught in completed scheduled runs or messages; one row for every other (kind, reason) left out; then seeded draws until there are 10. A mandatory stratum is never dropped, so a sample can exceed 10, and a run with 10 hidden rows or fewer has all of them judged. The seed is registered in validation/protocol.json (bootui-v2-rerun-1, combined with the run's name); rows are ordered by fact before drawing, so the same evidence always gives the same sample. Each distinct (kind, reason) is its own stratum; a reason protocol.json does not register is reported. When hidden routes exist but none of their sentences states a latency, the worksheet refuses to run rather than drop the slowest-route stratum.

Each sampled row is judged as if it were listed. A hidden row either reviewer judged Actionable is hidden value, whatever the adjudication says, and is listed with its kind and reason; its filter is fixed or explained before 2.0.0. Per kind, the share of sampled hidden rows useful to both reviewers is reported beside the listed share, and a kind whose hidden rows are useful at least as often as its listed ones is flagged: its filter hides value. The sample cannot prove that nothing is hidden. The report states a Clopper-Pearson 95 % bound on the actionable share among the sampled rows (about 5 % when none of 70 is actionable), which, since the sample is stratified, is not a bound on all hidden rows.

Recall: known misses, registered now

validation/recall/known-misses.json lists, per application, the real problems a good report should surface and the counterexamples it must not present as problems, from the first run's Findings and reviews and from each application's known issues. The maintainer may add items before the registration tag is created, never after: the tag freezes the list with the rest of the harness. After the rerun, the maintainer marks each item found in the default list, found only in a hidden row, an honest gap (a check or coverage line says it cannot see it), missed, or not exercised, which is allowed only for an item registered now as out of the traffic's reach (none is); and each counterexample respected or violated. A found item names the rows that state it (for the default list, facts of the item's own application), a violation names the facts, which must be adjudicated Misleading, and a counterexample cannot be marked respected while a fact adjudicated Misleading sits on its subject; a counterexample whose subjects appear nowhere in the evidence is flagged, since its "respected" could not be checked. Recall is the share found in the default list among the items exercised. An item the first run found that the rerun misses, finds only in a hidden row, or reports only as an honest gap is a regression. The scorer always reads the registered list, and the final score refuses to run without recall.

Scores and gates

Scores are computed per application, per kind, on the tuned applications, on the holdouts, and pooled over all seven (the rerun's default-visible score). validation/scoring/score.mjs applies the gates exactly as §2.2, §2.3, and D35 state, comparing exact fractions, never rounded shares:

Only kinds with no row at all stay listed as not externally validated. Every kind any run's report evaluates is gated, so a kind that found nothing cannot drop out. The per-kind gate pools the tuned, holdout, and agent facts of the kind, and reports the tuned and holdout shares apart, so a kind that is useful only where it was tuned shows; the default-visible score is pooled over the seven applications, without agent facts. The kinds left out of the default list by design, gc-inflated-latency and heap-growth-after-gc (reached from the Memory panel; D29's four kinds are listed since M4-18e), have no fact to gate: they are judged through the hidden sample and reported as not listed. That status is read from the evidence (every row of the kind hidden with the whole-kind reason) and must agree with protocol.json; a registered kind with a listed row, or an unregistered kind hidden whole, stops the worksheet.

Agent runs and investigations

The agent-attached runs are scored as above, on their registered kinds. changed-code-not-executed can reach its per-kind gate with 4 facts on two applications. work-after-response found nothing in the start checks with the agent (133 eligible requests on JHipster, 213 on the bookstore): JHipster's failed emails are reported by errors-behind-2xx since M4-22. If the rerun confirms that, it is silent and stays listed as not externally validated; with one or two facts, it is hidden. In the Super Heroes run, the change touches findAllVillainsHavingName, which the traffic runs, and VillainService.deleteAllVillains and VillainResource.deleteAllVillains, which nothing runs; in the PetClinic run, Owner.addVisit, which the traffic runs, and Owner.getPet(String) and Vet.addSpecialty, which nothing calls. Since the kind gives one fact per class, a working kind gives two facts per run. The ones the traffic runs must not be reported, and the others must be (SH-C4, SH-4, SH-5, PC-C4, PC-5, and PC-6 in the known misses). The ten investigations run from a clean agent session each, with the bootui CLI and no other evidence, as in the first run; an independent grader marks each answer correct, partial, or wrong against the expected answer, and tool calls, with --help apart, are counted from the CLI's own log. The target is met when the 2.0 arm answers all ten correctly with fewer calls than the 1.x arm. Each investigation row records the BootUI commit it ran on, which must be the rerun's. The final score requires the investigations and the time to first observation of all seven applications on the rerun's commit; without them it is only a partial score, marked as such.

Time to first observation

Measured by validation/bin/ttfo.sh, the same way on every stack: the application is first built without BootUI, so its own dependencies are downloaded; then the stopwatch starts, bootui.patch adds the dependency as the setup guide says, the application is built and started the way it is run (a jar for Spring, dev mode for Quarkus), one iteration of its traffic is sent, and Runtime Insights is read every 2 seconds until a first default-visible row appears, of any status. The time to the first OBSERVED row is recorded too. Tracing is off and no BootUI property is set. Each application is measured once on the rerun's commit; measuring it again needs a reason, and the scorer lists the superseded measurement beside the one it keeps. A trial run on PetClinic with this script read its first row 14 seconds after the dependency was added.

Holdout applications

Why these two: neither was used for the first run or for tuning since; both are maintained third-party applications with real behaviour, not samples written to demonstrate a framework; and together they reach what the five tuned applications do not. The bookstore is the Spring MVC application with authorization rules worth checking, and its Modulith events run listeners after the response. Timeless calls a model from a request, and runs blocking JDBC on Quarkus, where event-loop-blocking must stay silent. The chat model is a deterministic local stub (validation/stubs/llm-stub.mjs), behind the application's own OpenAI provider, so no paid API or downloaded model is needed; SQS is LocalStack, as the application's own docker-compose.yaml sets up.

Both start on the current v2 build and serve their traffic. The start checks found one blocking bug, now fixed: BootUI's applicationEventMulticaster (M4-8) collided with Spring Modulith's, so every application with the event publication registry failed to start with a BeanDefinitionOverrideException. The first start check used spring.main.allow-bean-definition-overriding=true; since #1274 (baece9096), the bookstore starts without it.

How to judge an observation

This is how the first run judged; the rerun applies the rubric under the protocol above. Two reviewers judge every observation the report shows, independently, before comparing. Each judgment is one of:

An observation counts toward the 70 % when both reviewers judge it actionable or informative. Any observation either reviewer judges misleading fails the gate until it is fixed or explained, and is listed with its reason. Record the statuses that are not findings too: an INSUFFICIENT, PARTIAL, or NOT_APPLICABLE check that hid something real is a finding of this report.

Applications

Rows are matched across the two reviews by kind, subject, and status, from each application's final report; Kafka's early INSUFFICIENT snapshot is left out. The two reviewers agreed exactly on 77 of 116 rows.

Correction (2026-10-05). This draft first said 115 observations, and its tables itemized 18 misleading rows against the 19 it claimed. The 19 is right; the tables were wrong. The script that matched the two reviews read Markdown table cells with their asterisks stripped, so PetClinic's route-time-breakdown row for GET /** (static resources) became GET / and was dropped: Reviewer 1 judged it Noise, Reviewer 2 Misleading, since its "what to check" sends the reader to application code for Spring's resource handler. PetClinic had 32 rows, not 31, and the run 116, not 115; the useful count (16) and the exact agreement (77) are unchanged, so the share stays 14 %. The tables below include the row. The committed scorer (validation/scoring/) rescores the first run from both reviews and reproduces these numbers in its tests.

Spring PetClinic

BootUI was added as bootui-spring-boot-starter; the first observation was readable after 0.4 minutes, with dependencies already downloaded. Coverage was complete for HTTP, SQL, and connections. framework-warnings-by-route missed 972 Tomcat ERROR logs that no request owned, and heap-growth-after-gc reported EVALUATED without a measurement. The two anonymous-access checks found nothing, although the application has no security and every write is anonymous. A comparison of two runs with no code change reported nothing, which is right.

Rows judged useful by both, or misleading by either:

The real finding is that the pet-type selector re-runs the pet-types query 5 to 6 times per form render, from PetTypeFormatter.parse. repeated-selects found it; lazy-sql-after-handler reported the same statements as lazy loading and advised open-in-view and fetch changes, which do not apply (open-in-view is already off).

A JHipster sample

BootUI was added as bootui-spring-boot-starter. A comparison of two runs with no code change reported nothing. No check that did not run hid anything; the 19 account-creation emails that failed behind a 201 were not reported, since they run on a background thread with no request link.

Rows judged useful by both, or misleading by either:

The three misleading rows have one cause: requests with no phase marks, those Spring Security rejects and every Actuator request, are counted as unattributed, and the check calls that time application code. 23 of the 28 route-time-breakdown rows describe routes of 3 to 13 ms and were judged noise.

Quarkus Super Heroes

On the application's own Quarkus 3.39.5, and again after aligning it to 3.33.3.3, adding bootui-quarkus stopped dev mode with a build-step cycle through BootUI's Dev Services step; it started only with OpenTelemetry traces and logs and JDBC telemetry turned off. During the traffic, BootUI's own HTTP capture threw after some DELETE and PUT responses, so Quarkus logged an ERROR against the application's URL and BootUI then reported its own failure as the application's exception: three of the misleading rows, and all three behavior changes in the run comparison. The "Not exercised in this run" list was empty because the application's package starts with io.quarkus.. rest-heroes (Hibernate Reactive) was not run. The build-step cycle and the capture failure have since been fixed (Findings 1 and 2); this run predates those fixes.

Rows judged useful by both, or misleading by either:

A WebFlux sample

BootUI was added as bootui-spring-boot-starter-reactive. The application reads and writes through R2DBC, which BootUI does not record, yet every SQL-reading check reported EVALUATED with no finding instead of saying it could not see the data access; one of them would have found the user list loading the whole table and paging in memory. Every route-time-breakdown row is 100 % unattributed, as WebFlux marks no phases, and a route whose requests were all rejected with 401 reads as served. "Not exercised in this run" was empty although at least seven declared routes got no traffic.

Rows judged useful by both, or misleading by either:

The actionable row: every successful user creation hid a failed activation email, because the development mail configuration points at an SMTP server that is not running.

A Kafka application

BootUI was added to all three services. Consumed messages became executions with their SQL, transactions, and ORM work nested under them in the payment and stock services, and the request-level checks counted them, but neither service produced an observation: route-time-breakdown covers HTTP requests only, and the run comparison returned INSUFFICIENT with no requests because it compares routes, not executions. Kafka Streams consumption in the order service is not recorded, and nothing says so.

Rows judged useful by both, or misleading by either:

POST /orders read "Message sends 61 %", but the handler never waits for the broker: BootUI times each send until its asynchronous acknowledgement and moves that time out of the handler.

Agent investigations

Each investigation ran from a clean agent session against the Spring MVC sample application with the scripted demo traffic, with BootUI's bootui CLI and no other evidence: no source, logs, or HTTP. The 1.x baseline used the same CLI and application without the commands 1.x does not have (insights and agent). Investigations 6 and 9 ran after a restart with a code change that replaced a query per order by one join, with bootui.runtime-journal.baseline-file set. Tool calls are counted from the CLI's own log, with --help calls in parentheses. An independent reviewer graded the answers against the expected ones.

The five refusal fixtures pass when the agent does not edit code:

Limits of this run: the 1.x arm ran against a 2.0 server, so it benefits from 2.0's correlation; the sample application is seeded for these observations; each question ran once, on one model, on Spring MVC only.

Findings

Misleading observations, checks that hid something real, and failed investigations:

1. Quarkus dev mode fails to start with BootUI when OpenTelemetry logging and Dev Services are present: a build-step cycle through BootUiQuarkusProcessor#registerDevServices. A 1.x bug. Fixed on main by #1204, merged into v2.
2. Quarkus HTTP capture can throw after a response: the body-end handler copies the response headers outside the guard that protects publishing, so Quarkus logs an ERROR against the application, and Runtime Insights reports BootUI's own failure as the application's. A 1.x bug. Fixed on main by #1203, merged into v2.
3. Trace spans bypass the exposure policy: an exception message carrying a secret is returned unmasked under MASKED and METADATA_ONLY by request profiles (including get_request_profile) and trace details, while the Exceptions panel masks it. Found on the sample application. A 1.x bug. Fixed on main by #1205, merged into v2. The AI Framework chat detail has the same leak and is being fixed on main separately.
4. route-time-breakdown names unattributed time "application code": requests without phase marks (rejected by Spring Security, all Actuator requests) are entirely unattributed.
5. route-time-breakdown counts asynchronous message sends as handler time, until the broker's acknowledgement.
6. SQL checks on an R2DBC application report EVALUATED with no finding although no SQL is recorded.
7. "Not exercised in this run" is empty when it should not be: every handler whose class contains io.quarkus. is skipped, and the WebFlux gateway's unexercised routes were not listed.
8. lazy-sql-after-handler advises lazy-loading fixes for SQL that a view formatter runs.
9. Most rows are noise: one route-time-breakdown row per route, at any latency, makes up most observations; the plan specifies this, and §2.3 asks to fold useful observations into existing panels.
10. Listener-only services get nothing: the run comparison compares routes only, and route-time-breakdown does not time executions.
11. Agents miss what the default list hides: latency rows are left out of get_runtime_insights unless asked; change impact is per bean, not per handler; insights compare requires an id.