August 16, 2026
A claim in our own copy went unverified for months because a probe timed out 0.7 seconds early
The finding
Our validators check whether a simulator is reachable before comparing against it. The check allowed 20 seconds; the environment took 20.7 to start. It failed closed, silently, and every run since had skipped the comparison while printing a line that read like a normal limitation.
We publish an audit of our own tools roughly monthly. One finding stayed open for months and read, in our own notes, as blocked on hardware we did not have.
It was not.
What happened
Our SFQ engine compares itself against JoSIM, an exact circuit simulator. Before running that comparison, the code asks a simple question: is JoSIM reachable? The check ran the simulator's binary lookup with a 20-second timeout.
On the machine in question the environment holding JoSIM takes 20.7 seconds to cold start.
So the check timed out, a broad except clause turned the timeout into
False, and every validator printed "JoSIM/WSL not available" and skipped —
a message indistinguishable from JoSIM genuinely being absent. Our own state
document then recorded the work as blocked on getting a machine with JoSIM
installed. JoSIM had been installed the whole time.
It missed by seven tenths of a second.
The part worth taking away
The timeout was the trigger. The defect was that failing to determine something and determining it to be false produced identical output.
That is a general shape and it is worth checking for in your own tooling: a capability probe that returns a bare boolean cannot distinguish "no" from "I could not tell", and a claim resting on it inherits that ambiguity without anyone noticing.
The fix was not just a longer timeout. The probe now retries, caches a positive result so a long run does not re-probe against an idle environment, and — when it genuinely cannot tell — prints a warning stating that the comparison was skipped and that the run does not substantiate the claim.
What we found once it ran
The comparison itself split. Our analytic engine reproduces JoSIM within the bars we set: junction phase RMS 0.022 rad against a 0.1 bar, firing-time error 0.002 ps against a 1 ps bar. Our neural operator does not — it fails its trace and event gates and an end-to-end check, and is safe in production only because it defers roughly a quarter of near-threshold cases to the exact simulator, at which point its error on what it keeps goes to zero.
So the claim we had been making was true of one half of the engine and not the other. It now says which.