Independent verification · August 2026

An AI said it found new black holes. I built my own instrument and checked.

A 2026 physics paper claims a neural network discovered spacetimes nobody had written down before. Nobody had checked it from the outside. I did, on a desktop computer, in about a day.

Their method holds up under my instrument. Their recipe is also unstable: I trained it five times, changing nothing but a random seed, and only two runs out of five came out as black holes.
Where the horizon sits Schematic causal diagram. Light travels along 45 degree lines. The textbook Schwarzschild horizon is drawn dashed; the horizon measured in the candidate spacetime lies further out, so its trapped region is larger. no escape light escapes later in time further out in space → horizon of the textbook black hole horizon I measured in the candidate
Schematic. Light always travels along the 45° diagonals, so anything above the horizon line can never get back out — that shaded wedge is what makes an object a black hole. My measurements put the candidate's horizon further out than the textbook one, which is how I know it is not a relabelled Schwarzschild solution. The underlying numbers are in the repository.

Three things a black hole has to pass

All three must hold at once. I wrote separate code for each, sharing nothing with the authors' pipeline, and calibrated each against textbook answers before pointing it at their work.

2 of 5 runs

Is spacetime empty there?

0.228 best run · limit 0.286 · my textbook copy 0.123

The limit was computed, committed and tagged in version control before any candidate existed, so it could not be moved to fit the answer. Three of the five runs miss it, up to 1.100.

5 of 5 runs

Is the geometry genuinely different?

2.27–2.58 a Schwarzschild black hole reads exactly 1

The same code returns 1.0000000000000004 on the textbook solution and 1.0000017 on a network trained to imitate it. Every run lands far outside that noise, and they cluster tightly.

5 of 5 runs

Is there a point of no return?

11 of 12 probe points inside a trapped region

Its horizon sits at a measured distance of 0.63 where the textbook one would be at 0.10, pinned down to ±0.003. On the same test both control models reproduce the known horizon exactly.

Five training runs of the same recipe Independent vacuum residual for five candidates that differ only in random seed: 0.228, 0.233, 0.317, 0.915 and 1.100, against a pass limit of 0.286. Two pass. The training loss each run reported is shown alongside and does not follow the same order. my textbook copy 0.123 seed 126 0.228 its loss 0.0132 seed 124 0.233 its loss 0.0118 seed 125 0.317 its loss 0.0110 seed 123 0.915 seed 127 1.100 dashed line: pass limit 0.286, fixed before any of these runs existed seed 127 reported the lowest loss of the five and has the worst geometry
Five runs, identical code and settings, only the random seed differs. Two land inside the limit, three do not, and the worst is nearly five times the best. The number each run reports about itself, its training loss, varies by just 21 percent and in this sample runs the wrong way round.

The part that surprised me

The three properties are not equally reliable. The exotic shape and the trapped region showed up in all five runs, tightly clustered. The plain requirement that spacetime be empty — the actual Einstein equation — held in only two of five.

Worse, the number these networks report about themselves does not see it. Training loss barely moved across the five runs (21 percent) while the real geometric error moved by a factor of five, and the run that looked best by its own loss was the worst when measured from outside — the ranking is reversed, with a rank correlation of −0.90. I went looking for this only after seeing the sweep, on five runs, so I file it as a lead rather than a result. It is also exactly the failure mode an outside instrument exists to catch.

That is the argument for this kind of audit in one paragraph: when a network reports on its own success, someone has to check with a different ruler.

How one person did this in a day

An AI agent wrote the code and ran the experiments. My job was the part that decides whether any of it is worth anything: what to ask, what counts as proof, and what we are not allowed to claim.

hour 0

Set one rule above all others

The AI is never allowed to mark its own scientific work as verified. A stage passes only when deterministic checks pass against files whose checksums are recorded. That became a hard constraint of the system, not a preference.
hour 1

Refused a comfortable answer

Their reported error was 0.00000000016; my instrument said 0.0000033 on the same network, four orders of magnitude apart. I demanded that be explained with measurements rather than prose. It was: their number is squared where mine is not, and on a common scale the two agree to 0.7 percent.
hour 9

Gave up the graphics card, on evidence

I approved installing GPU support. Measured, it ran ten times slower than the processor, because this physics needs double precision that consumer graphics cards deliberately cripple. We recorded that and went back to the processor overnight.
hour 14

Froze the goalposts before looking

The pass limits were derived from the textbook baseline, committed and tagged in version control before a single candidate had been trained. Nobody can accuse us of tuning the test to the result, because the timestamps are public.
hour 22

Wrote to the authors first

Their trained models were never published, so everything here is my own retraining of their configuration. I asked them for the real ones and offered to send them all my results before anything becomes public. That letter went from my own mailbox.

What this is not

  • Not a discovery. The paper's training objective explicitly rewards all three properties I measured. Finding them confirms the method does what it was built to do; it does not prove that a new exact solution of Einstein's equations exists.
  • Not a verdict on their results. Their trained models are unpublished, so I tested their recipe, not their output. That distinction stays in every sentence.
  • Not peer-reviewed. No qualified relativist has reviewed this yet. Until one does, it stands as an independent measurement, not as accepted physics.
  • Five runs, one configuration. The instability is measured on five seeds of a single configuration. The apparent inversion between training loss and real geometric quality is suggestive at that sample size, not established.
  • Not identical measurement points. Each run is judged on hidden points derived from its own checkpoint fingerprint, which keeps me from choosing flattering coordinates but means the runs are not compared at exactly the same places.

Every number here comes from a file in the public repository, each with a recorded SHA-256 checksum, kept alongside the runs that failed. Code and data: github.com/AndreiPLK/spacetime-verifier · archived as doi:10.5281/zenodo.21915627. Paper under audit: arXiv:2607.05489. Instrument, measurements and honest failures by Andrey Pluzhnik (ORCID 0009-0005-5660-2603), with an AI agent doing the typing.