Experiment results
Experiments
Results from checks we ran ourselves. Each page states what was tested, what happened, and the limits of the result.
- Snorkel reward-code reproduction — we re-ran Snorkel's published notebook on the public dataset: all 13 blog counts matched, including 127 of 180 asked and 41 of 180 peeked.
- Stroq attack suite — we ran Stroq's own attack command on version 0.30.0: 36 scenarios, 15 blocked, 21 asked, 0 passed through.
- Aider /test passes arguments through — two runs of aider 0.86.2 show /test with arguments runs them as typed, and bare /test runs the configured test command with nothing added.
How these pages are checked: how we check things.