I just came across this paper arxiv.org/abs/2509.22908 which coins the term "vericoding" for using LLMs to generate formally verified code, and presents benchmark results for several languages
January 15, 2026 at 9:52 AM
I just came across this paper arxiv.org/abs/2509.22908 which coins the term "vericoding" for using LLMs to generate formally verified code, and presents benchmark results for several languages
A Benchmark for Vericoding: Formally Verified Program Synthesis
Ionel Emilian Chiosa, Sergiu Bursuc, Theodore Ehrenborg et al.
Action editor: Serguei Barannikov
https://openreview.net/forum?id=Zgh5kpGAm8
#vericoding #coding #benchmark
Ionel Emilian Chiosa, Sergiu Bursuc, Theodore Ehrenborg et al.
Action editor: Serguei Barannikov
https://openreview.net/forum?id=Zgh5kpGAm8
#vericoding #coding #benchmark
September 27, 2026 at 8:21 AM
A Benchmark for Vericoding: Formally Verified Program Synthesis
Ionel Emilian Chiosa, Sergiu Bursuc, Theodore Ehrenborg et al.
Action editor: Serguei Barannikov
https://openreview.net/forum?id=Zgh5kpGAm8
#vericoding #coding #benchmark
Ionel Emilian Chiosa, Sergiu Bursuc, Theodore Ehrenborg et al.
Action editor: Serguei Barannikov
https://openreview.net/forum?id=Zgh5kpGAm8
#vericoding #coding #benchmark
Did a survey of all LLM-based VeriCoding benchmarks
Seems like everyone's focusing on single-file programs.
Have you ever seen a REAL verified system? a file-system? a OS? the specs for every function are HUGE.
It looks nothing like your fibonacci leetcode spec. We're cooked.
Seems like everyone's focusing on single-file programs.
Have you ever seen a REAL verified system? a file-system? a OS? the specs for every function are HUGE.
It looks nothing like your fibonacci leetcode spec. We're cooked.
April 30, 2026 at 9:23 PM
Did a survey of all LLM-based VeriCoding benchmarks
Seems like everyone's focusing on single-file programs.
Have you ever seen a REAL verified system? a file-system? a OS? the specs for every function are HUGE.
It looks nothing like your fibonacci leetcode spec. We're cooked.
Seems like everyone's focusing on single-file programs.
Have you ever seen a REAL verified system? a file-system? a OS? the specs for every function are HUGE.
It looks nothing like your fibonacci leetcode spec. We're cooked.
#vericoding
I hope this has the same level of success as Ariane 5
"We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications — in contrast to vibe coding, which generates potentially buggy code...."
arxiv.org/html/2509.22...
I hope this has the same level of success as Ariane 5
"We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications — in contrast to vibe coding, which generates potentially buggy code...."
arxiv.org/html/2509.22...
A benchmark for vericoding: formally verified program synthesis
arxiv.org
April 17, 2026 at 8:21 AM
#vericoding
I hope this has the same level of success as Ariane 5
"We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications — in contrast to vibe coding, which generates potentially buggy code...."
arxiv.org/html/2509.22...
I hope this has the same level of success as Ariane 5
"We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications — in contrast to vibe coding, which generates potentially buggy code...."
arxiv.org/html/2509.22...
Open-source specification manager and Code-by-Contract developer environment to help engineers get closer to verifying code is correct.
github.com/scidonia/axi...
#Python #FormalVerification #SoftwareEngineering #AI #Vericoding #OpenSource #ModelContextProtocol #SoftwareArchitecture #DevTools
github.com/scidonia/axi...
#Python #FormalVerification #SoftwareEngineering #AI #Vericoding #OpenSource #ModelContextProtocol #SoftwareArchitecture #DevTools
GitHub - scidonia/axiomander: Code by Contract Dev Environment
Code by Contract Dev Environment. Contribute to scidonia/axiomander development by creating an account on GitHub.
github.com
September 27, 2026 at 9:47 AM
Open-source specification manager and Code-by-Contract developer environment to help engineers get closer to verifying code is correct.
github.com/scidonia/axi...
#Python #FormalVerification #SoftwareEngineering #AI #Vericoding #OpenSource #ModelContextProtocol #SoftwareArchitecture #DevTools
github.com/scidonia/axi...
#Python #FormalVerification #SoftwareEngineering #AI #Vericoding #OpenSource #ModelContextProtocol #SoftwareArchitecture #DevTools
Cheap, fast, or correct?
With AI and formal verification, you no longer have to pick just two.
We have entered the age of vericoding.
Humans write the specs, and AI + theorem provers do the rest.
Read more: scidonia.ai/blog/ai-guid...
#formalverification #vericoding
With AI and formal verification, you no longer have to pick just two.
We have entered the age of vericoding.
Humans write the specs, and AI + theorem provers do the rest.
Read more: scidonia.ai/blog/ai-guid...
#formalverification #vericoding
AI-Guided Certified Program Refinement | Scidonia
Software's core problem is knowing what it should do. AI-guided certified program refinement — vericoding — makes specification the artefact we tinker with, and lets AI guess program transformations t...
scidonia.ai
September 24, 2026 at 8:02 AM
Cheap, fast, or correct?
With AI and formal verification, you no longer have to pick just two.
We have entered the age of vericoding.
Humans write the specs, and AI + theorem provers do the rest.
Read more: scidonia.ai/blog/ai-guid...
#formalverification #vericoding
With AI and formal verification, you no longer have to pick just two.
We have entered the age of vericoding.
Humans write the specs, and AI + theorem provers do the rest.
Read more: scidonia.ai/blog/ai-guid...
#formalverification #vericoding
this is a point made by e.g. tegmark in the vericoding paper, if we wanted to send someone to the moon, we don't send a physicist that understands the three body problem really well, we have them encode their determinsitic knowledge about the problem in a large body of algorithms which we then get
September 15, 2026 at 9:17 PM
this is a point made by e.g. tegmark in the vericoding paper, if we wanted to send someone to the moon, we don't send a physicist that understands the three body problem really well, we have them encode their determinsitic knowledge about the problem in a large body of algorithms which we then get
Is vericoding a solution?
Maybe not.
If it is difficult to read code generated by a machine, it is much more difficult, I believe, to read a formal specification generated by a machine.
Maybe not.
If it is difficult to read code generated by a machine, it is much more difficult, I believe, to read a formal specification generated by a machine.
The problem with LLM as guarantee of correctness is one of infinite regress.
The software engineers must now validate the validation.
The software engineers must now validate the validation.
April 17, 2026 at 9:34 AM
Is vericoding a solution?
Maybe not.
If it is difficult to read code generated by a machine, it is much more difficult, I believe, to read a formal specification generated by a machine.
Maybe not.
If it is difficult to read code generated by a machine, it is much more difficult, I believe, to read a formal specification generated by a machine.
▶️ Watch San Diego Alignment Workshop video: youtu.be/zBX7FCH3JAE&...
📄 Provably safe systems: the only path to controllable AGI - arxiv.org/abs/2309.01933
📄 A benchmark for vericoding: formally verified program synthesis - arxiv.org/abs/2509.22908
📄 Provably safe systems: the only path to controllable AGI - arxiv.org/abs/2309.01933
📄 A benchmark for vericoding: formally verified program synthesis - arxiv.org/abs/2509.22908
Max Tegmark - Provably Safe AI [Alignment Workshop]
Max Tegmark (MIT) presents vericoding as an alternative to directly deploying powerful neural networks, instead using them to generate formally verified code. His research shows significant progress:…
youtu.be
February 3, 2026 at 4:30 PM
▶️ Watch San Diego Alignment Workshop video: youtu.be/zBX7FCH3JAE&...
📄 Provably safe systems: the only path to controllable AGI - arxiv.org/abs/2309.01933
📄 A benchmark for vericoding: formally verified program synthesis - arxiv.org/abs/2509.22908
📄 Provably safe systems: the only path to controllable AGI - arxiv.org/abs/2309.01933
📄 A benchmark for vericoding: formally verified program synthesis - arxiv.org/abs/2509.22908
Here’s a relevant paper evaluating LLM generation of formally verified code from specifications. They compare Dafny, Verus/Rust, and Lean code written by nine different LLMs. Seems like a promising path ahead! 🤓
arxiv.org/abs/2509.22908
arxiv.org/abs/2509.22908
A benchmark for vericoding: formally verified program synthesis
We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications - in contrast to vibe coding, which generates potentially buggy code from ...
arxiv.org
January 22, 2026 at 4:10 AM
Here’s a relevant paper evaluating LLM generation of formally verified code from specifications. They compare Dafny, Verus/Rust, and Lean code written by nine different LLMs. Seems like a promising path ahead! 🤓
arxiv.org/abs/2509.22908
arxiv.org/abs/2509.22908
"It's easier to verify a proof than to find it." Max Tegmark: "It's easier to verify a proof than to find it." Vericoding has AI generate formally verified code, achieving 82% success. Superintelligent AI may be easier to verify, thus more controllable. 👇
February 3, 2026 at 4:30 PM
"It's easier to verify a proof than to find it." Max Tegmark: "It's easier to verify a proof than to find it." Vericoding has AI generate formally verified code, achieving 82% success. Superintelligent AI may be easier to verify, thus more controllable. 👇
We used LLMs to prove PCF type preservation in Rocq from scratch. The cost? As low as $0.06.
When a mechanised proof costs less than a bug report, vericoding (using AI to write code and a prover to guarantee it) becomes the new standard.
scidonia.ai/blog/proof-a...
When a mechanised proof costs less than a bug report, vericoding (using AI to write code and a prover to guarantee it) becomes the new standard.
scidonia.ai/blog/proof-a...
Proof as Commodity | Scidonia
We proved type preservation for PCF — a classic typed lambda calculus benchmark — using two frontier LLMs and rocq-piler. DeepSeek v4 completed it in 21 minutes for $0.06. Claude Opus 4.8 took 14 minu...
scidonia.ai
June 18, 2026 at 9:28 AM
We used LLMs to prove PCF type preservation in Rocq from scratch. The cost? As low as $0.06.
When a mechanised proof costs less than a bug report, vericoding (using AI to write code and a prover to guarantee it) becomes the new standard.
scidonia.ai/blog/proof-a...
When a mechanised proof costs less than a bug report, vericoding (using AI to write code and a prover to guarantee it) becomes the new standard.
scidonia.ai/blog/proof-a...
Everyone says AI now writes production code. But 3,500 LLM-generated artifacts were just formally verified — over half broke under security prompts. We generate faster than we prove. "Vericoding" is the swing back to math.
Are you shipping code you cannot prove?
#DevTools
Are you shipping code you cannot prove?
#DevTools
August 8, 2026 at 5:05 PM
Everyone says AI now writes production code. But 3,500 LLM-generated artifacts were just formally verified — over half broke under security prompts. We generate faster than we prove. "Vericoding" is the swing back to math.
Are you shipping code you cannot prove?
#DevTools
Are you shipping code you cannot prove?
#DevTools
How much AI code is in prod that no human has read?
Our blog looks at formal verification and how you can take it step by step.
1 BDD (test concrete examples)
2 Executable Contracts (enforce Python rules)
3 Mechanised Proofs (guarantees)
Get the machine to verify code
scidonia.ai/blog/from-bd...
Our blog looks at formal verification and how you can take it step by step.
1 BDD (test concrete examples)
2 Executable Contracts (enforce Python rules)
3 Mechanised Proofs (guarantees)
Get the machine to verify code
scidonia.ai/blog/from-bd...
From Vibecoding to Vericoding: A Gradient, Not a Jump | Scidonia
You do not have to go from zero to verified in one step. You can start with Gherkin scenarios, graduate to executable contracts with specsaver, and then bring in a theorem prover when you are ready. C...
scidonia.ai
July 28, 2026 at 10:01 AM
How much AI code is in prod that no human has read?
Our blog looks at formal verification and how you can take it step by step.
1 BDD (test concrete examples)
2 Executable Contracts (enforce Python rules)
3 Mechanised Proofs (guarantees)
Get the machine to verify code
scidonia.ai/blog/from-bd...
Our blog looks at formal verification and how you can take it step by step.
1 BDD (test concrete examples)
2 Executable Contracts (enforce Python rules)
3 Mechanised Proofs (guarantees)
Get the machine to verify code
scidonia.ai/blog/from-bd...
Bursuc, Ehrenborg, Lin, Astefanoaei, Chiosa, Kukovec, Singh, Butterley, Bizid, Dougherty, Zhao, Tan, Tegmark: A benchmark for vericoding: formally verified program synthesis https://arxiv.org/abs/2509.22908 https://arxiv.org/pdf/2509.22908 https://arxiv.org/html/2509.22908
September 30, 2025 at 6:34 AM
Bursuc, Ehrenborg, Lin, Astefanoaei, Chiosa, Kukovec, Singh, Butterley, Bizid, Dougherty, Zhao, Tan, Tegmark: A benchmark for vericoding: formally verified program synthesis https://arxiv.org/abs/2509.22908 https://arxiv.org/pdf/2509.22908 https://arxiv.org/html/2509.22908
The approach to Vericoding in axiomander in a nutshell
June 17, 2026 at 10:55 AM
The approach to Vericoding in axiomander in a nutshell
Is anyone who knows anything about formal verification following me and if so is there any merit to this (vericoding)?
I think formal verification and 'vericoding' might become part of this picture - hunch (see below)
I'm not even talking about the environmental catastrophe, worker exploitation, theft etc in service of building and deploying these tools? Talking about how people are using them and the future if everyone ends up using them.
April 4, 2026 at 11:23 PM
Is anyone who knows anything about formal verification following me and if so is there any merit to this (vericoding)?
I think formal verification and 'vericoding' might become part of this picture - hunch (see below)
I'm not even talking about the environmental catastrophe, worker exploitation, theft etc in service of building and deploying these tools? Talking about how people are using them and the future if everyone ends up using them.
April 4, 2026 at 11:22 PM
I think formal verification and 'vericoding' might become part of this picture - hunch (see below)
someone coined it as 'vericoding' (as opposed to vibecoding) - arxiv.org/abs/2509.22908
A benchmark for vericoding: formally verified program synthesis
We present and test the largest benchmark for vericoding, LLM-generation of formally verified code from formal specifications - in contrast to vibe coding, which generates potentially buggy code from ...
arxiv.org
April 4, 2026 at 11:21 PM
someone coined it as 'vericoding' (as opposed to vibecoding) - arxiv.org/abs/2509.22908
Vericoding というワードが2025年後半に提唱されてから広まってきているらしい
May 31, 2026 at 1:44 PM
Vericoding というワードが2025年後半に提唱されてから広まってきているらしい