Can AI Improve Its Own Ability to Improve? A New Survey Grades the Evidence

Can an AI system improve its own ability to improve?

That question has a name, recursive self-improvement (RSI), and as of September 30 it has a 57-page answer.

Recursive Self-Improvement in AI: A Survey is now up as a preprint. Actava.ai co-founder and Chief AI Officer Weiran Yao is one of its 32 authors, alongside researchers from Tsinghua, Peking University, CMU, UC Berkeley, NTU, NUS, PolyU, and other academic and industry teams.

Actava.ai advisor Caiming Xiong is on the author list too, and Qingsong Wen is the corresponding author.

The short answer they landed on is "sometimes, in limited settings." The long answer is more useful, especially if you run healthcare operations and someone is selling you agents that learn.

A loop that repeats is cheap. A loop that compounds needs proof.

Most AI systems already run some kind of loop. Do the work, collect feedback, adjust, repeat.

The survey draws a hard line through that picture. Running the loop again tomorrow shows that you own a loop. Recursive improvement asks for evidence that each change the system keeps makes the next round of improvement more effective.

The paper's mechanism diagram has 2 loops in it, and the gap between them is the whole argument.

Here's the healthcare version. A utilization management team that clears more cases this month than last has improved. A team whose training program turns out stronger reviewers, faster, every year has improved how it improves.

Both are good. The second one compounds.

The survey packs the distinction into 8 words: "Recursion is a dependency; improvement is an observation." You can wire a system to feed on its own output in an afternoon. Whether it got better is something you have to go measure.

3 claims that usually get blurred into one

When someone calls an agent "self-improving," the survey says they could mean any of 3 things. Each one needs its own test.

Our read is that plenty of "self-improving" pitches stop at the first row, with the system grading its own homework.

The survey's proposed protocol closes those gaps one at a time. Test data stays sealed, and its scores never flow back into what gets accepted. Every comparison runs under the same budget, with a frozen-improver control beside it. The cost ledger counts the retries, the rejected candidates, and the failed attempts (the part of the bill that tends to fall off the slide).

One question from the paper belongs on every procurement checklist. What survives a reset? A revised answer survives nothing. A stored memory, a retrained weight, or a rewritten improvement procedure carries forward, and each one calls for a different test.

What the evidence shows so far

The authors went through the strongest published systems and reported the results with the caveats attached.

The Darwin Gödel Machine, a coding agent that rewrites its own agent code, climbed from 20% to 50% on a 200-task software benchmark subset over 80 iterations. That's real progress. The survey also points out that those scores include tasks used during the search, which keeps them short of a fully independent test.

Another system let a program rewrite its own improvement routine. With a stronger base model it got better in the early rounds. With weaker models it sometimes got worse.

The idea is 23 years old in its formal version (the Gödel Machine, 2003). Foundation models made parts of it practical through generated training data, program revision, and automated experiments.

The paper's summary is careful. Systems can keep useful changes. In limited settings, they improve how they produce the next update. The evidence stops short of open-ended or reliably accelerating self-improvement, and none of it shows a loop running free of goals and feedback supplied from outside.

3 constraints, and healthcare sits inside all of them

The survey names 3 things that limit every improvement loop. Feedback quality, interacting updates, and independent validation.

Healthcare operations hit all 3 before lunch. In our view, that makes healthcare the hardest and most useful place to do this work.

Feedback quality

A model that grades its own work can drift. The survey warns that internal scores can keep rising while performance on the real task stays flat.

The fix is information the loop can't produce by itself. Executable tests, fresh observations, expert review. Healthcare has this in unusual supply. A nurse reviewer overturns an agent's determination. A medical director rewrites a rationale. A claim pays or it denies.

Each of those is a labeled correction from someone accountable for the outcome. It's the scarcest input in the loop, and it usually evaporates in an inbox.

Interacting updates

Changes collide. A new prompt shifts what gets retrieved. A revised tool breaks a skill that passed last month. In healthcare, add a payer policy update that lands on a Tuesday and quietly invalidates a workflow that was fine on Monday.

The survey's answer is unglamorous. Protect the capabilities you already have, set a regression tolerance before the update ships, and keep a rollback you've tested.

Independent validation

The grader has to sit outside what the loop can edit. Sealed tasks, a protected evaluator, and accounting the system can't touch.

This one is close to home. We built Actava.ai χ-BENCH to give healthcare an independent measuring stick for long-horizon, policy-dense workflows. Across 30 agent and model configurations, the best one resolved 28% of tasks. None cleared 20% when the bar was passing 3 runs out of 3.

That second number makes the survey's point from another angle. One good run tells you little. The same result 3 times in a row is what production asks for. (A benchmark score is evidence, and production still gets the final vote.)

Frank Wang put the same idea in a 10-word title in September, The Flywheel Only Spins Where You Can Check the Work.

How we'd grade our own work

It'd be easy to cite this survey and then exempt ourselves. Here's Actava.ai CURA under the same 3 claims.

We trained CURA 1T through what our paper calls a human-gated self-evolution loop. Each round, a training agent picks a target capability, trains the model, evaluates benchmark trajectories, and reworks the data mix from the failures it finds. Humans gate each round.

Task gain. CURA improved over its base model on every healthcare benchmark in the paper.

Retention. It stayed competitive on out-of-domain reasoning and agentic benchmarks, the skills it had before we specialized it.

Improver gain. Untested. We make no claim there.

In the survey's vocabulary, that human gate is an external anchor. We put it there on purpose. In a domain where a wrong determination delays someone's care, the decision to accept an update belongs to a person with a name.

3 questions for any vendor that says "self-improving"

You can borrow the survey's framework without reading all 57 pages.

• What survives a reset, and can you show it to me?
• Who grades each update, and can the system edit its own grader?
• What did the rejected attempts cost, and how fast can you roll back?

If the claim is that the improvement process itself keeps getting better, ask for the control. Same starting agent, same budget, old improver against new.

The open problem

Building reliable improvement loops remains an open research problem. The survey says so in its abstract, and Weiran signed it.

Our bet is that healthcare is where the problem gets solved first, because the ingredients are already on the floor. Real workflows. Expert corrections. Outcomes someone verified.

The work is turning that production experience into evaluations, training data, and better agents the customer controls. We'll keep publishing what we learn, including the parts that don't work yet.

We welcome feedback, discussion, and research collaborations.