Baidu claims inference gains from ERNIE Evolve’s automated research

Baidu had already made efficiency a design priority for Flash, citing the repeated sampling needed in further training and the compute and memory demands of long-context agent applications. Evolve's inference work builds on that existing architecture and infrastructure effort; the reported speed gains do not establish a corresponding improvement in the model's knowledge or reasoning.

Evolve works on a supplied repository with an optimization objective and evaluation criteria. An outer loop selects research directions; an inner loop develops and tests candidate implementations. A Director draws on previous experiments and a knowledge base, a Selector chooses a candidate and hypothesis, and an Executor modifies the code in an isolated environment.

The evaluation boundary matters as much as the code-writing agents. Baidu says a fixed evaluator sits outside the repository and is invisible to the agents. Changes outside permitted scope are excluded, and improvements must exceed measurement noise. A separate Reviewer checks whether work follows the intended direction and exploits evaluation loopholes. Failed experiments are recorded alongside successful ones to inform later choices.

The general approach has precedent. On October 18, automated-research developer Recursive described its own system as proposing ideas, implementing and validating experiments, retaining context across research threads and checking for reward hacks and variance. Baidu's case therefore rests on its particular results and integration into its model engineering, rather than inventing the automated research loop.

Company-reported P800 generation speed reached 623.6% of its initial baseline. Baidu says this deployment initially left available optimized execution paths unused; the result applies to this workload. Source: Baidu. Source: Baidu ERNIE.

Company-reported deterministic-inference throughput reached 133.7% of an already optimized baseline, a 33.7% increase, with determinism preserved. This uses a different workload and baseline from the P800 result. Source: Baidu. Source: Baidu ERNIE.

Three internal workloads, three different baselines

The largest reported acceleration came on Kunlunxin P800 accelerators, where Baidu says ERNIE-5.5-Flash already ran but was not using all available optimized execution paths. Kunlunxin also represents a financial interest for Baidu: in a January 1 announcement, Baidu identified the chip business as a non-wholly owned subsidiary and proposed a separate Hong Kong listing to broaden its financing channels. That announcement did not guarantee a listing would occur.

Evolve's first changes enabled existing fused operations and speculative decoding, which proposes multiple tokens for verification. Later rounds reduced scheduling and synchronization overhead and corrected dispatch logic to use a low-latency communication path. The model and deployment setup were held unchanged, and edits were restricted to the Kunlunxin plugin.

Generation speed ultimately reached 623.6% of baseline—about 6.24 times the starting speed, or a 523.6% increase. That is a workload-specific comparison against an incompletely optimized deployment, not a general speed multiplier for the model. A company-supplied account puts the search at roughly 168 hours across four rounds.

The more demanding comparison is against infrastructure engineers had already tuned. For deterministic inference used in reinforcement-learning training, Baidu says Evolve increased throughput by 33.7% after roughly three days, while reducing average time per output token by 26% and time to first token by 12%. Generated tokens and log probabilities had to remain bitwise identical regardless of other requests sharing the batch; log probabilities also had to match the training system's full-sequence recomputation. Fixing the order of numerical reductions helps preserve that consistency but limits GPU utilization, ruling out some conventional speedups.

On a GPU service already optimized for single-request speculative decoding, Baidu reports a further 14.9% speed increase with accuracy unchanged. Only three of seven research directions produced improvements. Other proposed bottlenecks were ruled out by measurements. In the P800 work, a communication change actually reduced speed by 14% and was abandoned.

Those unsuccessful directions help explain what the system is doing: testing engineering hypotheses and rejecting bad ones. They also show why the final implementation's speed is insufficient to calculate the cost of discovering it.

Public records with a limited comparison scope

Baidu tested small-model training quality and speed, plus GPU kernels—the programs that implement individual accelerator operations. Its announcement reports the following comparisons against the strongest public results it selected at evaluation time:

The released research artifacts provide useful conditions behind those figures. NanoChat uses a 300-second training budget on one GPU. NanoGPT Track 1 targets a FineWeb validation loss of 3.28 on eight H100 GPUs; the repository reports 38.92 seconds across 18 runs. Track 3 targets the same loss with fixed data and architecture and reports 2,590 steps across eight runs. Using the announcement's rounded figures, the Track 1 improvement over its public comparison is about 2.5%.

The five-minute NanoChat limit applies to training a candidate, not to the whole automated search. Nor should its result be read as a pure neural-network architecture advance: Baidu says Evolve added corpus-based statistical and document-retrieval prediction branches alongside the network. The repository's reproduction setup requires a B200 GPU, about 200 GB of free host memory and at least 64 CPU cores. A short training run therefore should not be mistaken for a small total resource requirement.

Baidu attributes its strongest NanoChat comparison to Rekursiv's Auto²-Research report, and the NanoGPT and kernel comparisons to their official leaderboards. Its table also includes automated research systems from Recursive and Tencent, using the latter's Hyra-1.0. It explicitly excludes a like-for-like claim about Hyra-2.0 because that version had not reported results on these tasks.

The kernel figures need particular care. Baidu lists Recursive at 0.676 under evaluation stack v1.1, while Recursive's own October report gives 0.754. The announcements do not reconcile those figures, so they should not be treated as interchangeable measurements. Evolve's 0.819 also remains below the 0.827 leader in Baidu's table. These standings reflect Baidu's reported comparison; they have not been independently confirmed here against NVIDIA's leaderboard.

The repository exposes final code and measurement logs, not the full research system. Its kernel section lists seven kernels, whereas Baidu describes SOL-ExecBench as a 235-problem benchmark. It also warns that official measurements lock GPU clocks at 1,500 MHz and that faster readings with unlocked clocks are not comparable. Reproducing outputs under the specified conditions is an important next check; releasing artifacts alone does not independently validate the broader claims.

The next test is the cost and scope of improvement

Baidu plans to extend the loop from inference infrastructure into training-system optimization, then model-capability research and eventually evaluation tasks themselves. It says expansion must remain under human control, interruptible, and bounded by goals, permissions and resources, with continuing safety evaluation and oversight.

The published evidence does not establish a total search-compute budget, agent-inference bill or net financial return for these workloads. Elapsed research time and final service performance cannot answer those questions without resource usage and deployment volume. The two contemporaneous news reports also provide limited independent corroboration: one explicitly identifies its text as supplied by Baidu, and both closely track the company's account.

The next evidence that would materially strengthen the case is independent reproduction of the public results, fuller disclosure of search costs, and sustained gains after deployment against well-optimized baselines.