KTransformers Adds LoRA Fine-Tuning and Restricts a Scheduler Attack Surface

KTransformers reported approximately 400 tokens per second for full fine-tuning and 600 tokens per second for LoRA on Qwen3-30B-A3B with two AMD EPYC 9355 CPUs and two RTX 5090 GPUs. It separately reported more than 700 tokens per second for a LoRA setup using one RTX 4090 and an AMX-enabled CPU. Those are the project's own results from end-to-end runs, rather than isolated kernel benchmarks. They are not a controlled 400-versus-600 comparison: the announcement does not provide matching datasets, sequence lengths, batch sizes, training settings or validation measurements for the two figures.

Ampere support has stated feature limits

The bundled sglang-kt integration adds BF16 attention and KV-cache fallbacks for GPUs without native FP8 Tensor Cores, plus an FP8 MoE Marlin path for GPU experts, GPU prefill and dynamic expert updates. The stated goal is to expand DeepSeek V4 Flash inference to Ampere-generation GPUs; it is not a blanket claim of Ampere feature parity.

The implementation discussion provides the important qualification. Its author said an RTX A6000 passed tests for MiniMax-M2.7, Qwen3.6-35B-A3B and Qwen3.6-35B-A3B-FP8 only with MTP disabled and num-gpu-experts=0. The non-FP8 Qwen model could use GPU experts but not MTP; the FP8 version could do neither. The author described these as known Ampere issues in the fork, not issues fixed by the change. A reviewer later said the change worked well on a 4090 before the pull request was merged.

There is a second hardware qualification in the release: its new SYCL backend for GPTQ INT4 MoE inference on Intel integrated GPUs is explicitly experimental. Compatibility and performance depend on the GPU, driver, SYCL toolchain and model configuration. For operators, the relevant comparison is therefore their own target model and enabled features, not the broad label of an Ampere or Intel GPU.

Loopback binding reduces network reachability

The release changes the balance_serve scheduler's ZeroMQ ROUTER endpoint from all interfaces to 127.0.0.1. KTransformers says that reduces exposure to unauthenticated remote pickle deserialization.

The underlying issue report, filed by GitHub user geo-chen, said the default high-performance backend had bound the scheduler to all network interfaces, accepted messages without authentication and passed them to pickle.loads. The reporter said a reachable scheduler port could accept a crafted pickle stream and execute a shell command as the server process; the issue links the subsequent loopback-binding commit and was closed as completed.

That is a meaningful change in reachability: a host outside the machine can no longer connect to a service bound only to loopback. But the report's suggested remediation went further, proposing a schema-validated format or authenticated, allowlisted ZeroMQ messaging rather than untrusted pickle deserialization. The retained record supports saying the release reduced the exposed surface, not that it replaced the unsafe serialization mechanism.

What operators need to establish next

The next evidence needed is configuration-specific. Fine-tuning users need comparable full-parameter, LoRA and hybrid evaluations under the same workload and quality criteria before treating the throughput figures as a method comparison. Ampere users need to test whether their model requires MTP or GPU experts, features the implementation discussion limits in some tested combinations.

For balance_serve operators, the practical check is whether the installed version has the loopback binding and whether any deployment has deliberately exposed an equivalent scheduler path. The release resolves the reported default network binding; it does not, on the evidence retained here, establish a replacement for pickle-based message handling.