DeepSeek has released the core of the DeepInfer inference infrastructure, which it uses in large-scale artificial intelligence services, as open source. The package, for developers who want to run DeepSeek models in their own environment, offers DeepSeek V4.1 Flash support at the initial stage. However, the company clearly states that this version includes the offline inference core. Not all parts required for a complete online service are included in the package.
Inference refers to a trained model producing a response to incoming input. DeepInfer manages the computations, processing groups, and work distribution across hardware at this stage. Therefore, the announcement does not introduce a new chat model. It opens up some of the software used to run the current model at scale to developers.
The company shares the code under the Apache 2.0 license. The repository contains optimized computational components as well as a model and sampling runtime environment. Components such as FlashMLA, DeepGEMM and DeepEP are also included in this chain. In addition, the system offers tools to distribute jobs to different hardware and balance the load in expert mix models.
The first run example downloads DeepSeek V4.1 Flash model files into the Hugging Face cache. There is also the option to go for a local copy of the model. The term offline work here describes performing local inference when the model files are ready. This does not mean that no external sources or downloads are needed for the initial installation.
DeepInfer explains hardware support and missing components
The approach, called DSpark, aims to speed up response generation by first generating draft tokens and then validating them. The asynchronous scheduler, on the other hand, overlaps the job scheduling on the processor side with the operation of the accelerators. These are counted among the technical capabilities of the infrastructure. There is no universal conclusion that they will provide the same rate of acceleration or cost reduction for every user.
In the support list of the first release NVIDIA SM100 graphics processors and Huawei Ascend 950 includes accelerators. The installation environment requires Linux, Python 3.12 and an up-to-date compiler. There is CUDA 13 on the NVIDIA side and CANN 9.2 on the Ascend side. This list does not mean that any laptop can run the package directly.
On the other hand, DeepSeek reports that it did not include the key and value caching system required for the full service in this version. The service layout that separates different processing phases and load balancing between replicas are also among the areas that the developer must complete. Therefore, downloading the repository alone is not equivalent to establishing an open and production-ready model service.
There are two different features on the repeatability side. Transaction group-independent inference aims to keep the model result constant even if other requests next to the same request change. Deterministic sampling, on the other hand, tends to produce the same output with the same input and sampling parameters. For this, the developer needs to use a fixed randomness initial value and adjust the options together as described in the document. The company limits these two features to NVIDIA.
The package also supports limiting the output format with JSON schema or grammar rules. Some options for producing repeatable results with the same inputs only work on NVIDIA. The company’s published performance measurements are also based on multiple accelerator layouts. The document describes separate measurement environments using four SM100s for initial processing and 64 for response generation.