DeepSeek DeepInfer Explained: Self-Hosting Requirements vs API Access

DeepInfer is DeepSeek’s open-source inference framework. It runs a model; it is not a new model or a ready-to-use hosted API. For a developer choosing how to use DeepSeek, the practical question is whether operating the inference backend is part of the job. If you need to ship an application, first compare a hosted API against the full service you would have to build and maintain.

This guide reviews official documentation as of October 22, 2026. We have not deployed DeepInfer on supported accelerator hardware or reproduced its performance charts. The deployment checklist and cost worksheet below are our decision tools; the numerical example is deliberately hypothetical. Ofox provides API access and publishes this article, so we make the trade-offs explicit rather than assume that every reader should choose a hosted service.

What DeepInfer, the model and deepseek-recipe each do

A model deployment has several responsibilities. Treating their names as interchangeable makes both cost estimates and debugging harder.

The DeepInfer README lists DeepSeek-V4.1-Flash as the supported model and uses deepseek-ai/DeepSeek-V4.1-Flash-1017 as the default checkpoint. Read our 1017 model update guide for model-version changes; this article concerns the engine and deployment decision.

The separate deepseek-recipe repository converts Chat Completions, Responses and Messages formats. Its caller must still supply model inference, tool execution and HTTP transport. The included Python and Rust server examples use mock inference. Getting a JSON response from one of those demos is not evidence that a real model has generated it.

For an application developer, this suggests a useful debugging boundary: an HTTP response, a correctly rendered prompt and a successful model generation are three separate checks. Passing one does not establish the other two.

Check supported hardware before planning an installation

The current README lists NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with Linux, GCC 13 or newer and Python 3.12. The platform-specific requirements are CUDA 13 for NVIDIA and CANN 9.2 for Ascend. These are the documented targets at the review date, not a promise about future ports.

An RTX 4090 or H100 is not among those listed targets. Do not turn that absence into “DeepSeek models can never run on that card”: model availability through another engine is a different compatibility question. Equally, a community pull request proposing a port is not confirmation that the current release supports it.

Record the exact device architecture, driver/toolchain versions, checkpoint and engine revision before estimating build effort. If a prerequisite is missing, stop at a compatibility assessment. Renting an expensive machine merely to see whether an unsupported setup might compile is not a useful first experiment.

Three different questions must stay separate:

1. Supported: does this engine version document your platform?
2. Runnable: does the chosen configuration fit and complete inference correctly?
3. Efficient: does it meet your throughput and latency targets at the expected load?

The official deployment Q&A describes a smallest efficient deployment unit of 64 prefill GPUs plus 64 decode GPUs in its recommended production design. That is not an absolute minimum GPU count for every offline test. The README’s benchmark configurations also differ between prefill and decode. Neither number should be copied into a universal “minimum hardware” shopping list.

What remains between offline inference and an online API

The public README explicitly identifies three pieces still required for a complete service: a KV cache system, VE/Prefill/Decode disaggregation, and load balancing across data-parallel replicas. It notes that much of the engine functionality for disaggregation is already implemented. The release boundary matters: available engine primitives are not the same as an assembled, operated service.

For planning, assign an owner and an acceptance test to each service responsibility:

This table is a proposed operational acceptance checklist, not a claim that DeepInfer ships every feature in the left column. In particular, operating the hardware yourself does not automatically resolve privacy: you still control who can access the machines, logs, backups and cached prompts.

For a hosted option, ask equivalent questions of the provider. A compatible endpoint name alone does not establish identical tool behavior, cache accounting or retention terms. If you only want to connect a coding client, start with the existing Codex setup guide or Claude Code setup guide and confirm current route support before adding infrastructure.

Compare costs with the same workload and service target

A peak tokens-per-second chart cannot tell you your monthly cost. Begin with a representative workload: input length distribution, output length, repeat-prefix rate, image use if applicable, concurrent requests and daily peaks. Set a target for completed requests, output quality, time to first token and streaming speed. A cheap service that misses those targets is not an equivalent alternative.

For an API estimate, use separate prices for uncached input, cached input and output. Add cache-write, storage, platform or other charges only where the selected provider actually applies them. Treat unknown charges as unresolved, not zero. Token quantities must use the provider’s billing definitions, including reasoning tokens where applicable. Our DeepSeek API pricing guide explains the pricing topic; obtain a current quote for the exact route before making a purchase decision.

A general worksheet, using millions of billed tokens, is:

API monthly cost =
  uncached_input_millions × uncached_input_rate
  + cached_input_millions × cached_input_rate
  + billed_output_millions × output_rate
  + other applicable monthly charges

Self-host monthly cost =
  accelerator rental OR allocated hardware depreciation
  + host, network and storage costs
  + power costs not already included in rental
  + operations and support labor
  + redundancy and recovery capacity
  + workload-dependent costs

Do not charge the same accelerator once as rental and again as depreciation. Likewise, do not add electricity separately when the rental already includes it. Allocate shared staff time explicitly. For each unknown, collect a quote or measurement and show a range rather than hiding it in a precise total.

A reproducible example, not a DeepInfer price quote

Assume one workload bundle contains 100 million input tokens and 20 million billed output tokens, with 60% of input billed as cache hits. For illustration only, choose rates of $0.80/million uncached input, $0.20/million cached input, and $2.00/million output. These are invented worksheet inputs, not current DeepSeek or Ofox prices.

The API variable cost per bundle is 40 × 0.80 + 60 × 0.20 + 20 × 2.00 = $84. Assume an additional $16 monthly platform charge. For the self-hosted alternative, assume $50,000 in fixed monthly costs and $20 per bundle in additional variable costs. Neither assumption represents a real DeepInfer hardware quote.

Under these assumptions, the equality is 84x + 16 = 50,000 + 20x, or x = 781 bundles per month. That algebra is not a deployment recommendation. The fixed configuration must actually support that volume at the chosen latency and reliability targets. If serving 781 bundles requires another cluster, recalculate the fixed cost; the apparent break-even may disappear.

Changing cache hits from 60% to 20% raises the API variable cost in this example from $84 to $108 per bundle. Self-host costs and capacity can also change when less work is reused. Do not hold one side artificially constant while claiming a precise saving. Evaluate quiet, expected and peak months separately.

Run a pilot that can reject either option

For a team considering self-hosting, use this sequence before a production commitment:

1. Define the workload and success criteria. Preserve a representative, appropriately redacted sample. Set quality, latency, failure-rate and data-handling requirements before choosing a winner.
2. Resolve platform compatibility. Record the supported accelerator, software environment, checkpoint and code revision. Consult the repository’s current examples rather than an unversioned install command from a blog.
3. Verify offline correctness. Run the relevant official example and evaluation checks on that environment. Record results and failures. A successful build alone does not prove generation quality or protocol compatibility.
4. Test the real service path. Replace any mock backend, then verify a complete request and a streamed request. If tools are required, check generated arguments and application-side execution separately. Test invalid input, a timeout, client cancellation and an overloaded service.
5. Measure realistic load. Include short and long prompts, warm and cold caches, quiet periods and concurrency spikes. Capture completed work, latency distribution, failures and all billable resources.
6. Compare with a hosted route. Keep the model version and request behavior as comparable as possible. Document differences that prevent a fair comparison, including quantization, reasoning settings and missing features.
7. Recompute costs and decide. Keep a rollback route and a named operator. If either option fails a mandatory requirement, do not rescue it with a favorable average cost.

If the pilot fails, classify the failure before spending more: a compiler error belongs to the build environment; an out-of-memory failure needs model and runtime capacity analysis; a valid mock response is a test-harness result; malformed tool arguments require protocol/model-path investigation. None proves that a different API key or a larger machine will fix everything.

Choose a route your team can operate

A hosted API is a sensible first candidate when demand is uncertain, usage is intermittent, the team needs to ship quickly, or no one owns accelerator operations. Validate the provider’s limits and terms instead of assuming every hosted route is interchangeable.

Self-hosting deserves a pilot when you have compatible infrastructure, a team able to operate the complete service, sustained demand, or a concrete requirement that the available hosted options cannot meet. Existing hardware does not make the project free: count its opportunity cost and the people who will support it.

Before deciding, fill in this short record:

Task and exact model version:
Mandatory quality, latency and data requirements:
Expected / quiet / peak monthly workload:
Measured cache behavior and billed token categories:
Self-host platform and operator:
Hosted route and unresolved contract/feature questions:
All-in monthly cost range for each option:
Capacity evidence supporting the estimate:
Pilot outcome, rollback route and decision date:

DeepInfer is relevant because it exposes substantial inference engineering. Your adoption decision should still end with an operable service and a workload-specific estimate. Open-source availability alone establishes neither a reduction in your API bill nor a reason to rebuild a working application around a new backend.