DeepSeek open-source inference framework, social media calculation of inference gross margin or over 95%

DeepSeek open-source inference framework shifts the discussion in the large model industry from model capabilities to inference costs.

On October 20, DeepSeek announced the open-source of its internal inference framework DeepInfer, as well as the deepseek-recipe for token encoding and decoding. DeepSeek stated that DeepInfer serves large-scale inference for DeepSeek V4.1-Flash, aiming to increase throughput while reducing latency. The framework supports NVIDIA GPUs and Huawei Ascend NPUs and is licensed under the Apache-2.0 open-source license.

One background that has drawn attention to this open-source release is the inference service revenue disclosed by DeepSeek. According to a deployment Q&A released by DeepSeek on GitHub, V4.1-Flash processes 1.55 trillion tokens daily per cluster, with total revenue reaching $38 per GPU hour based on the price during off-peak hours.

Based on assumptions regarding cache rates, throughput, and pricing, social media estimates its inference gross margin could be as high as 84.2% to 96.1%. Analysts have pointed out that the API gross margins of mainstream models from OpenAI and Anthropic are similarly around 95%.

This disclosure provides an opportunity for external observers to analyze the economics of inference. However, the revenue figures alone do not indicate profit levels. To determine how much gross profit corresponds to the $38 per hour revenue, one must also consider costs such as hardware depreciation, energy, networking, caching storage, and operational maintenance.

Social Media Estimates Gross Margin Exceeding 96%

Overseas technology investment researcher Zephyr published calculations on social platform X, indicating that DeepSeek has effectively "revealed the core secret behind its business model." He estimates that DeepSeek's inference gross margin could reach 84.2% to 96.1%, correcting an earlier range of 87.76% to 96.94%.

He established a typical base model: assuming that in actual production, 97% of inputs hit the key-value cache, with 2% of inputs missing the cache and 1% as output.

Cached Input: Accounts for 97% of total token volume (very low unit price, extremely light computational load);

Uncached Input/Prefill: Accounts for 2% of total token volume;

Output/Decode: Accounts for 1% of total token volume.

Under this traffic structure, Zephyr combined the officially published single-card maximum throughput (80,000 tokens per second for prefill, 16,000 tokens per second for decode) to estimate output value. According to his model, a fully loaded GPU could theoretically process the corresponding token combinations and generate approximately $49 in revenue per hour.

If we consider the current comprehensive rental cost of a top-tier GPU (such as NVIDIA H100 or similar next-generation hardware) in the overseas public cloud market, which generally ranges from $1.5 to $4 per hour, the hardware-level inference gross margin formula is:

Under extreme testing conditions (producing $49 per hour, with computing costs ranging from $1.5 to $6):

Zephyr stated that under specific ideal loads, the corresponding inference business gross margin of the framework is between 87.76% and 96.94%. He candidly remarked in his tweet, "NVIDIA, SGLang, and vLLM's public performance has been directly crushed."

Subsequently, Zephyr revised the calculation parameters based on the real operational details disclosed by DeepSeek. He added, "These extreme gross margins are based on ideal scenarios, where service providers can produce about $49 per hour. DeepSeek officially disclosed that its actual revenue per card during non-peak hours is about $38, and the production environment can achieve approximately 80% of benchmark testing performance. Based on these latest figures, the actual operating gross margin falls between 84.2% and 96.1%."

Currently, in the overseas cloud service market, the comprehensive cost of renting a top-tier GPU generally ranges from $2 to $4 per hour. The significant gap between the $38 per card hourly output value and the unit computing cost is the core basis for driving up gross margin estimates.

Industry Horizontal Analysis: Gross Margin of Leading Labs

Following the release of this calculation, it quickly sparked a chain discussion within the industry regarding the actual profitability of leading AI labs.

Technology investment observer Alex Corrino questioned while sharing the above calculation, "What if I told you that the API gross margins of all top AI labs are likely above 80%?"

Zephyr immediately quoted the tweet and provided a more aggressive judgment: "For Anthropic and OpenAI, this number is actually 95% (applicable to mainstream models like Sol, Fable, Sonnet, Astra, etc.)."

Is Anthropic Definitely Making Big Money?

Under the premise that the entire industry generally has high gross margins, investment analyst Scott further translated this logic into the end-user "subscription" economic model, deducing the real cards of leading overseas players.

According to a practical report from research institution SemiAnalysis, the actual usable token limit provided by Anthropic for its premium member "Claude Max" reached 58.6 times the corresponding official API unit price. This led to speculation that high-usage subscription users would severely erode platform profits, resulting in losses.

Scott provided a reverse deduction formula:

Setting the baseline premise: Assume that Anthropic's API hardware gross margin is also as high as 99% (cost rate is only 1%).

Cost reverse deduction when subscription usage is fully utilized: Users pay 1 unit of subscription fee, and if they completely exhaust the equivalent of 58.6 units of API priced computing capacity, the actual underlying hardware cost borne by Anthropic is:

Actual hardware cost = 58.6 × 1% = 0.586 units of actual hardware cost = 58.6 × 1% = 0.586 units

Deducing the subscription gross margin under extreme conditions:

Claude Max extreme gross margin = 1 - 0.586 = 41.4%

Scott analyzed that:

"If DeepSeek's gross margin can reach above 95%, assuming Anthropic's API gross margin is 99%. Based on SemiAnalysis's testing, Claude Max provides up to 58.6 times the API pricing in usage limits. This means that even if all users fully utilize their quotas, Anthropic can still retain about 41% gross margin under the subscription model. In reality, the vast majority of users will not reach their weekly limits."

He pointed out, "In fact, as long as Anthropic's API profit margin remains above 98.3%, even if users fully utilize their quotas, its subscription business can achieve a break-even point. Anthropic is currently in a robust profit state."

Technology analysis account Teortaxes also commented, "In the face of the internally developed architectures of leading labs, all public benchmark tests on the market are insignificant. This is the result of serious engineering work; the inference business itself is a money-printing machine. You can imagine how anxious Dario (CEO of Anthropic) must be at this moment."

Technical Path Divergence: Software Reconstruction vs. Dedicated Chips

As the real inference cost structure emerges, the industry is also re-examining different technical cost-reduction routes.

In terms of hardware architecture and unit throughput costs, the industry has compared this solution with chip manufacturers like Groq, which focus on ultra-low latency.

Industry observers pointed out that the LPU custom hardware architecture represented by Groq relies on onboard SRAM, achieving extreme interactive generation speed, but its chip memory capacity is very small, requiring multiple hardware boards and large node interconnections to support mainstream large models, resulting in high overall hardware depreciation and procurement thresholds per token.

In contrast, DeepInfer demonstrates another ledger logic for the general chip path: purely through software scheduling to maximize concurrent throughput on general GPUs and commercial NPUs like Ascend, exchanging lower comprehensive hardware costs for profit margins in scaled API services.

However, several industry observers noted in discussions that the aforementioned gross margin estimates of 80% to 95% primarily focus on the direct financial difference between "token pricing and single-card hardware consumption." This micro-calculation does not account for the hundreds of millions of dollars in R&D expenses for the foundational model, the high-speed bandwidth network costs of global nodes, and the high salaries of systems engineering teams.

Next, it needs to be verified whether the open-source implementation can stably increase throughput and reduce the cost per effective output token under different hardware and actual business loads. For DeepSeek itself, external parties also need more data on cost and resource utilization to determine whether the high gross profit estimated by social media is close to the actual operating level.