DeepSeek: A Revolutionary Chinese Chatbot

DeepSeek, a Chinese chatbot, has created a stir in the world of Artificial Intelligence (AI) and potentially posed a threat to a $2 trillion business empire like Nvidia.

Background:

Training AI models has become extraordinarily expensive these days.
Organizations like OpenAI and Anthropic spend over $100 million on computing. This requires highly expensive Graphic Processing Units (GPUs) and large data centers—akin to needing an entire power plant to run a single factory.

DeepSeek’s Revolutionary Progress:

DeepSeek claimed:
“All of this is possible in just $5 million,”
and they turned this claim into reality.
DeepSeek’s models have outperformed leading AI models like GPT-4 and Claude in numerous tasks.

Secrets Behind DeepSeek’s Success:

1. Memory Optimization:
Traditional AI stores every number with 32 numerical points, but DeepSeek reduced this to just 8 numerical points.
Result? A 75% reduction in memory usage during training.

2. Multi-Token System:
Traditional models process words one by one, for example:
“The… cat… is… on… the… roof.”
In contrast, DeepSeek processes an entire sentence at once, operating at double the speed and achieving 90% accuracy.

3. Expert System:
Traditional AI keeps all its parameters active simultaneously. DeepSeek, however, only activates the relevant parameters for a task.
For example, out of 1.8 trillion parameters, only 37 billion are active at a given time.

Results Achieved by DeepSeek:

Training costs reduced from $100 million to just $5 million.

GPU requirements dropped from 100,000 to only 2,000.

API costs decreased by 95%.

Capability to operate on standard gaming GPUs.

Open-source code made publicly available.

A Threat to Nvidia:

Nvidia’s business model relies heavily on selling expensive GPUs. If AI technology becomes operable on standard gaming GPUs, it could significantly impact Nvidia’s business.

The DeepSeek Team:

DeepSeek’s team comprises fewer than 200 people, while companies like Meta spend more on employee salaries than DeepSeek’s entire budget.

Importance and Impact:

The development of AI will become more accessible.

Competition will increase.

Monopolies of large corporations may come to an end.

Hardware costs will drastically decrease.

Conclusion:

This could be the moment when the world of AI changes forever. While the speed of this transformation remains to be seen, the future appears to be one where AI becomes more affordable and available to everyone.

Research & Compilation:

Shariq Ali
Valueversity