Instructions to use unsloth/DeepSeek-R1-0528-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/DeepSeek-R1-0528-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="unsloth/DeepSeek-R1-0528-GGUF", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("unsloth/DeepSeek-R1-0528-GGUF", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("unsloth/DeepSeek-R1-0528-GGUF", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use unsloth/DeepSeek-R1-0528-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL # Run inference directly in the terminal: llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL # Run inference directly in the terminal: llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL # Run inference directly in the terminal: ./build/bin/llama cli -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Use Docker
docker model run hf.co/unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
- Llama new
- LM Studio
- Jan
- vLLM
How to use unsloth/DeepSeek-R1-0528-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unsloth/DeepSeek-R1-0528-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-0528-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
- SGLang
How to use unsloth/DeepSeek-R1-0528-GGUF with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-R1-0528-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-0528-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-R1-0528-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-0528-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use unsloth/DeepSeek-R1-0528-GGUF with Ollama:
ollama run hf.co/unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
- Unsloth Desktop
- Pi
How to use unsloth/DeepSeek-R1-0528-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use unsloth/DeepSeek-R1-0528-GGUF with Docker Model Runner:
docker model run hf.co/unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
- Lemonade
How to use unsloth/DeepSeek-R1-0528-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Run and chat with the model
lemonade run user.DeepSeek-R1-0528-GGUF-UD-Q4_K_XL
List all available models
lemonade list
- Hermes Agent
How to use unsloth/DeepSeek-R1-0528-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use unsloth/DeepSeek-R1-0528-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "unsloth/DeepSeek-R1-0528-GGUF:UD-Q4_K_XL" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
For DeepSeek-R1-0528-Qwen3-8B GGUFs, see here.
Learn how to run DeepSeek-R1-0528 correctly - Read our Guide.
See our collection for all versions of R1 including GGUF, 4-bit & 16-bit formats.
Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.
đ DeepSeek-R1-0528 Usage Guidelines
- Set the temperature between 0.5â0.7 (0.6 recommended) to reduce repetition and incoherence.
- Set Top_P value of 0.95 (recommended)
- R1-0528 uses the same chat template as the original R1 model:
<ď˝beginâofâsentenceď˝><ď˝Userď˝>What is 1+1?<ď˝Assistantď˝>It's 2.<ď˝endâofâsentenceď˝><ď˝Userď˝>Explain more!<ď˝Assistantď˝>
- For llama.cpp / GGUF inference, you should skip the BOS since itâll auto add it:
<ď˝Userď˝>What is 1+1?<ď˝Assistantď˝>
- For complete detailed instructions, see our guide: unsloth.ai/blog/deepseek-r1-0528
DeepSeek-R1-0528 Model Card
1. Introduction
The DeepSeek R1 model has undergone a minor version upgrade, with the current version being DeepSeek-R1-0528. In the latest update, DeepSeek R1 has significantly improved its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training. The model has demonstrated outstanding performance across various benchmark evaluations, including mathematics, programming, and general logic. Its overall performance is now approaching that of leading models, such as O3 and Gemini 2.5 Pro.
Compared to the previous version, the upgraded model shows significant improvements in handling complex reasoning tasks. For instance, in the AIME 2025 test, the modelâs accuracy has increased from 70% in the previous version to 87.5% in the current version. This advancement stems from enhanced thinking depth during the reasoning process: in the AIME test set, the previous model used an average of 12K tokens per question, whereas the new version averages 23K tokens per question.
Beyond its improved reasoning capabilities, this version also offers a reduced hallucination rate, enhanced support for function calling, and better experience for vibe coding.
2. Evaluation Results
DeepSeek-R1-0528
For all our models, the maximum generation length is set to 64K tokens. For benchmarks requiring sampling, we use a temperature of $0.6$, a top-p value of $0.95$, and generate 16 responses per query to estimate pass@1.
| Category | Benchmark (Metric) | DeepSeek R1 | DeepSeek R1 0528 |
|---|---|---|---|
| General | |||
| MMLU-Redux (EM) | 92.9 | 93.4 | |
| MMLU-Pro (EM) | 84.0 | 85.0 | |
| GPQA-Diamond (Pass@1) | 71.5 | 81.0 | |
| SimpleQA (Correct) | 30.1 | 27.8 | |
| FRAMES (Acc.) | 82.5 | 83.0 | |
| Humanity's Last Exam (Pass@1) | 8.5 | 17.7 | |
| Code | |||
| LiveCodeBench (2408-2505) (Pass@1) | 63.5 | 73.3 | |
| Codeforces-Div1 (Rating) | 1530 | 1930 | |
| SWE Verified (Resolved) | 49.2 | 57.6 | |
| Aider-Polyglot (Acc.) | 53.3 | 71.6 | |
| Math | |||
| AIME 2024 (Pass@1) | 79.8 | 91.4 | |
| AIME 2025 (Pass@1) | 70.0 | 87.5 | |
| HMMT 2025 (Pass@1) | 41.7 | 79.4 | |
| CNMO 2024 (Pass@1) | 78.8 | 86.9 | |
| Tools | |||
| BFCL_v3_MultiTurn (Acc) | - | 37.0 | |
| Tau-Bench (Pass@1) | - | 53.5(Airline)/63.9(Retail) |
DeepSeek-R1-0528-Qwen3-8B
Meanwhile, we distilled the chain-of-thought from DeepSeek-R1-0528 to post-train Qwen3 8B Base, obtaining DeepSeek-R1-0528-Qwen3-8B. This model achieves state-of-the-art (SOTA) performance among open-source models on the AIME 2024, surpassing Qwen3 8B by +10.0% and matching the performance of Qwen3-235B-thinking. We believe that the chain-of-thought from DeepSeek-R1-0528 will hold significant importance for both academic research on reasoning models and industrial development focused on small-scale models.
| AIME 24 | AIME 25 | HMMT Feb 25 | GPQA Diamond | LiveCodeBench (2408-2505) | |
|---|---|---|---|---|---|
| Qwen3-235B-A22B | 85.7 | 81.5 | 62.5 | 71.1 | 66.5 |
| Qwen3-32B | 81.4 | 72.9 | - | 68.4 | - |
| Qwen3-8B | 76.0 | 67.3 | - | 62.0 | - |
| Phi-4-Reasoning-Plus-14B | 81.3 | 78.0 | 53.6 | 69.3 | - |
| Gemini-2.5-Flash-Thinking-0520 | 82.3 | 72.0 | 64.2 | 82.8 | 62.3 |
| o3-mini (medium) | 79.6 | 76.7 | 53.3 | 76.8 | 65.9 |
| DeepSeek-R1-0528-Qwen3-8B | 86.0 | 76.3 | 61.5 | 61.1 | 60.5 |
3. Chat Website & API Platform
You can chat with DeepSeek-R1 on DeepSeek's official website: chat.deepseek.com, and switch on the button "DeepThink"
We also provide OpenAI-Compatible API at DeepSeek Platform: platform.deepseek.com
4. How to Run Locally
Please visit DeepSeek-R1 repository for more information about running DeepSeek-R1-0528 locally.
Compared to previous versions of DeepSeek-R1, the usage recommendations for DeepSeek-R1-0528 have the following changes:
- System prompt is supported now.
- It is not required to add "<think>\n" at the beginning of the output to force the model into thinking pattern.
The model architecture of DeepSeek-R1-0528-Qwen3-8B is identical to that of Qwen3-8B, but it shares the same tokenizer configuration as DeepSeek-R1-0528. This model can be run in the same manner as Qwen3-8B.
System Prompt
In the official DeepSeek web/app, we use the same system prompt with a specific date.
诼ĺŠć为DeepSeek-R1ďźçąćˇąĺşŚćąç´˘ĺ
Źĺ¸ĺé ă
äťĺ¤ŠćŻ{current date}ă
For example,
诼ĺŠć为DeepSeek-R1ďźçąćˇąĺşŚćąç´˘ĺ
Źĺ¸ĺé ă
äťĺ¤ŠćŻ2025ĺš´5ć28ćĽďźććä¸ă
Temperature
In our web and application environments, the temperature parameter $T_{model}$ is set to 0.6.
Prompts for File Uploading and Web Search
For file uploading, please follow the template to create prompts, where {file_name}, {file_content} and {question} are arguments.
file_template = \
"""[file name]: {file_name}
[file content begin]
{file_content}
[file content end]
{question}"""
For Web Search, {search_results}, {cur_date}, and {question} are arguments. For Chinese query, we use the prompt:
search_answer_zh_template = \
'''# 䝼ä¸ĺ
厚ćŻĺşäşç¨ćˇĺéçćśćŻçćç´˘çťć:
{search_results}
ĺ¨ćçťä˝ çćç´˘çťćä¸ďźćŻä¸Şçťćé˝ćŻ[webpage X begin]...[webpage X end]ć źĺźçďźX䝣襨ćŻçŻćçŤ çć°ĺç´˘ĺźă诡ĺ¨éĺ˝çć
ĺľä¸ĺ¨ĺĽĺćŤĺ°žĺźç¨ä¸ä¸ćă诡ćç
§ĺźç¨çźĺˇ[citation:X]çć źĺźĺ¨çćĄä¸ĺŻšĺşé¨ĺĺźç¨ä¸ä¸ćăĺŚćä¸ĺĽčŻćşčŞĺ¤ä¸Şä¸ä¸ćďźčݎĺĺşććç¸ĺ
łçĺźç¨çźĺˇďźäžĺŚ[citation:3][citation:5]ďźĺ莰ä¸čŚĺ°ĺźç¨éä¸ĺ¨ćĺčżĺĺźç¨çźĺˇďźčćŻĺ¨çćĄĺŻšĺşé¨ĺĺĺşă
ĺ¨ĺçćśďźčŻˇćł¨ć䝼ä¸ĺ çšďź
- äťĺ¤ŠćŻ{cur_date}ă
- ĺšśéćç´˘çťćçććĺ
厚é˝ä¸ç¨ćˇçéŽé˘ĺŻĺç¸ĺ
łďźä˝ éčŚçťĺéŽé˘ďźĺŻšćç´˘çťćčżčĄçĺŤăçéă
- 寚äşĺ丞繝çéŽé˘ďźĺŚĺ丞ććčŞç俥ćŻďźďźĺ°˝éĺ°çćĄć§ĺśĺ¨10个čŚçšäťĽĺ
ďźĺšśĺčŻç¨ćˇĺŻäťĽćĽçćç´˘ćĽćşăčˇĺžĺŽć´äżĄćŻăäźĺ
ćäžäżĄćŻĺŽć´ăćç¸ĺ
łçĺ丞饚ďźĺŚéĺż
čŚďźä¸čŚä¸ťĺ¨ĺčŻç¨ćˇćç´˘çťććŞćäžçĺ
厚ă
- 寚äşĺä˝çąťçéŽé˘ďźĺŚĺ莺ćďźďźčݎĺĄĺż
ĺ¨ćŁćç掾č˝ä¸ĺźç¨ĺŻšĺşçĺčçźĺˇďźäžĺŚ[citation:3][citation:5]ďźä¸č˝ĺŞĺ¨ćçŤ ćŤĺ°žĺźç¨ăä˝ éčŚč§ŁčŻťĺšśćŚćŹç¨ćˇçé˘çŽčŚćąďźéćŠĺéçć źĺźďźĺ
ĺĺŠç¨ćç´˘çťćĺšść˝ĺéčŚäżĄćŻďźçć珌ĺç¨ćˇčŚćąăćĺ
ˇććłćˇąĺşŚăĺŻćĺé ĺä¸ä¸ä¸ć§ççćĄăä˝ çĺä˝çŻĺš
éčŚĺ°˝ĺŻč˝ĺťśéżďźĺŻšäşćŻä¸ä¸ŞčŚçšç莺述čŚć¨ćľç¨ćˇçćĺžďźçťĺşĺ°˝ĺŻč˝ĺ¤č§ĺşŚçĺçčŚçšďźä¸ĺĄĺż
俥ćŻé大ă莺述诌尽ă
- ĺŚćĺçĺžéżďźčŻˇĺ°˝éçťćĺăĺ掾č˝ćťçťăĺŚćéčŚĺçšä˝çďźĺ°˝éć§ĺśĺ¨5个çšäťĽĺ
ďźĺšśĺĺšśç¸ĺ
łçĺ
厚ă
- 寚äşĺŽ˘č§çąťçéŽçďźĺŚćéŽé˘ççćĄé常çŽçďźĺŻäťĽéĺ˝čĄĽĺ
ä¸ĺ°ä¸¤ĺĽç¸ĺ
łäżĄćŻďźäťĽä¸°ĺŻĺ
厚ă
- ä˝ éčŚć šćŽç¨ćˇčŚćąĺĺçĺ
厚éćŠĺéăçžč§çĺçć źĺźďźçĄŽäżĺŻčŻťć§ĺźşă
- ä˝ çĺçĺşčŻĽçťźĺĺ¤ä¸Şç¸ĺ
łç˝éĄľćĽĺçďźä¸č˝éĺ¤ĺźç¨ä¸ä¸Şç˝éĄľă
- é¤éç¨ćˇčŚćąďźĺŚĺä˝ ĺççčŻč¨éčŚĺç¨ćˇćéŽçčŻč¨äżćä¸č´ă
# ç¨ćˇćśćŻä¸şďź
{question}'''
For English query, we use the prompt:
search_answer_en_template = \
'''# The following contents are the search results related to the user's message:
{search_results}
In the search results I provide to you, each result is formatted as [webpage X begin]...[webpage X end], where X represents the numerical index of each article. Please cite the context at the end of the relevant sentence when appropriate. Use the citation format [citation:X] in the corresponding part of your answer. If a sentence is derived from multiple contexts, list all relevant citation numbers, such as [citation:3][citation:5]. Be sure not to cluster all citations at the end; instead, include them in the corresponding parts of the answer.
When responding, please keep the following points in mind:
- Today is {cur_date}.
- Not all content in the search results is closely related to the user's question. You need to evaluate and filter the search results based on the question.
- For listing-type questions (e.g., listing all flight information), try to limit the answer to 10 key points and inform the user that they can refer to the search sources for complete information. Prioritize providing the most complete and relevant items in the list. Avoid mentioning content not provided in the search results unless necessary.
- For creative tasks (e.g., writing an essay), ensure that references are cited within the body of the text, such as [citation:3][citation:5], rather than only at the end of the text. You need to interpret and summarize the user's requirements, choose an appropriate format, fully utilize the search results, extract key information, and generate an answer that is insightful, creative, and professional. Extend the length of your response as much as possible, addressing each point in detail and from multiple perspectives, ensuring the content is rich and thorough.
- If the response is lengthy, structure it well and summarize it in paragraphs. If a point-by-point format is needed, try to limit it to 5 points and merge related content.
- For objective Q&A, if the answer is very brief, you may add one or two related sentences to enrich the content.
- Choose an appropriate and visually appealing format for your response based on the user's requirements and the content of the answer, ensuring strong readability.
- Your answer should synthesize information from multiple relevant webpages and avoid repeatedly citing the same webpage.
- Unless the user requests otherwise, your response should be in the same language as the user's question.
# The user's message is:
{question}'''
5. License
This code repository is licensed under MIT License. The use of DeepSeek-R1 models is also subject to MIT License. DeepSeek-R1 series (including Base and Chat) supports commercial use and distillation.
6. Citation
@misc{deepseekai2025deepseekr1incentivizingreasoningcapability,
title={DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning},
author={DeepSeek-AI},
year={2025},
eprint={2501.12948},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2501.12948},
}
7. Contact
If you have any questions, please raise an issue or contact us at service@deepseek.com.
- Downloads last month
- 14,163
1-bit
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Model tree for unsloth/DeepSeek-R1-0528-GGUF
Base model
deepseek-ai/DeepSeek-R1-0528