[Benchmark] Add dummy prefill support for DeepSeek V3 MLA profiling#7776
[Benchmark] Add dummy prefill support for DeepSeek V3 MLA profiling#7776chang-wenbin wants to merge 6 commits into
Conversation
|
Thanks for your contribution! |
|
chang-wenbin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览
2 任务状态汇总2.1 Required任务 : 7/10 通过
2.2 可选任务 — 23/26 通过
3 失败详情（仅 required）Approval — 代码审批（置信度: 高）Approval
根因详情: 关键日志: 修复建议:
修复建议摘要: 请 jiangjiajun/liuyuanle/chenjian26/wanglongzhi 之一 Approve PR 关联变更: PR 修改了 |
chang-wenbin
changed the title
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #7776 +/- ##
==========================================
Coverage ? 71.48%
==========================================
Files ? 396
Lines ? 55862
Branches ? 8737
==========================================
Hits ? 39931
Misses ? 13173
Partials ? 2758
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-11 20:09:06
📋 Review 摘要
PR 概述：为 MoE 模型在 dummy 推理模式下添加 FD_RUN_DUMMY_FOR_PROFILE 环境变量支持，实现 gate 随机替换与端到端耗时测量
变更范围：fastdeploy/envs.py、model_executor/layers/moe/（3 个 backend）、worker/gpu_model_runner.py、worker/worker_process.py
影响面 Tag：[Benchmark] [OP]
📝 PR 规范检查
存在两处问题：
## Modifications描述的是linear.py、deepseek_v3.py等未出现在 diff 中的文件，明显是历史遗留/错贴内容；## Motivation及## Usage or Command中仍使用旧环境变量名RUN_DUMMY_FOR_PROFILE，与实际代码改用的FD_RUN_DUMMY_FOR_PROFILE不一致。
标题建议（可直接复制）：
[Benchmark] Add FD_RUN_DUMMY_FOR_PROFILE support for MoE dummy profiling
PR 描述建议（可直接复制）：
## Motivation
为支持 MoE 模型（含 DeepSeek V3）在 dummy 模式（不加载真实权重）下完成前向推理，用于性能 profiling 和 benchmark 场景。通过环境变量 `FD_RUN_DUMMY_FOR_PROFILE=1` 开启端到端耗时测量，并使用随机数据替换 gate 输出以避免零权重导致路由退化。同时移除 EP profiling 模式下的 token 数量上限（32）限制，并将旧环境变量 `RUN_DUMMY_FOR_PROFILE` 统一迁移至 `FD_RUN_DUMMY_FOR_PROFILE`。
## Modifications
- `fastdeploy/envs.py`：新增环境变量 `FD_RUN_DUMMY_FOR_PROFILE`（默认 0），统一管理 dummy profiling 开关
- `fastdeploy/model_executor/layers/moe/fused_moe_blackwell_backend.py`：在 `apply_ep_prefill`、`apply_ep_decode`、`apply_tp` 中，当 `FD_RUN_DUMMY_FOR_PROFILE=1` 时用随机数据替换 gate 输出
- `fastdeploy/model_executor/layers/moe/fused_moe_cutlass_backend.py`：在 4 处 gate_out 计算后添加 dummy 随机替换
- `fastdeploy/model_executor/layers/moe/fused_moe_deepgemm_backend.py`：在 3 处 gate_out 计算后添加 dummy 随机替换
- `fastdeploy/worker/gpu_model_runner.py`：将 EP 模式下 `input_length <= 32` 的限制改为仅在非 profiling 模式下生效；在 `FD_RUN_DUMMY_FOR_PROFILE=1` 时增加分布式 barrier 和端到端耗时打印
- `fastdeploy/worker/worker_process.py`：将旧的 `os.getenv("RUN_DUMMY_FOR_PROFILE", "0")` 改为使用 `envs.FD_RUN_DUMMY_FOR_PROFILE`
## Usage or Command
FD_RUN_DUMMY_FOR_PROFILE=1 python -m fastdeploy.worker.worker_process ...
## Accuracy Tests
N/A（本 PR 为 profiling/benchmarking 工具，不影响模型推理精度）
## Checklist
- [ ] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 📝 PR 规范 | N/A | Modifications 段落描述了不在 diff 中的文件；Motivation/Usage 中引用了旧环境变量名 RUN_DUMMY_FOR_PROFILE |
| 🟡 建议 | N/A | fused_moe_triton/marlin/wint2_backend.py 三个 MoE backend 未添加 FD_RUN_DUMMY_FOR_PROFILE 支持，若使用这些 backend 时 dummy profiling 行为不一致 |
| ❓ 疑问 | worker_process.py:1336 |
上方注释仍引用旧环境变量名 RUN_DUMMY_FOR_PROFILE |
总体评价
改动整体思路清晰，正确通过环境变量控制 dummy profiling 模式，EP 限制放开逻辑合理。主要问题是 PR 描述与实际变更文件不符需要更新，以及 triton/marlin/wint2 三个 MoE backend 未同步添加 dummy gate 支持，建议补齐以保证一致性。
5 participants
Motivation
为支持 DeepSeek V3 MLA 模型在 dummy 模式（不加载真实权重）下完成前向推理，用于性能 profiling 和 benchmark 场景。通过环境变量
RUN_DUMMY_FOR_PROFILE=1开启端到端耗时测量，同时修正 value padding 的条件判断逻辑。Modifications
fastdeploy/model_executor/layers/linear.py：KVBatchLinear.__init__在load_choices == "dummy"时直接创建零初始化的k_b_proj_weight/v_b_proj_weight参数，无需加载真实权重fastdeploy/model_executor/models/deepseek_v3.py：forward中为 value padding 添加条件判断，仅当qk_head_dim != v_head_dim时才执行 pad 操作，避免无效计算fastdeploy/worker/gpu_model_runner.py：将 EP 模式下input_length <= 32的限制改为仅在非 profiling 模式下生效；在RUN_DUMMY_FOR_PROFILE=1时增加分布式 barrier 和端到端耗时打印Usage or Command
Accuracy Tests
N/A（本 PR 为 profiling/benchmarking 工具，不影响模型推理精度）
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.