Skip to content

[Op][Optimization]Kernel fusion: cast+sigmoid+bias+noauxtc #7777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ShaneGZhu wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

[Op][Optimization]Kernel fusion: cast+sigmoid+bias+noauxtc #7777

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
68443e9
[Ops][Optimization]Kernel fusion: cast+sigmoid+bias+noauxtc
ShaneGZhu May 11, 2026
ec06609
Merge remote-tracking branch 'origin/develop' into get_moe_score
ShaneGZhu May 11, 2026
49563e4
Add unit_test file
ShaneGZhu May 11, 2026
a132799
Fixed the 1e-8 precision issue that had been introduced; the current …
ShaneGZhu May 12, 2026
0495e88
clean
ShaneGZhu May 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions custom_ops/gpu_ops/cpp_extensions.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,15 @@ std::vector<paddle::Tensor> NoauxTc(paddle::Tensor& scores,
bool renormalize,
float routed_scaling_factor);

std::vector<paddle::Tensor> grouped_topk(
paddle::Tensor& gating_output,
paddle::Tensor& e_score_correction_bias,
int n_group,
int topk_group,
int topk,
bool renormalize,
float routed_scaling_factor);

std::vector<paddle::Tensor> FusedCastSigmoidBias(const paddle::Tensor& input,
const paddle::Tensor& bias,
std::string cast_type);
Expand Down Expand Up @@ -1707,6 +1716,8 @@ PYBIND11_MODULE(fastdeploy_ops, m) {

m.def("noaux_tc", &NoauxTc, "noaux_tc for Deepseekv3 MoE compute");

m.def("grouped_topk", &grouped_topk, "fused grouped topk for MoE routing");

m.def("fused_cast_sigmoid_bias",
&FusedCastSigmoidBias,
"Fused cast+sigmoid+bias for MoE gating scores",
Expand Down
Loading
Loading