Support Quantization #209

Merged

wooway777 merged 16 commits intoInfiniTensor:demo131from

qinyiqun:demo131_quant

Feb 11, 2026

Commits on Feb 11, 2026

issue/204 - support graph in server scripts
wooway777
committed
issue/208 - adapt to ali ppu
wooway777
committed
issue/194 - add quantization modify configs accordingly

qinyiqun
authored and
wooway777
committed
issue/175 - qy device support

xgqdut2016
authored and
wooway777
committed
Issue/170 - Add HYGON support and improve device type handling.

gongchensu
authored and
wooway777
committed
Issue/193: feats for deployment

Ceng23333
authored and
wooway777
committed
skip responding eos token

Ceng23333
authored and
wooway777
committed
issue/143 use add_rmsnorm, nt flash attn, nt kv caching

PanZezhong1725
authored and
wooway777
committed
issue/204 - support graph in server scripts
wooway777
committed
issue/208 - adapt to ali ppu
wooway777
committed
rebase main

qinyiqun
authored and
wooway777
committed
issue/216 feat: support static kv cache in server

ma-hang
authored and
wooway777
committed
fix llm server cache config

qinyiqun
authored and
wooway777
committed
demo131 - resolve mishandled conflicts
wooway777
committed
demo131 - further adjust attn and caching logic
wooway777
committed
demo131 - resolve merge requirements
wooway777
committed