[Store] Enable NUMA-segmented allocation for RDMA RealClient-only mode#1838
[Store] Enable NUMA-segmented allocation for RDMA RealClient-only mode#1838ykwd merged 1 commit intokvcache-ai:mainfrom
Conversation
Previously, NUMA-aware global segment allocation was gated behind `!ipc_socket_path_.empty()`, meaning it only activated in standalone (Dummy+Real Client) mode. In practice, the vast majority of deployments (mooncake_store_service, sglang/vLLM integration, Python store.setup()) use RealClient-only mode where ipc_socket_path is always empty, so NUMA optimization never triggered. Remove the ipc_socket_path condition so that all RDMA deployments benefit from NUMA-segmented memory allocation and NUMA-aware NIC routing via selectDevice.
00fish0
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the setup_internal method in mooncake-store/src/real_client.cpp to ensure that NUMA node discovery and distribution for RDMA protocols occur regardless of whether the IPC socket path is set. This change expands the application of NIC-based NUMA optimization beyond standalone mode. I have no feedback to provide as no review comments were present.
There was a problem hiding this comment.
Pull request overview
This PR enables NUMA-segmented global segment allocation for all RDMA deployments in Mooncake Store by removing the previous standalone-mode (ipc_socket_path_ non-empty) gating, allowing more deployments to benefit from NUMA-aware NIC routing.
Changes:
- Remove the
!ipc_socket_path_.empty()condition so NUMA-segmented allocation can activate wheneverprotocol == "rdma". - Update the in-code comment to reflect the broader applicability (not standalone-only).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // For RDMA, auto-discover NUMA nodes with NICs and distribute | ||
| // global_segment across them for full NIC utilization. | ||
| std::vector<int> seg_numa_nodes; | ||
| if (!ipc_socket_path_.empty() && protocol == "rdma") { | ||
| if (protocol == "rdma") { | ||
| seg_numa_nodes = client_->GetNicNumaNodes(); |
There was a problem hiding this comment.
Removing the ipc_socket_path_ gate makes NUMA-segmented allocation take effect in more RDMA deployments. In that mode, page_sz is derived from get_hugepage_size_from_env() when should_use_hugepage is true, but allocate_buffer_numa_segments() currently uses a plain anonymous mmap (no hugepage flags), so enabling segmented mode can effectively bypass hugepage-backed allocation for multi-NUMA NIC hosts. Consider either (a) explicitly documenting that segmented allocation takes precedence over hugepages, or (b) extending allocate_buffer_numa_segments() to support hugepage mappings when should_use_hugepage is enabled.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Description
Previously, NUMA-aware global segment allocation (#1580) was gated behind
!ipc_socket_path_.empty(), meaning it only activated in Dummy+Real Client mode. In practice, the vast majority of deployments (mooncake_store_service, sglang/vLLM integration, Python store.setup()) use RealClient-only mode.Remove the ipc_socket_path condition so that all RDMA deployments benefit from NUMA-segmented memory allocation and NUMA-aware NIC routing via selectDevice.
Module
mooncake-transfer-engine)mooncake-store)mooncake-ep)mooncake-integration)mooncake-p2p-store)mooncake-wheel)mooncake-pg)mooncake-rl)Type of Change
How Has This Been Tested?
Checklist
./scripts/code_format.shbefore submitting.