Skip to content

[Store] Add generic batched metadata-backed buffer fetch#1834

Open
zxpdemonio wants to merge 4 commits intokvcache-ai:mainfrom
openanolis:cruz/batch_query
Open

[Store] Add generic batched metadata-backed buffer fetch#1834
zxpdemonio wants to merge 4 commits intokvcache-ai:mainfrom
openanolis:cruz/batch_query

Conversation

@zxpdemonio
Copy link
Copy Markdown
Collaborator

Description

For batched reads, upper-layer components may first fetch object metadata for a group of keys and keep the results in a query cache. Before this change, the Mooncake Store standalone client path could not directly reuse those cached query results for
get_buffer-style reads, so it had to issue another metadata lookup before starting data transfer.

This meant query cache could not be effectively applied to this path, and batched reads paid an unnecessary extra metadata round trip.

This PR introduces the generic Mooncake Store primitives needed to support query-cache-backed batched buffer fetches, without tying the implementation to any Engram-specific logic. To achieve that, we:

  • add batched metadata query support for standalone clients
  • add batched buffer fetch APIs that reuse cached query results when their leases are still valid
  • preserve per-key query outcomes, including misses and lease validation, when cached metadata is consumed
  • add the minimal standalone client-side plumbing required to pass cached query results through the existing dummy/real client path
  • update documentation and add dedicated tests for the new batch query + cached metadata read flow

Module

  • Transfer Engine (mooncake-transfer-engine)
  • Mooncake Store (mooncake-store)
  • Mooncake EP (mooncake-ep)
  • Integration (mooncake-integration)
  • P2P Store (mooncake-p2p-store)
  • Python Wheel (mooncake-wheel)
  • PyTorch Backend (mooncake-pg)
  • Mooncake RL (mooncake-rl)
  • CI/CD
  • Docs
  • Other

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Breaking change
  • Documentation update
  • Other

How Has This Been Tested?

Ran the following tests:

  • ctest --output-on-failure -R '^(dummy_client_get_buffer_test|batch_query_buffer_fetch_test)$'

Both tests passed.

Checklist

  • I have performed a self-review of my own code.
  • I have formatted my own code using ./scripts/code_format.sh before submitting.
  • I have updated the documentation.
  • I have added tests to prove my changes are effective.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a two-step read optimization for batched buffer operations in the Mooncake Store, allowing callers to reuse metadata and avoid redundant lookups. Key changes include the addition of batch_query and batch_get_buffer_with_query_results to both DummyClient and RealClient, along with RPC utilities for serializing query results and a centralized RPC service registration. The implementation handles hot cache integration and ensures that expired leases are rejected. A new test suite, batch_query_buffer_fetch_test.cpp, has been added to verify these flows. I have no feedback to provide as there are no review comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant