Skip to content

Comments

eventstore: add config for zstd compression#4262

Open
lidezhu wants to merge 6 commits intomasterfrom
ldz/add-compression-config
Open

eventstore: add config for zstd compression#4262
lidezhu wants to merge 6 commits intomasterfrom
ldz/add-compression-config

Conversation

@lidezhu
Copy link
Collaborator

@lidezhu lidezhu commented Feb 24, 2026

What problem does this PR solve?

Issue Number: close #4257

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

Summary by CodeRabbit

  • New Features
    • Introduced a new configuration option to control zstd compression for event storage values. Compression is enabled by default to optimize storage efficiency and can be disabled through server configuration if needed.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 24, 2026
@gemini-code-assist
Copy link

Summary of Changes

Hello @lidezhu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the event store's configuration by introducing an explicit option to enable or disable Zstd compression for large data values. This change provides greater flexibility for users to manage storage efficiency and performance characteristics based on their specific deployment needs, ensuring that compression can be toggled without altering the compression threshold.

Highlights

  • Configurable Zstd Compression: Introduced a new configuration option, enableZstdCompression, to explicitly control whether Zstd compression is applied to large values in the event store.
  • Conditional Compression Logic: Modified the event store's write logic to only apply Zstd compression if the new enableZstdCompression flag is true and the value size exceeds the defined compressionThreshold.
  • Test Coverage for Disabling Compression: Added a dedicated test helper and a new test case to verify that Zstd compression can be effectively disabled via the new configuration, ensuring expected behavior.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • logservice/eventstore/event_store.go
    • Added a new boolean field enableZstdCompression to the eventStore struct.
    • Updated the New function to initialize enableZstdCompression from the global server configuration.
    • Modified the writeEvents method to apply Zstd compression only if enableZstdCompression is true and the value exceeds the compressionThreshold.
  • logservice/eventstore/event_store_test.go
    • Introduced a new test helper function setZstdCompressionForTest to temporarily modify the EnableZstdCompression configuration for testing.
    • Added TestWriteToEventStoreZstdCompressionDisabled to verify that compression is not applied when the enableZstdCompression flag is set to false.
  • pkg/config/debug.go
    • Added EnableZstdCompression field to the EventStoreConfig struct with TOML and JSON tags.
    • Set the default value for EnableZstdCompression to true in NewDefaultEventStoreConfig.
  • server.toml
    • Added a new configuration entry enable-zstd-compression under [debug.event-store] with a default value of true and a comment explaining its purpose.
Activity
  • No specific activity (comments, reviews, or progress updates) was found in the provided context.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 24, 2026

📝 Walkthrough

Walkthrough

The changes introduce a feature flag to conditionally enable zstd compression in the event store. A new EnableZstdCompression configuration field is added to EventStoreConfig (default true), wired into the eventStore struct, and used to gate compression logic during event writes.

Changes

Cohort / File(s) Summary
Configuration
pkg/config/debug.go
Added EnableZstdCompression field to EventStoreConfig with default value true and appropriate TOML/JSON tags.
Event Store Implementation
logservice/eventstore/event_store.go
Introduced enableZstdCompression field to eventStore struct, wired from global server config during initialization, and updated writeEvents to check this flag in addition to size threshold before applying zstd compression.
Event Store Tests
logservice/eventstore/event_store_test.go
Added test helper setZstdCompressionForTest to toggle compression flag for test execution, and new test TestWriteToEventStoreZstdCompressionDisabled verifying that entries are not compressed when the flag is disabled.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related issues

Poem

🐰 A flag hops in with gentle care,
Compression waits at config's fare,
Toggle on or toggle off,
Event store won't cough or scoff!
Zstd bows to the master's choice. ✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description is largely incomplete with most critical sections unfilled: 'What is changed and how it works?' is empty, all test checkboxes marked without clarification, and release note left in template form. Complete the 'What is changed and how it works?' section explaining the zstd compression config feature, clarify which tests were added, answer the compatibility/documentation questions, and provide a proper release note.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and directly describes the main change: adding configuration for zstd compression to the eventstore, which is the core purpose of the changeset.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch ldz/add-compression-config

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a configuration flag enable-zstd-compression to control zstd compression in the event store. The changes include adding the flag to the configuration files, propagating it to the eventStore struct, and using it to conditionally apply compression. A new test is also added to verify the behavior when compression is disabled. The implementation is straightforward and correct. I have one suggestion to improve the robustness of the new test case.

lidezhu and others added 3 commits February 24, 2026 22:06
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/config/debug.go`:
- Around line 85-100: The EventStoreConfig.EnableZstdCompression field must be
changed to a pointer (*bool) so you can distinguish omitted vs explicit false
during json/toml unmarshal; update the struct type in EventStoreConfig and set a
pointer default in NewDefaultEventStoreConfig (allocate a bool true and assign
its address), then in DebugConfig.ValidateAndAdjust() check if
c.EventStore.EnableZstdCompression == nil and, if so, allocate and assign the
default true pointer so partial configs receive the intended default.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 75e291b and b25f486.

📒 Files selected for processing (4)
  • logservice/eventstore/event_store.go
  • logservice/eventstore/event_store_test.go
  • pkg/config/debug.go
  • server.toml

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@logservice/eventstore/event_store_test.go`:
- Around line 904-906: Line with "value :=" is not indented to match the
surrounding tab-indented block (between "key := []byte(\"large-key\")" and
"entry := common.RawKVEntry"), which breaks gofmt/gci; fix it by adding the same
leading tab indentation as the other statements so "value := bytes.Repeat(...)"
aligns with "key :=" and "entry :=", keeping the rest of the test unchanged.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b25f486 and aa75bc5.

📒 Files selected for processing (1)
  • logservice/eventstore/event_store_test.go

@lidezhu
Copy link
Collaborator Author

lidezhu commented Feb 24, 2026

/test all

@lidezhu
Copy link
Collaborator Author

lidezhu commented Feb 24, 2026

/run-check-issue-tirage-complete

@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 24, 2026

@lidezhu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cdc-pulsar-integration-heavy 1399611 link false /test pull-cdc-pulsar-integration-heavy

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@lidezhu
Copy link
Collaborator Author

lidezhu commented Feb 25, 2026

/run-check-issue-tirage-complete

@lidezhu lidezhu changed the title add config for zstd compression eventstore: add config for zstd compression Feb 25, 2026
@lidezhu
Copy link
Collaborator Author

lidezhu commented Feb 25, 2026

/retest

@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 25, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: asddongmen
Once this PR has been reviewed and has the lgtm label, please assign flowbehappy for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-1-more-lgtm Indicates a PR needs 1 more LGTM. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add a config to support disable zstd compression inside event store

2 participants