MDEV-38814 Reduce pessimistic update fallbacks#4909
MDEV-38814 Reduce pessimistic update fallbacks#4909iMineLink wants to merge 1 commit intoMariaDB:11.8from
Conversation
|
|
|
Another discussion scenario: consider for any reason, a page is left in an "under flew" state. |
There was a problem hiding this comment.
Pull request overview
This PR (draft) aims to reduce unnecessary pessimistic-update fallbacks and related index lock upgrades in InnoDB B-tree update code paths (MDEV-38814), by refining when optimistic updates reject/trigger merge/split behavior and by adding debug instrumentation and tests to track the effects.
Changes:
- Add new debug-only counters for index lock upgrades and optimistic-update overflow/underflow outcomes, exported via
SHOW GLOBAL STATUSin debug builds. - Adjust
btr_cur_optimistic_update()to only returnDB_UNDERFLOWfor record-shrinking updates (avoiding underflow-triggered merge attempts for record growth). - Enforce
BTR_CUR_PAGE_REORGANIZE_LIMITinbtr_cur_pessimistic_update()by skippingbtr_cur_insert_if_possible()in some overflow cases and forcing the split path; add a newindex_lock_upgradeMTR test.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| storage/innobase/include/btr0cur.h | Declares new debug-only atomic counters for instrumentation. |
| storage/innobase/handler/ha_innodb.cc | Exposes new debug counters via InnoDB status variables. |
| storage/innobase/btr/btr0cur.cc | Implements counters, changes optimistic underflow condition, and adds pessimistic overflow handling logic. |
| mysql-test/suite/innodb/t/innodb_status_variables.test | Excludes new debug-only status variables from the generic status variable listing test. |
| mysql-test/suite/innodb/t/index_lock_upgrade.test | Adds a new debug-only test to measure lock upgrades and page maintenance metrics during insert/update. |
| mysql-test/suite/innodb/r/innodb_status_variables.result | Updates expected output to reflect exclusions. |
| mysql-test/suite/innodb/r/index_lock_upgrade.result | Adds expected output for the new test. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
A high rate of index lock SX-to-X upgrades was traced to two patterns in
btr_cur_*_update() that turn benign UPDATEs into pessimistic fallbacks:
1. btr_cur_optimistic_update() returns DB_UNDERFLOW whenever the page
would drop below BTR_CUR_PAGE_COMPRESS_LIMIT after the
delete-then-insert, even when the record itself is growing. On a
freshly split page this re-triggers a merge the moment the next
update lands, defeating the split that just happened.
2. btr_cur_pessimistic_update() handles the DB_OVERFLOW fallback by
calling btr_cur_insert_if_possible() unconditionally. When the
uncompressed page cannot satisfy BTR_CUR_PAGE_REORGANIZE_LIMIT after
a reorganize, that retry fails too and the same page churns through
pessimistic_update without ever splitting.
Introduce a system variable innodb_reduce_pessimistic_update_fallbacks
(BOOL, default OFF) that gates both fixes:
* btr_cur_optimistic_update() returns DB_UNDERFLOW only when the new
record is strictly smaller than the old one, so growing updates do
not trip the merge path.
* btr_cur_pessimistic_update() skips btr_cur_insert_if_possible() and
falls through to a page split when the optimistic error was
DB_OVERFLOW, the page is uncompressed with at least two records, the
record is growing, and a reorganize would not free
BTR_CUR_PAGE_REORGANIZE_LIMIT bytes.
Same-size updates take the optimistic path under both fixes (no merge,
no split), matching the intent that only size-changing updates
contribute to space pressure. The trade-off is that an opportunistic
merge that previously triggered on any update to a sparse page now
requires an actual shrink to fire.
To make the impact measurable, expose seven debug-only atomic counters
via SHOW GLOBAL STATUS:
Innodb_btr_cur_n_index_lock_upgrades
Innodb_btr_cur_pessimistic_insert_calls
Innodb_btr_cur_pessimistic_update_calls
Innodb_btr_cur_pessimistic_delete_calls
Innodb_btr_cur_pessimistic_update_optim_err_underflows
Innodb_btr_cur_pessimistic_update_optim_err_overflows
Innodb_mtr_n_index_x_lock_calls
A new test, innodb.index_lock_upgrade, drives a 1000-row INSERT /
UPDATE / DELETE workload on a 4K-page table across three shapes (PK
only; PK + secondary index on a DATETIME column; same with
ROW_FORMAT=COMPRESSED, KEY_BLOCK_SIZE=2). It snapshots all seven
counters plus the index_page_* INNODB_METRICS between phases. The
test runs in two combinations (`off` and `on`) to lock in counter
deltas for both the legacy and the optimized paths; the compressed
table also covers the deliberate scope limitation that the
pessimistic-side branch is gated to uncompressed pages while the
optimistic-side change applies uniformly.
|
I force pushed a new version of the changes, suited for merging into an LTS version, as a single commit. |
Alternative PR to address MDEV-38814.
This attempt, compared to #4887, tries to address the issue at a higher level, changing the optimistic rejection and pessimistic fallback handling in InnoDB btr updates, without changing locking rules.
EDITED as per last force push.
The optimizations are better described in the commit message, and they aim at avoiding that:
btr_cur_optimistic_update(), an UPDATE that is growing a record's size pay the price of a pessimistic fallback for underflow; This is tested to happen on freshly split pages.btr_cur_pessimistic_update(), an UPDATE that is growing a record's size and that is causing the page size to overflow as per reorganization limit, gets reorganized anyway bybtr_cur_insert_if_possible(); This forces a split sooner and avoids subsequent optimistic update failures on the same page.I believe this PR addresses the MDEV's issue in a better way than the previous attempt, trying to optimize index maintenance operations.
The optimizations are tested in
index_lock_upgrade.testby toggling the gating variableINNODB_REDUCE_PESSIMISTIC_UPDATE_FALLBACKSwhich isOFFby default.When merging upwards into preview versions, the default might be changed to
ON.There is evident reduction in index lock upgrades when the optimization is active and the workload includes record-size-growing UPDATEs (such as NULL to not-NULL updates).
Enabling
INNODB_REDUCE_PESSIMISTIC_UPDATE_FALLBACKSdoes reduce though self-healing capabilities of the B-tree, favoring page splits and reducing page-reorganizations and page-merges attempts, which might have been useful for reasons not considered here.So in general, enabling
INNODB_REDUCE_PESSIMISTIC_UPDATE_FALLBACKSunder UPDATE heavy-scenarios (e.g. NULL to not-NULL updates) might favor sparser trees while reducing contention (due to less B-tree maintenance attempts).Test suite has been run successfully with: