Skip to content

perf: cache BaseVec len in Cell<u64> to avoid stable memory reads#415

Merged
sasa-tomic merged 4 commits intomainfrom
perf/basevec-len-cache
Mar 20, 2026
Merged

perf: cache BaseVec len in Cell<u64> to avoid stable memory reads#415
sasa-tomic merged 4 commits intomainfrom
perf/basevec-len-cache

Conversation

@sasa-tomic
Copy link
Contributor

@sasa-tomic sasa-tomic commented Mar 18, 2026 •
edited
Loading

Summary

  • Cache BaseVec::len() in a Cell<u64> instead of reading from stable memory on every call
  • MinHeap calls len() O(log n) times per push/pop (via bounds checks in sift_up/sift_down), making this a significant overhead — eliminates ~13-26 stable memory reads per MinHeap operation on heaps of 10K elements
  • Cell<u64> is necessary because all BaseVec data methods use &self (interior mutability via Memory trait)

@sasa-tomic
perf: cache BaseVec len in Cell<u64> to avoid stable memory reads
6007855 
BaseVec::len() previously read from stable memory on every call.
MinHeap calls len() O(log n) times per push/pop via sift_up/sift_down
bounds checks, making this a significant overhead. Cache the length
in a Cell<u64> and keep it in sync with the persisted value.
@sasa-tomic sasa-tomic requested a review from a team as a code owner March 18, 2026 17:39
@sasa-tomic sasa-tomic marked this pull request as draft March 18, 2026 17:53
@sasa-tomic sasa-tomic marked this pull request as ready for review March 19, 2026 14:51
@github-actions
Copy link

github-actions bot commented Mar 19, 2026 •
edited
Loading

canbench 🏋 (dir: ./benchmarks/vec) b503b4c 2026-03-20 10:35:54 UTC

✅ ./benchmarks/vec/canbench_results.yml is up to date
📦 canbench_results_vec.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   Improvements detected 🟢
    counts:   [total 16 | regressed 0 | improved 16 | new 0 | unchanged 0]
    change:   [max -350.00K | p75 -350.00K | median -489.07K | p25 -544.93K | min -1.32M]
    change %: [max -2.81% | p75 -7.72% | median -9.57% | p25 -10.71% | min -26.30%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                        | calls |    ins |  ins Δ% | HI |  HI Δ% | SMI |  SMI Δ% |
|--------|-----------------------------|-------|--------|---------|----|--------|-----|---------|
|   -    | vec_get_blob_128            |       | 18.71M |  -2.81% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_get_blob_64             |       | 10.75M |  -4.95% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_get_blob_32             |       |  6.65M |  -6.59% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_insert_u64              |       |  5.00M |  -7.06% |  0 |  0.00% |   1 |   0.00% |
|   -    | vec_get_blob_16             |       |  5.90M |  -7.94% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_insert_blob_128         |       |  3.82M |  -8.39% |  0 |  0.00% |  19 |   0.00% |
|   -    | vec_get_blob_8              |       |  5.21M |  -8.91% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_insert_blob_64          |       |  3.35M |  -9.47% |  0 |  0.00% |   9 |   0.00% |
|   -    | vec_get_blob_64_mem_manager |       | 12.33M |  -9.67% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_insert_blob_32          |       |  3.11M | -10.13% |  0 |  0.00% |   5 |   0.00% |
|   -    | vec_insert_blob_16          |       |  2.99M | -10.49% |  0 |  0.00% |   2 |   0.00% |
|   -    | vec_insert_blob_8           |       |  2.93M | -10.68% |  0 |  0.00% |   1 |   0.00% |
|   -    | vec_insert_blob_4           |       |  2.90M | -10.78% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_get_blob_4              |       |  4.28M | -11.19% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_get_blob_4_mem_manager  |       |  5.89M | -18.08% |  0 |  0.00% |   0 |   0.00% |
|   -    | vec_get_u64                 |       |  3.53M | -26.30% |  0 |  0.00% |   0 |   0.00% |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented Mar 19, 2026 •
edited
Loading

canbench 🏋 (dir: ./benchmarks/memory_manager) b503b4c 2026-03-20 10:36:01 UTC

✅ ./benchmarks/memory_manager/canbench_results.yml is up to date
📦 canbench_results_memory-manager.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 3 | regressed 0 | improved 0 | new 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@sasa-tomic
chore(bench): refresh vec canbench baseline after len cache change
2e8187a 
Update the persisted vec benchmark snapshot so CI accepts the expected instruction improvements from BaseVec length caching.
@github-actions
Copy link

github-actions bot commented Mar 20, 2026 •
edited
Loading

canbench 🏋 (dir: ./benchmarks/io_chunks) b503b4c 2026-03-20 10:36:45 UTC

✅ ./benchmarks/io_chunks/canbench_results.yml is up to date
📦 canbench_results_io_chunks.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   Improvements detected 🟢
    counts:   [total 18 | regressed 0 | improved 2 | new 0 | unchanged 16]
    change:   [max +14.00M | p75 0 | median 0 | p25 -125 | min -314.90M]
    change %: [max +0.84% | p75 0.00% | median 0.00% | p25 -0.00% | min -31.15%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 18 | regressed 0 | improved 0 | new 0 | unchanged 18]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                | calls |     ins |  ins Δ% | HI |  HI Δ% |   SMI |  SMI Δ% |
|--------|---------------------|-------|---------|---------|----|--------|-------|---------|
|   -    | write_chunks_vec_1m |       |   1.77B |  -6.59% |  0 |  0.00% | 1.54K |   0.00% |
|   -    | read_chunks_vec_1m  |       | 696.00M | -31.15% |  0 |  0.00% |     0 |   0.00% |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented Mar 20, 2026 •
edited
Loading

canbench 🏋 (dir: ./benchmarks/nns) b503b4c 2026-03-20 10:36:05 UTC

✅ ./benchmarks/nns/canbench_results.yml is up to date
📦 canbench_results_nns.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median -98 | p25 -439.05K | min -6.98M]
    change %: [max 0.00% | p75 0.00% | median -0.04% | p25 -0.16% | min -0.29%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented Mar 20, 2026 •
edited
Loading

canbench 🏋 (dir: ./benchmarks/btreeset) b503b4c 2026-03-20 10:36:10 UTC

✅ ./benchmarks/btreeset/canbench_results.yml is up to date
📦 canbench_results_btreeset.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 100 | regressed 0 | improved 0 | new 0 | unchanged 100]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented Mar 20, 2026

canbench 🏋 (dir: ./benchmarks/btreemap) b503b4c 2026-03-20 10:37:42 UTC

✅ ./benchmarks/btreemap/canbench_results.yml is up to date
📦 canbench_results_btreemap.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 303 | regressed 0 | improved 0 | new 0 | unchanged 303]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 303 | regressed 0 | improved 0 | new 0 | unchanged 303]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 303 | regressed 0 | improved 0 | new 0 | unchanged 303]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@sasa-tomic sasa-tomic merged commit 29fad0d into main Mar 20, 2026
16 checks passed
@sasa-tomic sasa-tomic deleted the perf/basevec-len-cache branch March 20, 2026 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adambratschikaye adambratschikaye adambratschikaye approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sasa-tomic @adambratschikaye