Skip to content

Added shallow search for data.table in tables()#7580

Merged
MichaelChirico merged 38 commits intomasterfrom
feat/adding_list_search_to_tables
Mar 7, 2026
Merged

Added shallow search for data.table in tables()#7580
MichaelChirico merged 38 commits intomasterfrom
feat/adding_list_search_to_tables

Conversation

@manmita
Copy link
Contributor

@manmita manmita commented Jan 9, 2026

Closes #2606

added arg depth = 1L to tables() one for shallow search
if depth is 0 then its the data.table
if depth is 1, we loop through list-like objects using is.list and which are not data.table
if depth > 1, we throw error

added name for the nested list found parent[[1]] or parent$child
pre-allocating info to avoid reallocation cost

@manmita
Copy link
Contributor Author

manmita commented Jan 9, 2026

Hello,

I created a new PR in replacement of #7568

Reasons: There was some git issue there and the merge became too complex and I changed the algo because I didnt know previously that rbind or cbind would cost for re-allocation

The current PR considers that part and avoids appends

Previous PR : creating seperate data.table called info and rbind at the end
This PR: pre-allocates for a total-sized data.table and fills the info

@manmita
Copy link
Contributor Author

manmita commented Jan 9, 2026

In reply to previous comment of @jangorecki

An example of when this new feature could be useful?

To support lists which occur due to split.data.table or fread like the following

list(data.table(a = 1, b = 4:6)),
      data.table(a = 2, b = 7:10))

The original code supported data.table() top level and this code adds support for list(data.table) if the arg shallow_search = TRUE

@manmita
Copy link
Contributor Author

manmita commented Jan 9, 2026

Example of the original code and the new feature is as follows

> A = list(data.table(a = 1, b = 4:6),
      data.table(a = 2, b = 7:10))
> B = list(data.table(a = 1, b = 4:6), 1:5)
> C = data.table(a = 1, b = 4:6)
> tables()
   NAME NROW NCOL MB COLS    KEY
1:    C    3    2  0  a,b [NULL]
Total: 0MB using type_size
> tables(shallow_search = TRUE)
     NAME NROW NCOL MB COLS    KEY
1: A[[1]]    3    2  0  a,b [NULL]
2: A[[2]]    4    2  0  a,b [NULL]
3: B[[1]]    3    2  0  a,b [NULL]
4:      C    3    2  0  a,b [NULL]
Total: 0MB using type_size
> D = list(d = data.table(a = 1, b = 4:6), x = 1:5)
> tables(shallow_search = TRUE)
     NAME NROW NCOL MB COLS    KEY
1: A[[1]]    3    2  0  a,b [NULL]
2: A[[2]]    4    2  0  a,b [NULL]
3: B[[1]]    3    2  0  a,b [NULL]
4:      C    3    2  0  a,b [NULL]
5:    D$d    3    2  0  a,b [NULL]
Total: 0MB using type_size

tables() work same as before and tables(shallow_search = TRUE) searches 1 level

@codecov
Copy link

codecov bot commented Jan 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.03%. Comparing base (6c6615c) to head (5a5a364).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7580   +/-   ##
=======================================
  Coverage   99.02%   99.03%           
=======================================
  Files          87       87           
  Lines       16896    16930   +34     
=======================================
+ Hits        16732    16767   +35     
+ Misses        164      163    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link

github-actions bot commented Jan 9, 2026

No obvious timing issues in HEAD=feat/adding_list_search_to_tables
Comparison Plot

Generated via commit 5a5a364

Download link for the artifact containing the test results: ↓ atime-results.zip

Task Duration
R setup and installing dependencies 3 minutes and 13 seconds
Installing different package versions 23 seconds
Running and plotting the test cases 4 minutes and 13 seconds

@manmita manmita requested a review from MichaelChirico March 6, 2026 21:50
@manmita manmita requested a review from MichaelChirico March 7, 2026 22:23
@MichaelChirico
Copy link
Member

LGTM thanks!

@MichaelChirico MichaelChirico merged commit 8d3343c into master Mar 7, 2026
12 checks passed
@MichaelChirico MichaelChirico deleted the feat/adding_list_search_to_tables branch March 7, 2026 22:34
@manmita
Copy link
Contributor Author

manmita commented Mar 7, 2026

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tables could look for en-list-ed data.tables as well

3 participants