Skip to content

WIP: Google Sheets: auto-pad col_types when column count changes#1479

Open
hidekoji wants to merge 4 commits intomasterfrom
fix/issue-33700
Open

WIP: Google Sheets: auto-pad col_types when column count changes#1479
hidekoji wants to merge 4 commits intomasterfrom
fix/issue-33700

Conversation

@hidekoji
Copy link
Copy Markdown
Collaborator

@hidekoji hidekoji commented Feb 2, 2026

Summary

  • Adds automatic padding of col_types with ? (guess/default) when Google Sheet has more columns than specified
  • Preserves existing column type configurations for original columns
  • Works for both Desktop refresh and Server-side scheduled refresh

Problem

When a Google Sheet's column structure changes (columns are added) after initial import, refreshing the data source fails with:

Length of col_types is not compatible with columns found in sheets:
- 14 column types specified.
- But there are 15 columns found in sheets.

Solution

Instead of failing, the system now:

  1. Detects the col_types length mismatch error
  2. Extracts the actual column count from the error message
  3. Pads the col_types string with ? for extra columns
  4. Retries the read operation with the padded col_types

Test plan

  • Test with Google Sheet that has exactly the specified column count
  • Test with Google Sheet that has more columns than specified col_types
  • Test with guessDataType=TRUE and guessDataType=FALSE
  • Test with col_types=NULL (no column types specified)
  • Test server-side scheduled refresh

🤖 Generated with Claude Code

When a Google Sheet has more columns than specified in col_types,
automatically pad with '?' (guess/default) for the extra columns
instead of failing with "Length of col_types is not compatible" error.

This fixes issues where:
- User imports Google Sheet with column types specified
- Additional columns are added to the sheet later
- On refresh (Desktop or Server), the data source now succeeds
  with the new columns using automatic type detection

Fixes exploratory-io/tam#33700

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves Google Sheets refresh robustness by automatically handling col_types/column-count mismatches when a sheet gains new columns.

Changes:

  • Add .pad_col_types_for_column_mismatch() to extend col_types with ? based on the column count parsed from the googlesheets4 error message.
  • Add .read_sheet_with_col_types_padding() wrapper to retry googlesheets4::read_sheet() after padding on the specific mismatch error.
  • Add unit tests for the padding helper.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
R/google_sheets.R Adds internal helpers and routes getGoogleSheet() reads through retry-with-padding logic to avoid failures when column counts increase.
tests/testthat/test_google_sheets.R Adds unit coverage for the col_types padding helper behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread R/google_sheets.R Outdated
Comment thread R/google_sheets.R Outdated
Comment thread tests/testthat/test_google_sheets.R
hidekoji and others added 3 commits February 3, 2026 23:07
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@hidekoji hidekoji marked this pull request as ready for review February 4, 2026 07:49
@hidekoji hidekoji changed the title fix(googlesheets): auto-pad col_types when column count changes WIP: Google Sheets: auto-pad col_types when column count changes Feb 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants