Skip to content

fix: Respect display.progress_bar=None in background threads#16715

Open
shuoweil wants to merge 52 commits into
mainfrom
shuowei-anywidget-extraneous-output
Open

fix: Respect display.progress_bar=None in background threads#16715
shuoweil wants to merge 52 commits into
mainfrom
shuowei-anywidget-extraneous-output

Conversation

@shuoweil
Copy link
Copy Markdown
Contributor

@shuoweil shuoweil commented Apr 17, 2026

Fixes extraneous progress bar output during asynchronous query executions (such as Colab SQL cells and visualization plugins) when display.progress_bar = None is set via option_context.
Previously, background query callbacks evaluated thread-local display options within the background worker thread, falling back to default settings ("auto").

Changes

  • EventEnvelope: Introduced a dataclass wrapping query execution events with the initiating thread's active progress_bar configuration.
  • Query Initiation: Updated start_query_with_job and start_query_job_optional to capture the calling thread's display options and emit events wrapped in an EventEnvelope.
  • Event Listeners: Updated progress_callback and metrics.on_event to unwrap EventEnvelope and respect the preserved option context, successfully suppressing extraneous output in background execution threads.

Verified at here:
before: screen/48zpTTWWjZ5wF9Y
after: screen/7RS7exRrxgPDEqb

Fixes #<461829560> 🦕

@shuoweil shuoweil self-assigned this Apr 17, 2026
@shuoweil shuoweil requested review from a team as code owners April 17, 2026 23:31
@shuoweil shuoweil requested review from GarrettWu and sycai and removed request for a team and sycai April 17, 2026 23:31
@shuoweil shuoweil force-pushed the shuowei-anywidget-extraneous-output branch from de201ee to d887714 Compare April 17, 2026 23:33
@shuoweil shuoweil marked this pull request as draft April 17, 2026 23:35
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to pass the progress_bar configuration through BigQuery events, allowing for more granular control over progress bar display. It updates several event classes to include a progress_bar field and modifies the callback logic to prioritize this field over global settings. Feedback suggests defining the 'fallback_to_global' sentinel as a constant to improve maintainability and refactoring the event mapping logic in create_bq_event_callback to reduce code duplication.

Comment thread packages/bigframes/bigframes/core/events.py Outdated
Comment thread packages/bigframes/bigframes/session/_io/bigquery/__init__.py Outdated
@shuoweil shuoweil force-pushed the shuowei-anywidget-extraneous-output branch from dce2e04 to 42e4a50 Compare April 17, 2026 23:49
@shuoweil shuoweil marked this pull request as ready for review April 20, 2026 22:54
@shuoweil shuoweil requested review from TrevorBergeron, chelsea-lin and sycai and removed request for sycai April 23, 2026 17:56
@shuoweil shuoweil closed this Apr 27, 2026
@tswast tswast reopened this Apr 28, 2026
@tswast
Copy link
Copy Markdown
Contributor

tswast commented Apr 28, 2026

This PR doesn't touch the thread localility of the configuration object. Please continue with this fix.

@shuoweil shuoweil requested a review from tswast May 13, 2026 03:43
except ImportError:
return

if isinstance(envelope, bigframes.core.events.EventEnvelope):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it not be EventEnvelope type?

Copy link
Copy Markdown
Contributor Author

@shuoweil shuoweil May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Having a consistent API contract for subscribers is definitely cleaner and removes the boilerplate isinstance checks.

I've updated Publisher.publish to automatically wrap any raw Event into an EventEnvelope. Now all subscribers consistently receive an EventEnvelope by default.

return

progress_bar = bigframes._config.options.display.progress_bar
if isinstance(envelope, bigframes.core.events.EventEnvelope):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@shuoweil shuoweil requested a review from GarrettWu May 13, 2026 21:44
@tswast
Copy link
Copy Markdown
Contributor

tswast commented May 14, 2026

FYI: This may cause some conflicts with #17093 May want to investigate if the ExecutionSpec object could be used similarly.

@shuoweil shuoweil force-pushed the shuowei-anywidget-extraneous-output branch from 91706d5 to 6d8b079 Compare May 14, 2026 20:52
@shuoweil
Copy link
Copy Markdown
Contributor Author

shuoweil commented May 14, 2026

FYI: This may cause some conflicts with #17093 May want to investigate if the ExecutionSpec object could be used similarly.

PR #17093 is merged and all conflicts are fully resolved.

We investigated using ExecutionSpec but determined EventEnvelope is necessary for event publishing. Several lifecycle events (like SessionClosed) occur outside of query execution plans, and threading ExecutionSpec through every event emitter across the codebase would tightly couple publishing to executor internals.

Our implementation now seamlessly combines both patterns: EventEnvelope dynamically enforces thread-local display options for multi-threaded workloads, while falling back to #17093's session-level closure for single-threaded flows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants