fix: Respect display.progress_bar=None in background threads#16715
fix: Respect display.progress_bar=None in background threads#16715shuoweil wants to merge 52 commits into
Conversation
de201ee to
d887714
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to pass the progress_bar configuration through BigQuery events, allowing for more granular control over progress bar display. It updates several event classes to include a progress_bar field and modifies the callback logic to prioritize this field over global settings. Feedback suggests defining the 'fallback_to_global' sentinel as a constant to improve maintainability and refactoring the event mapping logic in create_bq_event_callback to reduce code duplication.
dce2e04 to
42e4a50
Compare
|
This PR doesn't touch the thread localility of the configuration object. Please continue with this fix. |
| except ImportError: | ||
| return | ||
|
|
||
| if isinstance(envelope, bigframes.core.events.EventEnvelope): |
There was a problem hiding this comment.
Can it not be EventEnvelope type?
There was a problem hiding this comment.
Good point! Having a consistent API contract for subscribers is definitely cleaner and removes the boilerplate isinstance checks.
I've updated Publisher.publish to automatically wrap any raw Event into an EventEnvelope. Now all subscribers consistently receive an EventEnvelope by default.
| return | ||
|
|
||
| progress_bar = bigframes._config.options.display.progress_bar | ||
| if isinstance(envelope, bigframes.core.events.EventEnvelope): |
# Conflicts: # packages/bigframes/bigframes/session/_io/bigquery/__init__.py
|
FYI: This may cause some conflicts with #17093 May want to investigate if the ExecutionSpec object could be used similarly. |
91706d5 to
6d8b079
Compare
PR #17093 is merged and all conflicts are fully resolved. We investigated using ExecutionSpec but determined EventEnvelope is necessary for event publishing. Several lifecycle events (like SessionClosed) occur outside of query execution plans, and threading ExecutionSpec through every event emitter across the codebase would tightly couple publishing to executor internals. Our implementation now seamlessly combines both patterns: EventEnvelope dynamically enforces thread-local display options for multi-threaded workloads, while falling back to #17093's session-level closure for single-threaded flows. |
# Conflicts: # packages/bigframes/bigframes/formatting_helpers.py
Fixes extraneous progress bar output during asynchronous query executions (such as Colab SQL cells and visualization plugins) when
display.progress_bar = Noneis set viaoption_context.Previously, background query callbacks evaluated thread-local display options within the background worker thread, falling back to default settings (
"auto").Changes
EventEnvelope: Introduced a dataclass wrapping query execution events with the initiating thread's activeprogress_barconfiguration.start_query_with_jobandstart_query_job_optionalto capture the calling thread's display options and emit events wrapped in anEventEnvelope.progress_callbackandmetrics.on_eventto unwrapEventEnvelopeand respect the preserved option context, successfully suppressing extraneous output in background execution threads.Verified at here:
before: screen/48zpTTWWjZ5wF9Y
after: screen/7RS7exRrxgPDEqb
Fixes #<461829560> 🦕