Skip to content

Add task log capture and export #821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tobiasraabe wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Add task log capture and export #821

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
76215ea
Add task log capture and export (closes #169)
tobiasraabe Mar 22, 2026
ff1e21b
Fix Python 3.10 log capture import
tobiasraabe Mar 22, 2026
977cb25
Add live terminal log streaming
tobiasraabe Mar 22, 2026
18236df
Merge remote-tracking branch 'origin/main' into codex/issue-169-log-c…
tobiasraabe Mar 22, 2026
1c9cea3
Fix logging docs and remove demo script
tobiasraabe Mar 22, 2026
8401ba8
Adjust logging animation spacing
tobiasraabe Mar 22, 2026
b47e06f
fix.
tobiasraabe Mar 22, 2026
9d15e65
Merge remote-tracking branch 'origin/main' into codex/issue-169-log-c…
tobiasraabe Mar 22, 2026
891aab1
Fix rendered CLI option separators
tobiasraabe Mar 22, 2026
857b794
Document logging handler tradeoffs
tobiasraabe Mar 22, 2026
72a646b
Add logging guide and docs navigation
tobiasraabe Mar 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ tests/test_jupyter/*.txt
.ruff_cache
.venv
docs/jupyter_execute
.DS_Store
30 changes: 20 additions & 10 deletions docs/AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,7 @@
# Documentation

- Link to existing docs/API refs instead of re-explaining concepts - reduces duplication
and keeps info in sync - Prevents documentation drift and outdated explanations by
maintaining a single source of truth for each concept
- Link to canonical docs rather than duplicating content - prevents drift and
maintenance burden - Consolidating documentation into existing files with
cross-references keeps information consistent and reduces the effort needed to
update multiple locations when changes occur.
## General

- Document only public APIs and user-facing behavior - exclude internals, framework
abstractions, and implementation plumbing - Users need actionable documentation on
what they can use, not confusing details about internal mechanics they can't control
Expand All @@ -17,9 +12,6 @@
comprehensive coverage vs. fragmented mentions - Prevents users from missing
features when they approach from different contexts (CLI vs. API) and allows
features to be documented holistically rather than buried in subsections.
- Avoid `# ruff: noqa` or `# type: ignore` in doc examples - ensures examples stay
correct and runnable - Skip directives hide bugs and type errors in documentation
code that users will copy, leading to broken examples in the wild
- Explicitly mark parameters/features as 'optional' in docs, even when types show it -
reduces cognitive load for readers - Users shouldn't need to parse type signatures
to understand optionality; explicit labels make documentation scannable and
Expand All @@ -31,3 +23,21 @@
- Strip boilerplate from docs examples - show only the feature being demonstrated -
Reduces cognitive load and helps readers focus on the specific API or pattern being
taught without distraction from scaffolding code.

## Linking

- Link to existing docs/API refs instead of re-explaining concepts - reduces duplication
and keeps info in sync - Prevents documentation drift and outdated explanations by
maintaining a single source of truth for each concept
- Link to canonical docs rather than duplicating content - prevents drift and
maintenance burden - Consolidating documentation into existing files with
cross-references keeps information consistent and reduces the effort needed to
update multiple locations when changes occur.

## Code Examples

- Avoid `# ruff: noqa` or `# type: ignore` in doc examples - ensures examples stay
correct and runnable - Skip directives hide bugs and type errors in documentation
code that users will copy, leading to broken examples in the wild
- Code file examples should have a title that shows the file name.
- Important lines should be highlighted or annotated with a comment.
6 changes: 3 additions & 3 deletions docs/source/_static/md/commands/build-arguments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
| Argument | Description |
| ------------ | ---------------------------------------------------------- |
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
| Argument | Description |
| ----------------------- | ---------------------------------------------------------- |
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |
73 changes: 43 additions & 30 deletions docs/source/_static/md/commands/build-options.md
View file
Open in desktop

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions docs/source/_static/md/commands/clean-arguments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
| Argument | Description |
| ------------ | ---------------------------------------------------------- |
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
| Argument | Description |
| ----------------------- | ---------------------------------------------------------- |
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |
30 changes: 15 additions & 15 deletions docs/source/_static/md/commands/clean-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
| Option | Default | Description |
| -------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `-c, --config FILE` | - | Path to configuration file. |
| `-d, --directories` | `false` | Remove whole directories. |
| `--database-url TEXT` | - | Url to the database. |
| `-e, --exclude PATTERN` | - | A filename pattern to exclude files from the cleaning process. |
| `--editor-url-scheme TEXT` | `file` | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
| `--hook-module TEXT` | - | Path to a Python module that contains hook implementations. |
| `--ignore TEXT` | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
| `-k EXPRESSION` | - | Select tasks via expressions on task ids. |
| `-m MARKER_EXPRESSION` | - | Select tasks via marker expressions. |
| \`--mode \[dry-run | force | interactive\]\` |
| `-q, --quiet` | `false` | Do not print the names of the removed paths. |
| `--strict-markers` | `false` | Raise errors for unknown markers. |
| `-h, --help` | - | Show this message and exit. |
| Option | Default | Description |
| ------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <code>-c, --config FILE</code> | - | Path to configuration file. |
| <code>-d, --directories</code> | <code>false</code> | Remove whole directories. |
| <code>--database-url TEXT</code> | - | Url to the database. |
| <code>-e, --exclude PATTERN</code> | - | A filename pattern to exclude files from the cleaning process. |
| <code>--editor-url-scheme TEXT</code> | <code>file</code> | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
| <code>--hook-module TEXT</code> | - | Path to a Python module that contains hook implementations. |
| <code>--ignore TEXT</code> | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
| <code>-k EXPRESSION</code> | - | Select tasks via expressions on task ids. |
| <code>-m MARKER_EXPRESSION</code> | - | Select tasks via marker expressions. |
| <code>--mode [dry-run\|force\|interactive]</code> | <code>dry-run</code> | Choose 'dry-run' to print the paths of files/directories which would be removed, 'interactive' for a confirmation prompt for every path, and 'force' to remove all unknown paths at once. |
| <code>-q, --quiet</code> | <code>false</code> | Do not print the names of the removed paths. |
| <code>--strict-markers</code> | <code>false</code> | Raise errors for unknown markers. |
| `-h, --help` | - | Show this message and exit. |
6 changes: 3 additions & 3 deletions docs/source/_static/md/commands/collect-arguments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
| Argument | Description |
| ------------ | ---------------------------------------------------------- |
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
| Argument | Description |
| ----------------------- | ---------------------------------------------------------- |
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |
24 changes: 12 additions & 12 deletions docs/source/_static/md/commands/collect-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
| Option | Default | Description |
| -------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `-c, --config FILE` | - | Path to configuration file. |
| `--database-url TEXT` | - | Url to the database. |
| `--editor-url-scheme TEXT` | `file` | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
| `--hook-module TEXT` | - | Path to a Python module that contains hook implementations. |
| `--ignore TEXT` | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
| `-k EXPRESSION` | - | Select tasks via expressions on task ids. |
| `-m MARKER_EXPRESSION` | - | Select tasks via marker expressions. |
| `--nodes` | `false` | Show a task's dependencies and products. |
| `--strict-markers` | `false` | Raise errors for unknown markers. |
| `-h, --help` | - | Show this message and exit. |
| Option | Default | Description |
| ------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| <code>-c, --config FILE</code> | - | Path to configuration file. |
| <code>--database-url TEXT</code> | - | Url to the database. |
| <code>--editor-url-scheme TEXT</code> | <code>file</code> | Use file, vscode, pycharm or a custom url scheme to add URLs to task ids to quickly jump to the task definition. Use no_link to disable URLs. |
| <code>--hook-module TEXT</code> | - | Path to a Python module that contains hook implementations. |
| <code>--ignore TEXT</code> | - | A pattern to ignore files or directories. Refer to 'pathlib.Path.match' for more info. |
| <code>-k EXPRESSION</code> | - | Select tasks via expressions on task ids. |
| <code>-m MARKER_EXPRESSION</code> | - | Select tasks via marker expressions. |
| <code>--nodes</code> | <code>false</code> | Show a task's dependencies and products. |
| <code>--strict-markers</code> | <code>false</code> | Raise errors for unknown markers. |
| `-h, --help` | - | Show this message and exit. |
16 changes: 8 additions & 8 deletions docs/source/_static/md/commands/command-list.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
| Command | Description |
| ----------------------------------------- | ------------------------------------------------------------- |
| [`build`](../../../commands/build.md) | Collect tasks, execute them and report the results. |
| [`clean`](../../../commands/clean.md) | Clean the provided paths by removing files unknown to pytask. |
| [`collect`](../../../commands/collect.md) | Collect tasks and report information about them. |
| [`dag`](../../../commands/dag.md) | Create a visualization of the directed acyclic graph. |
| [`markers`](../../../commands/markers.md) | Show all registered markers. |
| [`profile`](../../../commands/profile.md) | Show information about resource consumption. |
| Command | Description |
| ----------------------- | ------------------------------------------------------------- |
| [`build`](build.md) | Collect tasks, execute them and report the results. |
| [`clean`](clean.md) | Clean the provided paths by removing files unknown to pytask. |
| [`collect`](collect.md) | Collect tasks and report information about them. |
| [`dag`](dag.md) | Create a visualization of the directed acyclic graph. |
| [`markers`](markers.md) | Show all registered markers. |
| [`profile`](profile.md) | Show information about resource consumption. |
6 changes: 3 additions & 3 deletions docs/source/_static/md/commands/dag-arguments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
| Argument | Description |
| ------------ | ---------------------------------------------------------- |
| `[PATHS]...` | Paths where pytask looks for task files and configuration. |
| Argument | Description |
| ----------------------- | ---------------------------------------------------------- |
| <code>[PATHS]...</code> | Paths where pytask looks for task files and configuration. |
Loading
Loading