-
Notifications
You must be signed in to change notification settings - Fork 170
DOCS: Add guide on sharing dataset metadata #2510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
VeckoTheGecko
wants to merge
2
commits into
Parcels-code:v4-dev
Choose a base branch
from
VeckoTheGecko:updated-contributing-templates
base: v4-dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| name: 🔼 Parcels feature requests, and other enhancements | ||
| description: Suggest an improvement to the Parcels codebase | ||
| labels: ["needs-triage"] | ||
| body: | ||
| - type: input | ||
| id: version | ||
| attributes: | ||
| label: What version of Parcels are you running? | ||
| description: | | ||
| Please specify the version of Parcels you're currently using (e.g., v3.1.4, main branch, or commit hash). Put "N/A" if not applicable (e.g., for a documentation suggestion). | ||
| placeholder: e.g., v3.1.4 | ||
| validations: | ||
| required: true | ||
| - type: textarea | ||
| id: problem | ||
| attributes: | ||
| label: Is your feature request related to a problem? | ||
| description: | | ||
| Please do a quick search of existing issues to make sure that this feature request has not been asked before. | ||
|
|
||
| Provide a clear and concise description of what the problem is. For example: | ||
| - "I'm trying to simulate particle behavior with [specific physics], but Parcels doesn't support..." | ||
| - "When working with [specific data format/kernel], I find it difficult to..." | ||
|
|
||
| If you would like to see Parcels work with a specific dataset, please also [follow our instructions on how to share dataset metadata](https://docs.oceanparcels.org/en/v4-dev/development/posting-issues.html). | ||
| validations: | ||
| required: true | ||
| - type: textarea | ||
| id: solution | ||
| attributes: | ||
| label: Describe the solution you'd like | ||
| description: | | ||
| A clear and concise description of what you want to happen. Include details about: | ||
| - How you envision using this feature in your Parcels simulations | ||
| - What the API or interface might look like | ||
| - Any specific use cases or scientific applications this would enable | ||
| validations: | ||
| required: true | ||
| - type: textarea | ||
| id: alternatives | ||
| attributes: | ||
| label: Describe alternatives you've considered | ||
| description: | | ||
| A clear and concise description of any alternative solutions or features you've considered. For example: | ||
| - Workarounds you're currently using | ||
| - Other ocean/particle modeling tools that have this feature | ||
| - Different approaches to solving the same problem | ||
| validations: | ||
| required: false | ||
| - type: textarea | ||
| id: additional-context | ||
| attributes: | ||
| label: Additional context | ||
| description: | | ||
| Add any other context about the feature request here. This might include: | ||
| - Links to relevant scientific papers or methods | ||
| - Example code snippets showing how you'd like to use this feature | ||
| - Screenshots or diagrams illustrating the concept | ||
| - Information about your use case or research domain | ||
| validations: | ||
| required: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| --- | ||
| file_format: mystnb | ||
| kernelspec: | ||
| name: python3 | ||
| --- | ||
|
|
||
| # Participating in the issue tracker | ||
|
|
||
| We love hearing from our community! | ||
| We want to be able to support you in your workflows, and learn about how you use Parcels. | ||
| In open source projects, getting feedback from users is hard - you posting | ||
| issues and participating in the issue tracker is really useful for us and | ||
| helps future development and squash bugs. | ||
|
|
||
| Parcels provides issue templates for when posting issues. | ||
| Following these templates provides structure and ensures that we have all the necessary information we need to help you. | ||
|
|
||
| ## "Parcels doesn't work with my input dataset" | ||
|
|
||
| Parcels is designed to work with a large range of input datasets. | ||
|
|
||
| When extending support for various input datasets, or trying to debug problems | ||
| that only occur with specific datasets, having the dataset metadata is very valuable. | ||
|
|
||
| This metadata could include information such as: | ||
|
|
||
| - CF compliant metadata | ||
| - descriptions about the origin of the dataset, or additional comments | ||
| - the shapes and data types of the arrays | ||
|
|
||
| This also allows us to see if your metadata is broken/non-compliant with standards - where we can then suggest fixes for you (and maybe we can tell the data provider!). | ||
| Since version 4 of Parcels we rely much more on metadata to discover information about your input data. | ||
|
|
||
| Sharing this metadata often provides enough debugging information to solve your problem, instead of having to share a whole dataset. | ||
|
|
||
| Sharing dataset metadata is made easy in Parcels. | ||
|
|
||
| ### Step 1. Users | ||
|
|
||
| As a user with access to your dataset, you would do: | ||
|
|
||
| ```{code-cell} | ||
| import json | ||
|
|
||
| import xarray as xr | ||
|
|
||
| # defining an example dataset to illustrate | ||
| # (you would use `xr.open_dataset(...)` instead) | ||
| ds = xr.Dataset(attrs={"description": "my dataset"}) | ||
|
|
||
| output_file = "my_dataset.json" | ||
| with open(output_file, "w") as f: | ||
| json.dump(ds.to_dict(data=False), f) # write your dataset to a JSON excluding array data | ||
| ``` | ||
|
|
||
| Then attach the produced JSON file alongside your issue | ||
|
|
||
| ### Step 2. Maintainers and developers | ||
|
|
||
| As developers looking to inspect the metadata, we would do: | ||
|
|
||
| ```{code-cell} | ||
| from parcels._datasets.utils import from_xarray_dataset_dict | ||
|
|
||
| with open(output_file) as f: | ||
| d = json.load(f) | ||
| ds = from_xarray_dataset_dict(d) | ||
| ``` | ||
|
|
||
| From there we can take a look the metadata of your dataset! |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The motivation for adding more structure to the feature suggestion template is for when v4 is released. It will be important to trace which version of Parcels people are using (e.g., "I want to do xyz, by the way I'm on v3.1.2" then the answer might just be "use v4").
Perhaps this template is a bit verbose. Open to copy edits here or anywhere in this PR