Skip to content

NaN-handling for ensemble CRPS#118

Open
frazane wants to merge 16 commits into
mainfrom
nan-handling-crps
Open

NaN-handling for ensemble CRPS#118
frazane wants to merge 16 commits into
mainfrom
nan-handling-crps

Conversation

@frazane
Copy link
Copy Markdown
Owner

@frazane frazane commented Mar 30, 2026

This pull request introduces handling of NaN values in ensemble CRPS scoring functions via NaN policy selection. Three policies are supported:

  • 'propagate': if any member of an ensemble is NaN, the function returns NaN
  • 'omit': the function will omit NaN values from the computation
  • 'raise': the function will raise an error if any NaNs are present in the input arrays

These are now values given to a new nan_policy parameter for crps_ensemble, twcrps_ensemble, owcrps_ensemble, and vrcrps_ensemble.

Comment thread scoringrules/core/crps/_approx.py Outdated
Comment thread scoringrules/core/crps/_approx.py Outdated
Comment thread scoringrules/_crps.py
Comment thread scoringrules/core/crps/_approx.py Outdated
Comment thread scoringrules/core/crps/_approx.py Outdated
Comment thread scoringrules/core/crps/_approx.py Outdated
Comment thread scoringrules/core/crps/_gufuncs.py
Comment thread scoringrules/core/utils.py
Comment thread scoringrules/core/utils.py
Comment thread tests/test_crps_nan_policy.py Outdated
Comment thread tests/test_crps_nan_policy.py Outdated
@frazane
Copy link
Copy Markdown
Owner Author

frazane commented Apr 26, 2026

@sallen12 I reimplemented the NaN-handling like you suggested, using the ensemble weighting functionality and setting NaN members weights (and values) to zero. I only realized this was indeed an option after merging main into this branch. My bad!

This is now greatly simplified. There is still a couple of edge cases that need to be handled (AKR estimators). Perhaps we can have a chat about this soon.

@frazane frazane force-pushed the nan-handling-crps branch from 685255c to ab55a30 Compare May 2, 2026 13:59
@frazane frazane marked this pull request as ready for review May 2, 2026 14:02
@frazane frazane requested a review from sallen12 May 2, 2026 14:02
@frazane
Copy link
Copy Markdown
Owner Author

frazane commented May 2, 2026

@nicholasloveday any feedback here would be welcome!

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I left the ad-hoc AKR estimators for the 'omit' nan policy, but we are not using them in the high level API. We will decide later. For now, choosing AKR estimators or INT with 'omit' policy will raise a NotImplementedException.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants