Allow exceptions that fail pickle deserialisation to be easily identified downstream.

[timw6n]

Something we see quite a lot in our production dask (Coiled-powered) setup is exceptions failing to be deserialised when sent from the cluster to the client. This is usually due to either version mismatch or code being present within the cluster software environment that is not there on the client. Obviously this is not an ideal situation and one we're working to resolve.

However, the below change should make it easier to handle this situation in the meantime, as it makes such errors obvious (and able to be easily caught and handled, e.g. by adding a sensible user-facing message) rather than having to catch Exception and fish about for if the first argument is a bytestring that looks like a pickled object.

I think this change should be compatible with any existing code that is attempting to do the above. If a breaking change were possible, would be nicer to add a "Unreadable pickled exception received" message to UnreadablePickledException and attach the actual pickled output as a separate field.

Closes #xxxx

• Tests added / passed
• Passes pre-commit run --all-files

(I hope all the tests will pass on CI, seeing failures locally but I think that's due to not having working IPv6.)

[GPUtester]

Can one of the admins verify this patch?

Admins can comment ok to test to allow this one PR to run or add to allowlist to allow all future PRs from the same author to run.

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    25 files  ± 0      25 suites  ±0   10h 27m 19s ⏱️ + 2m 12s
 4 130 tests + 1   4 015 ✅ ± 0    110 💤 ±0  5 ❌ +1 
47 710 runs  +12  45 608 ✅ +12  2 097 💤 +1  5 ❌  - 1 

For more details on these failures, see this check.

Results for commit bfccb53. ± Comparison against base commit 36020d6.