Should we be implementing __array__?

[asmeurer]

Right now the array type implements __array__, but just as convenience. This was done back when the package was numpy.array_api.

I'm wondering if we should actually remove this. It isn't part of the array API standard, and could just lead people incorrectly thinking it works. And more notably, it is leading to bugs in array_api_strict itself (like #66) where you can forget to use ._array when passing an array to a NumPy function and it will still work (but maybe not in the same way, if that function also passes the dtype argument for instance).

[asmeurer]

I tested removing it and found some more issues where weren't using ._array like in #66. I pushed fixes for this to #68.

At the same time, if you remove __array__, then NumPy converts it to an object array:

>>> import numpy as np
>>> import array_api_strict as xp
>>> np.asarray(xp.asarray([10]))
array(Array([10], dtype=array_api_strict.int64), dtype=object)

Or we can explicitly make __array__ raise an exception, which would make np.asarray(<array_api_strict Array>) fail.

[asmeurer]

A small hiccup here is that we implicitly use this logic in asarray for conversion from lists of 0-D arrays:

array-api-strict/array_api_strict/_creation_functions.py

Line 97 in 019d935

res = np.array(obj, dtype=_np_dtype, copy=copy)

I believe that should be supported in the standard, but the logic in asarray seems to check __array__ first. The docs don't mention any kind of fallback exception that can be used https://numpy.org/devdocs/user/basics.interoperability.html#the-array-method. Maybe there's some internal function in NumPy for converting lists-of-lists that we can reuse?

[asmeurer]

Prototype of this without the fix to asarray at #69.

[mdhaber]

If this has to be removed, I'd suggest that the error message at least suggest how such a conversion should be performed (e.g. from_dlpack?). This is causing 5.7k+ test failures in SciPy right now (see scipy/scipy#21828).

[rgommers]

+1 to include a hint about how to fix. Writing that hint is hard - which is a problem.

There are more relevant comments on gh-69, so let's keep the discussion there I'd say.

[asmeurer]

So do you think this change should be reverted? I can also issue a warning in __array__ instead of removing it outright.

[rgommers]

I can also issue a warning in __array__ instead of removing it outright.

A warning doesn't help, the SciPy test suite will turn those straight into errors. And there'll be either zero or a million warnings probably.