clib.conversion._to_numpy: Add tests for pandas.Series with pyarrow numeric dtypes

[seisman]

This PR adds tests for pandas.Series with pyarrow numeric dtypes. Available pyarrow numeric dtypes are at https://arrow.apache.org/docs/python/api/datatypes.html.

Similar to the issue reported in #3584, there are also dtype conversion behavior changes in pandas 2.2.

With pandas>=2.2, everything works as expected:

In [1]: import pandas as pd

In [2]: import numpy as np

In [3]: x = pd.Series([1, 2, 3], dtype="int32[pyarrow]")

In [4]: np.ascontiguousarray(x)
Out[4]: array([1, 2, 3], dtype=int32)

In [5]: x = pd.Series([1, pd.NA, 3], dtype="int32[pyarrow]")

In [6]: np.ascontiguousarray(x)
Out[6]: array([ 1., nan,  3.])

With pandas 2.1, missing values needs to convert to numpy float dtype explicitly.

In [1]: import numpy as np

In [2]: import pandas as pd

In [3]: x = pd.Series([1, 2, 3], dtype="int32[pyarrow]")

In [4]: np.ascontiguousarray(x)
Out[4]: array([1, 2, 3], dtype=int32)

In [5]: x = pd.Series([1, pd.NA, 3], dtype="int32[pyarrow]")

In [6]: np.ascontiguousarray(x)
Out[6]: array([1, <NA>, 3], dtype=object)

[seisman]

We can't use pd.ArrowDtype(pa.int8()) here because pa is not defined when pyarrow is not installed. So we have to use the string aliases.

[seisman]

This test is exactly the same as the test test_to_numpy_pandas_series_numpy_dtypes_numeric above, but I don't think we should merge them into a single test.

To merge them into a single test, we have to change the pytest.param to

pytest.param("float64[pyarrow]", np.float64, id="float64[pyarrow]", marks=skip_if_no(package="pyarrow")),

which is too long to fit in one line and will make the pytest.params too long to read.

[weiji14]

Managed to fit it into a single test like so:

from pygmt.helpers.testing import skip_if_no

pa_marks = {"marks": skip_if_no(package="pyarrow")}

@pytest.mark.parametrize(
    ("dtype", "expected_dtype"),
    [
        ...,
        pytest.param("int8[pyarrow]", np.int8, id="int8[pyarrow]", **pa_marks),
        pytest.param("int16[pyarrow]", np.int16, id="int16[pyarrow]", **pa_marks),
        pytest.param("int32[pyarrow]", np.int32, id="int32[pyarrow]", **pa_marks),
        pytest.param("int64[pyarrow]", np.int64, id="int64[pyarrow]", **pa_marks),
        pytest.param("uint8[pyarrow]", np.uint8, id="uint8[pyarrow]", **pa_marks),
        pytest.param("uint16[pyarrow]", np.uint16, id="uint16[pyarrow]", **pa_marks),
        pytest.param("uint32[pyarrow]", np.uint32, id="uint32[pyarrow]", **pa_marks),
        pytest.param("uint64[pyarrow]", np.uint64, id="uint64[pyarrow]", **pa_marks),
        pytest.param("float16[pyarrow]", np.float16, id="float16[pyarrow]", **pa_marks),
        pytest.param("float32[pyarrow]", np.float32, id="float32[pyarrow]", **pa_marks),
        pytest.param("float64[pyarrow]", np.float64, id="float64[pyarrow]", **pa_marks),
    ],
)

The longest line is just under 88 characters.

[seisman]

Nice changes. We probably should merge this PR into #3584, so that we can test all pandas dtypes (including pyarrow-backed) in a single PR.

[weiji14]

Sure, I see that this branch is targetting to_numpy/pandas_numeric/#3584, so you can merge this in then.

[seisman]

Again, we can't use pa.int8() here because pa is undefined if pyarrow is not installed.