Tests fail in Linux aarch64 and ppc64le

[joaander]

Describe the bug

freud's unit tests fail on Linux aarch64.

To Reproduce

Build the conda-forge recipe with unit tests as in conda-forge/freud-feedstock#48

Error output

https://app.travis-ci.com/github/conda-forge/freud-feedstock/jobs/569538711

System configuration (please complete the following information):

• OS: Linux aarch64
• Version of Python: all
• Version of freud: cmake-updates branch.

Additional context

I will disable the aarch64 tests on the conda-forge build for now. We can remove this exception once the tests or the bugs that the tests expose are fixed to ensure that the provided aarch64 binaries are functional.

[joaander]

These are the failures:

=================================== FAILURES ===================================
______________ TestDoctests.test_docstring[freud.box.Box.center] _______________
self = <test_doctests.TestDoctests object at 0xffff86063790>
docstring = <DocTest freud.box.Box.center from /home/conda/feedstock_root/build_artifacts/freud_1652099215006/_test_env_placehold_...lacehold_placehold_placehold_/lib/python3.7/site-packages/freud/box.cpython-37m-aarch64-linux-gnu.so:None (4 examples)>
    @pytest.mark.parametrize(
        "docstring", fetch_doctests(), ids=lambda docstring: docstring.name
    )
    def test_docstring(self, docstring):
        optionflags = doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE
        runner = doctest.DocTestRunner(optionflags=optionflags)
        runner.run(docstring)
        results = runner.summarize()
        if results.failed:
>           raise AssertionError(results)
E           AssertionError: TestResults(failed=1, attempted=4)
tests/test_doctests.py:28: AssertionError
----------------------------- Captured stdout call -----------------------------
Trying:
    import freud
Expecting nothing
ok
Trying:
    box = freud.Box.cube(10)
Expecting nothing
ok
Trying:
    points = [[-1, -1, 0], [-1, 1, 0], [2, 0, 0]]
Expecting nothing
ok
Trying:
    box.center(points)
Expecting:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
**********************************************************************
File "$PREFIX/lib/python3.7/site-packages/freud/box.cpython-37m-aarch64-linux-gnu.so", line ?, in freud.box.Box.center
Failed example:
    box.center(points)
Expected:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
Got:
    array([[-0.81540704, -1.0000002 ,  0.        ],
           [-0.81540704,  1.0000002 ,  0.        ],
           [ 2.1845937 ,  0.        ,  0.        ]], dtype=float32)
**********************************************************************
1 items had failures:
   1 of   4 in freud.box.Box.center
4 tests in 1 items.
3 passed and 1 failed.
***Test Failed*** 1 failures.
____ TestDoctests.test_docstring[freud.box.__test__.Box.center (line 433)] _____
self = <test_doctests.TestDoctests object at 0xffff83ad2290>
docstring = <DocTest freud.box.__test__.Box.center (line 433) from /home/conda/feedstock_root/build_artifacts/freud_1652099215006/...lacehold_placehold_placehold_/lib/python3.7/site-packages/freud/box.cpython-37m-aarch64-linux-gnu.so:None (4 examples)>
    @pytest.mark.parametrize(
        "docstring", fetch_doctests(), ids=lambda docstring: docstring.name
    )
    def test_docstring(self, docstring):
        optionflags = doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE
        runner = doctest.DocTestRunner(optionflags=optionflags)
        runner.run(docstring)
        results = runner.summarize()
        if results.failed:
>           raise AssertionError(results)
E           AssertionError: TestResults(failed=1, attempted=4)
tests/test_doctests.py:28: AssertionError
----------------------------- Captured stdout call -----------------------------
Trying:
    import freud
Expecting nothing
ok
Trying:
    box = freud.Box.cube(10)
Expecting nothing
ok
Trying:
    points = [[-1, -1, 0], [-1, 1, 0], [2, 0, 0]]
Expecting nothing
ok
Trying:
    box.center(points)
Expecting:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
**********************************************************************
File "$PREFIX/lib/python3.7/site-packages/freud/box.cpython-37m-aarch64-linux-gnu.so", line ?, in freud.box.__test__.Box.center (line 433)
Failed example:
    box.center(points)
Expected:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
Got:
    array([[-0.81540704, -1.0000002 ,  0.        ],
           [-0.81540704,  1.0000002 ,  0.        ],
           [ 2.1845937 ,  0.        ,  0.        ]], dtype=float32)
**********************************************************************
1 items had failures:
   1 of   4 in freud.box.__test__.Box.center (line 433)
4 tests in 1 items.
3 passed and 1 failed.
***Test Failed*** 1 failures.
__________________________ TestVoronoi.test_random_2d __________________________
self = <test_locality_Voronoi.TestVoronoi object at 0xffff7bd60250>
    def test_random_2d(self):
        # Test that voronoi tessellations of random systems have the same
        # number of points and polytopes
        L = 10  # Box length
        N = 5000  # Number of particles
        box, points = freud.data.make_random_system(L, N, is2D=True, seed=100)
        vor = freud.locality.Voronoi()
        vor.compute((box, points))
    
        # Verify the polytopes and volumes
        npt.assert_equal(len(vor.polytopes), len(points))
        npt.assert_equal(len(vor.volumes), len(points))
        npt.assert_almost_equal(np.sum(vor.volumes), box.volume)
    
        # Verify the neighbor distances
        wrapped_distances = np.linalg.norm(
            box.wrap(
                points[vor.nlist.point_indices] - points[vor.nlist.query_point_indices]
            ),
            axis=-1,
        )
>       npt.assert_allclose(wrapped_distances, vor.nlist.distances)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0
E       
E       Mismatched elements: 471 / 30000 (1.57%)
E       Max absolute difference: 5.9604645e-08
E       Max relative difference: 1.19187085e-07
E        x: array([0.058061, 0.065299, 0.144335, ..., 0.062287, 0.122593, 0.049249],
E             dtype=float32)
E        y: array([0.058061, 0.065299, 0.144335, ..., 0.062287, 0.122593, 0.049249],
E             dtype=float32)
tests/test_locality_Voronoi.py:35: AssertionError
__________________________ TestVoronoi.test_random_3d __________________________
self = <test_locality_Voronoi.TestVoronoi object at 0xffff7bd60690>
    def test_random_3d(self):
        # Test that voronoi tessellations of random systems have the same
        # number of points and polytopes
        L = 10  # Box length
        N = 5000  # Number of particles
        box, points = freud.data.make_random_system(L, N, is2D=False, seed=100)
        vor = freud.locality.Voronoi()
        vor.compute((box, points))
    
        # Verify the polytopes and volumes
        npt.assert_equal(len(vor.polytopes), len(points))
        npt.assert_equal(len(vor.volumes), len(points))
        npt.assert_almost_equal(np.sum(vor.volumes), box.volume)
    
        # Verify the neighbor distances
        wrapped_distances = np.linalg.norm(
            box.wrap(
                points[vor.nlist.point_indices] - points[vor.nlist.query_point_indices]
            ),
            axis=-1,
        )
>       npt.assert_allclose(wrapped_distances, vor.nlist.distances)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0
E       
E       Mismatched elements: 792 / 77107 (1.03%)
E       Max absolute difference: 1.1920929e-07
E       Max relative difference: 1.5723617e-07
E        x: array([0.470718, 0.383203, 1.065738, ..., 0.709391, 0.195902, 0.545836],
E             dtype=float32)
E        y: array([0.470718, 0.383203, 1.065738, ..., 0.709391, 0.195902, 0.545836],
E             dtype=float32)
tests/test_locality_Voronoi.py:72: AssertionError

[joaander]

Tests also fail on ppc64le:

=================================== FAILURES ===================================
______________ TestDoctests.test_docstring[freud.box.Box.center] _______________
self = <test_doctests.TestDoctests object at 0x7e9b780257c0>
docstring = <DocTest freud.box.Box.center from /home/conda/feedstock_root/build_artifacts/freud_1652108036483/_test_env_placehold_...ehold_placehold_placehold_/lib/python3.9/site-packages/freud/box.cpython-39-powerpc64le-linux-gnu.so:None (4 examples)>
    @pytest.mark.parametrize(
        "docstring", fetch_doctests(), ids=lambda docstring: docstring.name
    )
    def test_docstring(self, docstring):
        optionflags = doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE
        runner = doctest.DocTestRunner(optionflags=optionflags)
        runner.run(docstring)
        results = runner.summarize()
        if results.failed:
>           raise AssertionError(results)
E           AssertionError: TestResults(failed=1, attempted=4)
tests/test_doctests.py:28: AssertionError
----------------------------- Captured stdout call -----------------------------
Trying:
    import freud
Expecting nothing
ok
Trying:
    box = freud.Box.cube(10)
Expecting nothing
ok
Trying:
    points = [[-1, -1, 0], [-1, 1, 0], [2, 0, 0]]
Expecting nothing
ok
Trying:
    box.center(points)
Expecting:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
**********************************************************************
File "$PREFIX/lib/python3.9/site-packages/freud/box.cpython-39-powerpc64le-linux-gnu.so", line ?, in freud.box.Box.center
Failed example:
    box.center(points)
Expected:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
Got:
    array([[-0.81540704, -1.0000002 ,  0.        ],
           [-0.81540704,  1.0000002 ,  0.        ],
           [ 2.1845937 ,  0.        ,  0.        ]], dtype=float32)
**********************************************************************
1 items had failures:
   1 of   4 in freud.box.Box.center
4 tests in 1 items.
3 passed and 1 failed.
***Test Failed*** 1 failures.
____ TestDoctests.test_docstring[freud.box.__test__.Box.center (line 433)] _____
self = <test_doctests.TestDoctests object at 0x7e9b780c1280>
docstring = <DocTest freud.box.__test__.Box.center (line 433) from /home/conda/feedstock_root/build_artifacts/freud_1652108036483/...ehold_placehold_placehold_/lib/python3.9/site-packages/freud/box.cpython-39-powerpc64le-linux-gnu.so:None (4 examples)>
    @pytest.mark.parametrize(
        "docstring", fetch_doctests(), ids=lambda docstring: docstring.name
    )
    def test_docstring(self, docstring):
        optionflags = doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE
        runner = doctest.DocTestRunner(optionflags=optionflags)
        runner.run(docstring)
        results = runner.summarize()
        if results.failed:
>           raise AssertionError(results)
E           AssertionError: TestResults(failed=1, attempted=4)
tests/test_doctests.py:28: AssertionError
----------------------------- Captured stdout call -----------------------------
Trying:
    import freud
Expecting nothing
ok
Trying:
    box = freud.Box.cube(10)
Expecting nothing
ok
Trying:
    points = [[-1, -1, 0], [-1, 1, 0], [2, 0, 0]]
Expecting nothing
ok
Trying:
    box.center(points)
Expecting:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
**********************************************************************
File "$PREFIX/lib/python3.9/site-packages/freud/box.cpython-39-powerpc64le-linux-gnu.so", line ?, in freud.box.__test__.Box.center (line 433)
Failed example:
    box.center(points)
Expected:
    array([[-0.8154068, -1.0000002,  0.       ],
           [-0.8154068,  1.       ,  0.       ],
           [ 2.1845937,  0.       ,  0.       ]], dtype=float32)
Got:
    array([[-0.81540704, -1.0000002 ,  0.        ],
           [-0.81540704,  1.0000002 ,  0.        ],
           [ 2.1845937 ,  0.        ,  0.        ]], dtype=float32)
**********************************************************************
1 items had failures:
   1 of   4 in freud.box.__test__.Box.center (line 433)
4 tests in 1 items.
3 passed and 1 failed.
***Test Failed*** 1 failures.
__________________________ TestVoronoi.test_random_2d __________________________
self = <test_locality_Voronoi.TestVoronoi object at 0x7e9affd765e0>
    def test_random_2d(self):
        # Test that voronoi tessellations of random systems have the same
        # number of points and polytopes
        L = 10  # Box length
        N = 5000  # Number of particles
        box, points = freud.data.make_random_system(L, N, is2D=True, seed=100)
        vor = freud.locality.Voronoi()
        vor.compute((box, points))
    
        # Verify the polytopes and volumes
        npt.assert_equal(len(vor.polytopes), len(points))
        npt.assert_equal(len(vor.volumes), len(points))
        npt.assert_almost_equal(np.sum(vor.volumes), box.volume)
    
        # Verify the neighbor distances
        wrapped_distances = np.linalg.norm(
            box.wrap(
                points[vor.nlist.point_indices] - points[vor.nlist.query_point_indices]
            ),
            axis=-1,
        )
>       npt.assert_allclose(wrapped_distances, vor.nlist.distances)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0
E       
E       Mismatched elements: 471 / 30000 (1.57%)
E       Max absolute difference: 5.9604645e-08
E       Max relative difference: 1.19187085e-07
E        x: array([0.058061, 0.065299, 0.144335, ..., 0.062287, 0.122593, 0.049249],
E             dtype=float32)
E        y: array([0.058061, 0.065299, 0.144335, ..., 0.062287, 0.122593, 0.049249],
E             dtype=float32)
tests/test_locality_Voronoi.py:35: AssertionError
__________________________ TestVoronoi.test_random_3d __________________________
self = <test_locality_Voronoi.TestVoronoi object at 0x7e9affd760d0>
    def test_random_3d(self):
        # Test that voronoi tessellations of random systems have the same
        # number of points and polytopes
        L = 10  # Box length
        N = 5000  # Number of particles
        box, points = freud.data.make_random_system(L, N, is2D=False, seed=100)
        vor = freud.locality.Voronoi()
        vor.compute((box, points))
    
        # Verify the polytopes and volumes
        npt.assert_equal(len(vor.polytopes), len(points))
        npt.assert_equal(len(vor.volumes), len(points))
        npt.assert_almost_equal(np.sum(vor.volumes), box.volume)
    
        # Verify the neighbor distances
        wrapped_distances = np.linalg.norm(
            box.wrap(
                points[vor.nlist.point_indices] - points[vor.nlist.query_point_indices]
            ),
            axis=-1,
        )
>       npt.assert_allclose(wrapped_distances, vor.nlist.distances)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0
E       
E       Mismatched elements: 792 / 77107 (1.03%)
E       Max absolute difference: 1.1920929e-07
E       Max relative difference: 1.5723617e-07
E        x: array([0.470718, 0.383203, 1.065738, ..., 0.709391, 0.195902, 0.545836],
E             dtype=float32)
E        y: array([0.470718, 0.383203, 1.065738, ..., 0.709391, 0.195902, 0.545836],
E             dtype=float32)

[vyasr]

Prior to the voro++ switch the voronoi module was using double precision (unlike everything else in freud that uses single precision), but since we switched to voro++ I don't know how much control we even have over the underlying calculation precision. Aside from a minor 2D issue (#588) I'm honestly not even sure what precision problems we've dealt with since switching to voro++. #362 might provided some hints, otherwise @bdice should know more. Worst case we just bump the tolerance of these tests, but it would be nice to know what parts of the calculation are the most sensitive in case there is a way to avoid loss of precision by writing more numerically robust code.

[bdice]

It looks like all these errors are from differences in single precision handling of the input data. I would relax all of these tests to a lower precision. It seems that rtol=1e-6, atol=1e-6 would be fine from the absolute/relative differences seen here. Historically we have just relaxed tolerances whenever tests fail on account of single-precision numerical precision. Some tests (such as the failing ones in this code) don't have these arguments specified and thus fall back to numpy's defaults of rtol=1e-7, atol=0. Voronoi computations are somewhat more prone to these numerical problems because of the sheer number of floating point calculations.

[vyasr]

Yeah I'm fine relaxing the tolerances in the test. Voronoi is a tricky case because despite the calculations being in double precision we still run into these problems, which make me wonder if there are more numerically stable ways to do the calculation. Now that we're outsourcing to voro++ I guess that's largely out of our hands though, and so far as we know it's been good enough for users as is.