Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-358: add static types support #368

Merged
merged 160 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from 128 commits
Commits
Show all changes
160 commits
Select commit Hold shift + click to select a range
4ba78c1
gh-280: add `mypy` pre-commit hook
paddyroddy Oct 3, 2024
b23ded0
Add scientific Python recommended config
paddyroddy Oct 3, 2024
b9ee0eb
Ignore `npt.NDArray` without specified type
paddyroddy Oct 3, 2024
3f80029
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 4, 2024
79f1ac3
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 4, 2024
3856aa9
Add more `ignore[type-arg]`
paddyroddy Oct 4, 2024
58696dc
Add `None` return type for tests
paddyroddy Oct 4, 2024
d22891e
Add types from docstrings
paddyroddy Oct 4, 2024
e6c8f77
Add `rng` type
paddyroddy Oct 4, 2024
ef1c0d9
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 4, 2024
f53ffc5
Add other `rng`
paddyroddy Oct 4, 2024
46c75e9
Switch from `NDArray` to `ArrayLke`
paddyroddy Oct 7, 2024
cdc4312
Add typing to `array`
paddyroddy Oct 7, 2024
6e456e8
Add types to `user`
paddyroddy Oct 7, 2024
5b3a94c
Add `points` typing
paddyroddy Oct 7, 2024
ec53ba5
Typing for `shells`
paddyroddy Oct 7, 2024
e59a5fa
Add more typing
paddyroddy Oct 7, 2024
d4e5396
Turn off decorator warnings
paddyroddy Oct 7, 2024
a2a1a5f
Add return types
paddyroddy Oct 7, 2024
c2c145b
More typing
paddyroddy Oct 7, 2024
b28be77
Run `mypy_clean_slate`
paddyroddy Oct 8, 2024
9151d81
More fixing
paddyroddy Oct 8, 2024
cc07ade
`mypy` passes
paddyroddy Oct 8, 2024
6883e1c
Add space
paddyroddy Oct 8, 2024
c11e82f
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 9, 2024
3d894b4
Fix `mypy`
paddyroddy Oct 9, 2024
12ad51a
Fix mypy
paddyroddy Oct 9, 2024
5bf6a27
Fix linting
paddyroddy Oct 9, 2024
aab5b92
Separate out all `typing` imports
paddyroddy Oct 9, 2024
0ff8b08
Using `typing.Any` throughout
paddyroddy Oct 9, 2024
fe9daf2
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 10, 2024
c858c98
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 14, 2024
cc231ed
Fix `mypy`
paddyroddy Oct 14, 2024
6aff2ea
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 15, 2024
12723f8
Fix linting
paddyroddy Oct 15, 2024
52b2fc8
Undo `.gitignore`
paddyroddy Oct 15, 2024
050c36a
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 15, 2024
74a94de
Fix `mypy`
paddyroddy Oct 15, 2024
0864ddb
Merge branch 'main' into paddy/issue-280
paddyroddy Oct 15, 2024
2e2020f
Fix merge
paddyroddy Oct 15, 2024
bc62666
New line
paddyroddy Oct 15, 2024
27f4ab4
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 15, 2024
c1fdd6d
Fix formatting
paddyroddy Oct 15, 2024
6d6d447
Fix `Returns` in docstrings (I think entered in #357)
paddyroddy Oct 15, 2024
484c7bd
Switch numpy arrays to float/complex
paddyroddy Oct 15, 2024
7ddf501
Fix some more typing
paddyroddy Oct 15, 2024
7b66122
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 16, 2024
008e9a5
Remove some `untyped-def`
paddyroddy Oct 16, 2024
112c36c
Add typing for `redshifts_for_nz`
paddyroddy Oct 16, 2024
394f458
`positions_from_delta`
paddyroddy Oct 16, 2024
c9b4b63
`uniform_positions`
paddyroddy Oct 16, 2024
57f2820
`catpos`
paddyroddy Oct 16, 2024
205ae4c
Fix `return-value`
paddyroddy Oct 16, 2024
42cb3b8
`redundant-expr`
paddyroddy Oct 16, 2024
8f4f79f
Fix more typing
paddyroddy Oct 17, 2024
f165dcb
Add types to lists
paddyroddy Oct 17, 2024
3d86e19
Type count
paddyroddy Oct 17, 2024
e809b1c
`ndinterp`
paddyroddy Oct 17, 2024
cb2b05c
Type `z`
paddyroddy Oct 17, 2024
b9ddf83
Type `triaxial_axis_ratio`
paddyroddy Oct 17, 2024
9ebfa62
Add more typing
paddyroddy Oct 17, 2024
a5b0f7b
Type `ngal`
paddyroddy Oct 17, 2024
7fe9725
Fix `uniform_positions`
paddyroddy Oct 17, 2024
eaa525c
`var-annotated`
paddyroddy Oct 17, 2024
a8753f3
Tidy `catpos`
paddyroddy Oct 17, 2024
4ed953b
Fix `nz` typing
paddyroddy Oct 17, 2024
5d4b632
Fix lower/upper
paddyroddy Oct 17, 2024
ce10ed0
Fix `wht`
paddyroddy Oct 17, 2024
d6c1cfb
`cmask` and `count`
paddyroddy Oct 17, 2024
b1d0131
Type `a`
paddyroddy Oct 17, 2024
fca667e
Type `ellipticity_ryden04`
paddyroddy Oct 17, 2024
c58fc87
Fix `ellipticity_gaussian`
paddyroddy Oct 17, 2024
16e7b15
`ellipticity_intnorm`
paddyroddy Oct 17, 2024
7c9341b
Sort `deflect` typing
paddyroddy Oct 17, 2024
7b6d294
Fix typing
paddyroddy Oct 17, 2024
6e0f29d
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 18, 2024
0b8d071
Fix `arg-type`
paddyroddy Oct 18, 2024
6c8f924
`dtype`
paddyroddy Oct 18, 2024
7b2703f
`zsrc`
paddyroddy Oct 18, 2024
a6af6ea
Remove custom types
paddyroddy Oct 18, 2024
aff1581
Remove `collections.abc.Sequence`
paddyroddy Oct 18, 2024
e3722a7
`galaxy_shear`
paddyroddy Oct 18, 2024
b70ad8a
`trapz_product`
paddyroddy Oct 18, 2024
f9814ce
Switch to numpy empty
paddyroddy Oct 18, 2024
487810b
Add `pytest_mock`
paddyroddy Oct 18, 2024
1c2296a
Fix ext path
paddyroddy Oct 18, 2024
43ad02c
`assert isinstance`
paddyroddy Oct 18, 2024
2e73bce
More `assert isinstance`
paddyroddy Oct 18, 2024
cebe357
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 18, 2024
f08ab67
`ef`
paddyroddy Oct 18, 2024
41c129b
`xm`
paddyroddy Oct 18, 2024
015496c
Move `_test_append` into function
paddyroddy Oct 18, 2024
1e154e1
Fix typing of basic write
paddyroddy Oct 18, 2024
a10fab9
Fix results tuple
paddyroddy Oct 18, 2024
994fdf3
`uniform_positions`
paddyroddy Oct 21, 2024
85961b8
Fix `phz` typing
paddyroddy Oct 21, 2024
6e7efbe
Create conditional `alpha` function
paddyroddy Oct 21, 2024
48cdb53
Move `attr-defined` ignore nearer definition
paddyroddy Oct 21, 2024
b730b32
Sort `attr-defined`
paddyroddy Oct 21, 2024
3c097ca
Disable `warn_return_any`
paddyroddy Oct 21, 2024
aa92c4d
Fix `unreachable` code
paddyroddy Oct 22, 2024
1c77c15
List comprehension fixes assignment mypy
paddyroddy Oct 22, 2024
426db7b
Fix notebook
paddyroddy Oct 22, 2024
8971dad
Look at `broadcast_leading_axes`
paddyroddy Oct 22, 2024
ddfe641
Simplify `redshifts_from_nz`
paddyroddy Oct 23, 2024
3f9c158
Switch to arrays
paddyroddy Oct 23, 2024
3f63c2e
Simplify more types
paddyroddy Oct 23, 2024
4d78c5e
Remove double import
paddyroddy Oct 23, 2024
2cb042d
Fix tests
paddyroddy Oct 23, 2024
0a0ed29
Remove float
paddyroddy Oct 23, 2024
5603418
Fix return value
paddyroddy Oct 23, 2024
5e520ce
Undo number
paddyroddy Oct 23, 2024
9b5a011
Fix `broadcast_leading_axes`
paddyroddy Oct 23, 2024
f672986
Fix the `zsrc` errors
paddyroddy Oct 24, 2024
64d6991
Fix test
paddyroddy Oct 24, 2024
29a685e
Fix `fitsio` import
paddyroddy Oct 24, 2024
fada69c
Fix `union-attr`
paddyroddy Oct 24, 2024
22efda8
Fix `arg-type`
paddyroddy Oct 24, 2024
3a7a1d0
Fix assignment
paddyroddy Oct 24, 2024
88f41c6
Fix test broadcast
paddyroddy Oct 24, 2024
e7cdfd6
Fix `*rest`
paddyroddy Oct 24, 2024
61e273e
Fix assignment
paddyroddy Oct 24, 2024
a921881
Add typing metadata
paddyroddy Oct 24, 2024
1965509
`bias` can be array
paddyroddy Oct 24, 2024
05ad150
Create `bias_model_callable`
paddyroddy Oct 24, 2024
372226b
Fix `mypy`
paddyroddy Oct 24, 2024
c200a5d
Add to `git blame` file
paddyroddy Oct 24, 2024
9062f17
Ruff formating
paddyroddy Oct 24, 2024
0b4a446
Make size a tuple
paddyroddy Oct 24, 2024
6c47fcd
Change `extname`
paddyroddy Oct 24, 2024
f021549
Fix `ext`
paddyroddy Oct 24, 2024
c0177c5
Change to `np.sum`
paddyroddy Oct 24, 2024
6012cb6
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 24, 2024
437bba3
Change from `dtype` to `DTypeLike`
paddyroddy Oct 28, 2024
6a869b4
Change to `not`
paddyroddy Oct 28, 2024
00205d7
Merge branch 'main' into paddy/issue-358
paddyroddy Oct 28, 2024
397efe3
Change from complex to float
paddyroddy Oct 28, 2024
5f3d17e
Change `RadialWindow` to a `Sequence`
paddyroddy Oct 28, 2024
5a16b36
`mu` and `sigma` also floats
paddyroddy Oct 28, 2024
ea4a36b
Fix return types
paddyroddy Oct 28, 2024
43b20d0
Change test to satisfy `mypy`
paddyroddy Oct 28, 2024
938782c
Fix write/load for None type
paddyroddy Oct 28, 2024
152c214
Restore int
paddyroddy Oct 28, 2024
0dca63b
Fix types based on NumPy v2
paddyroddy Oct 30, 2024
7c30d01
Disable `ruff` `ANN` rules
paddyroddy Oct 30, 2024
4d6bcb4
Disable more `ANN` rules in `tests`
paddyroddy Oct 30, 2024
0df5a8c
Merge branch 'main' into paddy/issue-358
paddyroddy Nov 5, 2024
f5ea6cd
Merge branch 'main' into paddy/issue-358
paddyroddy Nov 5, 2024
4961d2e
Remove blame file
paddyroddy Nov 6, 2024
e96403f
Turn off global `warn_return_any` setting
paddyroddy Nov 6, 2024
0afdd0e
Merge branch 'main' into paddy/issue-358
paddyroddy Nov 6, 2024
515ed8c
Remove array part of floats
paddyroddy Nov 6, 2024
391a27a
Explicitly convert to float
paddyroddy Nov 6, 2024
7af8b51
Restore `ext` is `None`
paddyroddy Nov 6, 2024
b2df82c
Return none to triaxial
paddyroddy Nov 6, 2024
6ea68e0
Restore size
paddyroddy Nov 6, 2024
dc5bfdf
Restore `WeightFunc`
paddyroddy Nov 6, 2024
de9cdc9
Restore `ArrayLike1D`
paddyroddy Nov 6, 2024
882e7ec
Restore Cls
paddyroddy Nov 6, 2024
113d4aa
Only run on `glass`
paddyroddy Nov 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ b42067f6776dcd763827000d585a88e071b78af3
f9bac62f497a7288aa71fd4dbd948945c37f854e
# applied ruff-linting - https://github.com/glass-dev/glass/pull/232
e58d8616c03b8447aba90996905a98b42f18ba0a
# added static typing - https://github.com/glass-dev/glass/pull/368
paddyroddy marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions .gitignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is really a change that needs to happen in this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, but can we leave it? I had to run them to see the outputs for typing

Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ dist
.env
.coverage*
coverage*
.ipynb_checkpoints
2 changes: 1 addition & 1 deletion docs/conf.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would also make sense to me to include only glass/ in the type checking.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, it is only possible to exclude modules. Excluding folders doesn't seem supported python/mypy#17977

Copy link
Collaborator

@ntessore ntessore Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it not suffice to add packages = "glass" to [tool.mypy]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tried that and it didn't work... 🤔

Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@

html_logo = "_static/logo.png"
html_favicon = "_static/favicon.ico"
html_css_files = [] # type: ignore[var-annotated]
html_css_files: list[str] = []


# -- Intersphinx -------------------------------------------------------------
Expand Down
7 changes: 6 additions & 1 deletion examples/2-advanced/cosmic_shear.ipynb
paddyroddy marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,12 @@
"\n",
" # apply the shear fields to the ellipticities\n",
" gal_she = glass.galaxy_shear(\n",
" gal_lon, gal_lat, gal_eps, kappa_i, gamm1_i, gamm2_i\n",
" gal_lon,\n",
" gal_lat,\n",
" gal_eps,\n",
" kappa_i,\n",
" gamm1_i,\n",
" gamm2_i,\n",
" )\n",
"\n",
" # map the galaxy shears to a HEALPix map; this is opaque but works\n",
Expand Down
12 changes: 10 additions & 2 deletions examples/2-advanced/stage_4_galaxies.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,10 @@
"\n",
" # generate galaxy positions from the matter density contrast\n",
" for gal_lon, gal_lat, gal_count in glass.positions_from_delta(\n",
" ngal[i], delta_i, bias, vis\n",
" ngal[i],\n",
" delta_i,\n",
" bias,\n",
" vis,\n",
" ):\n",
" # generate random redshifts over the given shell\n",
" gal_z = glass.redshifts(gal_count, ws[i])\n",
Expand All @@ -297,7 +300,12 @@
"\n",
" # apply the shear fields to the ellipticities\n",
" gal_she = glass.galaxy_shear(\n",
" gal_lon, gal_lat, gal_eps, kappa_i, gamm1_i, gamm2_i\n",
" gal_lon,\n",
" gal_lat,\n",
" gal_eps,\n",
" kappa_i,\n",
" gamm1_i,\n",
" gamm2_i,\n",
" )\n",
"\n",
" # make a mini-catalogue for the new rows\n",
Expand Down
6 changes: 3 additions & 3 deletions glass/core/algorithm.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@


def nnls(
a: npt.ArrayLike,
b: npt.ArrayLike,
a: npt.NDArray[np.float64],
b: npt.NDArray[np.float64],
*,
tol: float = 0.0,
maxiter: int | None = None,
) -> npt.ArrayLike:
) -> npt.NDArray[np.float64]:
"""
Compute a non-negative least squares solution.

Expand Down
50 changes: 44 additions & 6 deletions glass/core/array.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,32 @@
"""Module for array utilities."""

from __future__ import annotations

import typing
from functools import partial

import numpy as np
import numpy.typing as npt


def broadcast_first(*arrays): # type: ignore[no-untyped-def]
def broadcast_first(
*arrays: npt.NDArray[np.float64],
) -> tuple[npt.NDArray[np.float64], ...]:
"""Broadcast arrays, treating the first axis as common."""
arrays = tuple(np.moveaxis(a, 0, -1) if np.ndim(a) else a for a in arrays)
arrays = np.broadcast_arrays(*arrays)
return tuple(np.moveaxis(a, -1, 0) if np.ndim(a) else a for a in arrays)


def broadcast_leading_axes(*args): # type: ignore[no-untyped-def]
def broadcast_leading_axes(
*args: tuple[
float | npt.NDArray[np.float64],
int,
],
) -> tuple[
tuple[int, ...],
typing.Unpack[tuple[npt.NDArray[np.float64], ...]],
]:
ntessore marked this conversation as resolved.
Show resolved Hide resolved
"""
Broadcast all but the last N axes.

Expand Down Expand Up @@ -49,7 +63,15 @@ def broadcast_leading_axes(*args): # type: ignore[no-untyped-def]
return (dims, *arrs)


def ndinterp(x, xp, fp, axis=-1, left=None, right=None, period=None): # type: ignore[no-untyped-def] # noqa: PLR0913
def ndinterp( # noqa: PLR0913
x: float | npt.NDArray[np.float64],
xp: npt.NDArray[np.float64],
fp: npt.NDArray[np.float64],
axis: int = -1,
left: float | None = None,
right: float | None = None,
period: float | None = None,
) -> npt.NDArray[np.float64]:
"""Interpolate multi-dimensional array over axis."""
return np.apply_along_axis(
partial(np.interp, x, xp),
Expand All @@ -61,7 +83,14 @@ def ndinterp(x, xp, fp, axis=-1, left=None, right=None, period=None): # type: i
)


def trapz_product(f, *ff, axis=-1): # type: ignore[no-untyped-def]
def trapz_product(
f: tuple[npt.NDArray[np.float64], npt.NDArray[np.float64]],
*ff: tuple[
npt.NDArray[np.float64],
npt.NDArray[np.float64],
],
axis: int = -1,
) -> npt.NDArray[np.float64]:
"""Trapezoidal rule for a product of functions."""
x, _ = f
for x_, _ in ff:
Expand All @@ -72,10 +101,19 @@ def trapz_product(f, *ff, axis=-1): # type: ignore[no-untyped-def]
y = np.interp(x, *f)
for f_ in ff:
y *= np.interp(x, *f_)
return np.trapz(y, x, axis=axis) # type: ignore[attr-defined]
return np.trapz( # type: ignore[attr-defined]
y,
x,
axis=axis,
)


def cumtrapz(f, x, dtype=None, out=None): # type: ignore[no-untyped-def]
def cumtrapz(
f: npt.NDArray[np.int_] | npt.NDArray[np.float64],
x: npt.NDArray[np.int_] | npt.NDArray[np.float64],
dtype: type | None = None,
paddyroddy marked this conversation as resolved.
Show resolved Hide resolved
out: npt.NDArray[np.float64] | None = None,
) -> npt.NDArray[np.float64]:
"""Cumulative trapezoidal rule along last axis."""
if out is None:
out = np.empty_like(f, dtype=dtype)
Expand Down
4 changes: 2 additions & 2 deletions glass/ext/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"""


def _extend_path(path, name) -> list: # type: ignore[no-untyped-def, type-arg]
def _extend_path(path: list[str], name: str) -> list[str]:
import os.path
from pkgutil import extend_path

Expand All @@ -17,7 +17,7 @@ def _extend_path(path, name) -> list: # type: ignore[no-untyped-def, type-arg]
filter(
os.path.isdir,
(os.path.join(p, _mod) for p in extend_path(path, _pkg)), # noqa: PTH118
)
),
)


Expand Down
Loading