-
Notifications
You must be signed in to change notification settings - Fork 298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The ahi_hsd
reader returns Brightness Temperatures as float64
#2880
Comments
Looks like this is specific to BTs:
|
Just about every scalar used in the BT calibration is a 64-bit float. Is it "the right thing to do" to cast them to 32-bit floats? |
We talked on slack after I noticed that counts are not returned as integers and have no _FillValue defined (because it is NaN). I think we agreed this should be changed, but means I really don't have the time to tackle this any more. Here's the changes I started to make so far: Subject: [PATCH] Initial AHI dtype work
---
Index: satpy/readers/ahi_hsd.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/satpy/readers/ahi_hsd.py b/satpy/readers/ahi_hsd.py
--- a/satpy/readers/ahi_hsd.py (revision 10c63025ffb9538973e61f15fb19436eda28938e)
+++ b/satpy/readers/ahi_hsd.py (date 1724428627166)
@@ -743,7 +743,7 @@
c1_ = self._header["calibration"]["c1_rad2tb_conversion"][0]
c2_ = self._header["calibration"]["c2_rad2tb_conversion"][0]
- return (c0_ + c1_ * Te_ + c2_ * Te_ ** 2).clip(0)
+ return (c0_ + c1_ * Te_ + c2_ * Te_ ** 2).clip(0).astype(data.dtype)
class _NominalTimeCalculator:
Index: satpy/tests/reader_tests/test_ahi_hsd.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/satpy/tests/reader_tests/test_ahi_hsd.py b/satpy/tests/reader_tests/test_ahi_hsd.py
--- a/satpy/tests/reader_tests/test_ahi_hsd.py (revision 10c63025ffb9538973e61f15fb19436eda28938e)
+++ b/satpy/tests/reader_tests/test_ahi_hsd.py (date 1724433974162)
@@ -361,12 +361,19 @@
fh.read_band(mock.MagicMock(), mock.MagicMock())
mask_space.assert_not_called()
- def test_read_band_from_actual_file(self, hsd_file_jp01):
+ @pytest.mark.parametrize(
+ ("calib", "exp_units"),
+ [
+ ("counts", "1"),
+ ("reflectance", "%")
+ ]
+ )
+ def test_read_vis_band_from_actual_file(self, hsd_file_jp01, calib, exp_units):
"""Test read_bands on real data."""
filename_info = {"segment": 1, "total_segments": 1}
filetype_info = {"file_type": "blahB01"}
fh = AHIHSDFileHandler(hsd_file_jp01, filename_info, filetype_info)
- key = {"name": "B01", "calibration": "counts", "resolution": 1000}
+ key = {"name": "B01", "calibration": calib, "resolution": 1000}
import dask
with dask.config.set({"array.chunk-size": "32MiB"}):
with warnings.catch_warnings():
@@ -380,10 +387,34 @@
"wavelength": 2,
"resolution": 1000,
})
+ assert data.attrs["units"] == exp_units
assert data.chunks == ((1100,) * 10, (1100,) * 10)
assert data.dtype == data.compute().dtype
assert data.dtype == np.float32
+ # def test_read_ir_band_from_actual_file(self, hsd_file_jp01):
+ # """Test read_bands on real data."""
+ # filename_info = {"segment": 1, "total_segments": 1}
+ # filetype_info = {"file_type": "blahB14"}
+ # fh = AHIHSDFileHandler(hsd_file_jp01, filename_info, filetype_info)
+ # key = {"name": "B14", "calibration": "counts", "resolution": 1000}
+ # import dask
+ # with dask.config.set({"array.chunk-size": "32MiB"}):
+ # with warnings.catch_warnings():
+ # # The header isn't valid
+ # warnings.filterwarnings("ignore", category=UserWarning, message=r"Actual .* header size")
+ # data = fh.read_band(
+ # key,
+ # {
+ # "units": "%",
+ # "standard_name": "toa_bidirectional_reflectance",
+ # "wavelength": 2,
+ # "resolution": 1000,
+ # })
+ # assert data.chunks == ((1100,) * 10, (1100,) * 10)
+ # assert data.dtype == data.compute().dtype
+ # assert data.dtype == np.float32
+
@mock.patch("satpy.readers.ahi_hsd.AHIHSDFileHandler._read_data")
@mock.patch("satpy.readers.ahi_hsd.AHIHSDFileHandler._mask_invalid")
@mock.patch("satpy.readers.ahi_hsd.AHIHSDFileHandler.calibrate")
|
Describe the bug
Brightness temperature data should be returned with the
float32
dtype, but for theahi_hsd
reader BTs are returned asfloat64
.I suspect this is a
numpy=2.0
issue with some coefficient used to convert radiance -> BT being a float64.To Reproduce
Expected behavior
The print statement to show
dtype('float32')
Actual results
The print statement shows
dtype('float64')
Environment Info:
main
The text was updated successfully, but these errors were encountered: