Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add MOD06 support to 'modis_l2' reader #812

Merged
merged 7 commits into from
Jun 1, 2020
Merged

Add MOD06 support to 'modis_l2' reader #812

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
dd16a39
Fix get_dataset not respecting file_key tag for "no byte manipulation…
BENR0 Jun 11, 2019
9ff2ff2
Add further datasets from mod06 to yaml
BENR0 Jun 11, 2019
7214c6f
Add file type mod06 to lat/lon 1000m resolution
BENR0 Jun 11, 2019
1104918
Fix removed duplicate key in yaml
BENR0 Jun 11, 2019
3fa8168
Change dataset descriptions for mod06 files in modis level 2 reader yaml
BENR0 Sep 23, 2019
9d37baa
Update satpy/etc/readers/modis_l2.yaml
mraspaud May 12, 2020
f1c32cc
Fix modisl2 yaml file
mraspaud May 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 134 additions & 27 deletions satpy/etc/readers/modis_l2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ file_types:
file_patterns:
- 'M{platform_indicator:1s}D35_L2.A{acquisition_time:%Y%j.%H%M}.{collection:03d}.{production_time:%Y%j%H%M%S}.hdf'
file_reader: !!python/name:satpy.readers.modis_l2.ModisL2HDFFileHandler
mod06_hdf:
file_patterns:
- 'M{platform_indicator:1s}D06_L2.A{acquisition_time:%Y%j.%H%M}.{collection:03d}.{production_time:%Y%j%H%M%S}.hdf'
file_reader: !!python/name:satpy.readers.modis_l2.ModisL2HDFFileHandler
hdf_eos_geo:
file_patterns:
- 'M{platform_indicator:1s}D03_A{start_time:%y%j_%H%M%S}_{processing_time:%Y%j%H%M%S}.hdf'
Expand All @@ -20,36 +24,13 @@ file_types:
file_reader: !!python/name:satpy.readers.modis_l1b.HDFEOSGeoReader

datasets:
cloud_mask:
# byte Cloud_Mask(Byte_Segment, Cell_Along_Swath_1km, Cell_Across_Swath_1km)
name: cloud_mask
resolution: [1000, 250]
# The dimension of the dataset where the byte information is stored
byte_dimension: 0
# Different logic depending on the resolution
byte:
- 1000: 0
- 250: [4, 5]
bit_start:
- 1000: 1
- 250: 0
bit_count:
- 1000: 2
- 250: 1
# Quality Assurance flag is necessary for 250m resolution dataset
quality_assurance:
- 250: True
file_key: Cloud_Mask
file_type: mod35_hdf
coordinates: [longitude, latitude]

longitude:
name: longitude
resolution:
5000:
file_type: mod35_hdf
file_type: [mod35_hdf, mod06_hdf]
1000:
file_type: [hdf_eos_geo, mod35_hdf]
file_type: [hdf_eos_geo, mod35_hdf, mod06_hdf]
500:
file_type: hdf_eos_geo
250:
Expand All @@ -62,16 +43,42 @@ datasets:
resolution:
5000:
# For EUM reduced (thinned) files
file_type: mod35_hdf
file_type: [mod35_hdf, mod06_hdf]
1000:
file_type: [hdf_eos_geo, mod35_hdf]
file_type: [hdf_eos_geo, mod35_hdf, mod06_hdf]
500:
file_type: hdf_eos_geo
250:
file_type: hdf_eos_geo
standard_name: latitude
units: degree

##########################
#Datasets in file mod35_l2
##########################
cloud_mask:
# byte Cloud_Mask(Byte_Segment, Cell_Along_Swath_1km, Cell_Across_Swath_1km)
name: cloud_mask
resolution: [1000, 250]
# The dimension of the dataset where the byte information is stored
byte_dimension: 0
# Different logic depending on the resolution
byte:
- 1000: 0
- 250: [4, 5]
bit_start:
- 1000: 1
- 250: 0
bit_count:
- 1000: 2
- 250: 1
# Quality Assurance flag is necessary for 250m resolution dataset
quality_assurance:
- 250: True
file_key: Cloud_Mask
file_type: mod35_hdf
coordinates: [longitude, latitude]

quality_assurance:
# byte Quality_Assurance(Cell_Along_Swath_1km, Cell_Across_Swath_1km, QA_Dimension)
name: quality_assurance
Expand All @@ -86,3 +93,103 @@ datasets:
file_type: mod35_hdf
coordinates: [longitude, latitude]

##########################
#Datasets in file mod06_l2
##########################
cloud_fraction:
name: cloud_fraction
resolution: 5000
file_key: Cloud_Fraction
file_type: mod06_hdf
coordinates: [longitude, latitude]

brightness_temperature:
name: brightness_temperature
resolution: 5000
file_key: Brightness_Temperature
file_type: mod06_hdf
coordinates: [longitude, latitude]

surface_temperature:
name: surface_temperature
resolution: 5000
file_key: Surface_Temperature
file_type: mod06_hdf
coordinates: [longitude, latitude]

surface_temperature_1km:
name: surface_temperature_1km
resolution: 1000
file_key: surface_temperature_1km
file_type: mod06_hdf
coordinates: [longitude, latitude]

surface_pressure:
name: surface_pressure
resolution: 5000
file_key: Surface_Pressure
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_top_height:
name: cloud_top_height
resolution: 5000
file_key: Cloud_Top_Height
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_top_height_1km:
name: cloud_top_height_1km
resolution: 1000
file_key: cloud_top_height_1km
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_top_temperature:
name: cloud_top_temperature
resolution: 5000
file_key: Cloud_Top_Temperature
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_top_temperature_1km:
name: cloud_top_temperature_1km
resolution: 1000
file_key: cloud_top_temperature_1km
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_effective_emissivity:
name: cloud_effective_emissivity
resolution: 5000
file_key: Cloud_Effective_Emissivity
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_effective_emissivity_1km:
name: cloud_effective_emissivity_1km
resolution: 5000
file_key: cloud_emissivity_1km
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_effective_radius_1km:
name: cloud_effective_radius_1km
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this name have the resolution in it when the others don't?

Copy link
Collaborator Author

@BENR0 BENR0 Jun 11, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure right now if this is true for that specific dataset but some of them have 5km and 1km datasets in the file and are named that way then too. I will check up on that. Eventually if only 1km is present the 1km can be removed from the name.

Regarding the available_datasets: hadn't thought about that but I guess that would be nice since the reader then probably would support most, if not all, level 2 files without the need of updating the yaml file.
Due to preparations for a talk I have to give next week I am a little short on time so I don't have capacity to have a look at it right away. I wonder if for the time being it would be possible to merge the mod06 addition since we want to use those files in a course we are giving right now and it would be nice if the students could just use the module installed from the master instead of my branch (all of them are python beginners and they are struggling enough already with the topics covered)? I would then do another PR when I come up with a general solution using the available_datasets method.

resolution: 1000
file_key: Cloud_Effective_Radius
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_optical_thickness:
name: cloud_optical_thickness
resolution: 1000
file_key: Cloud_Optical_Thickness
file_type: mod06_hdf
coordinates: [longitude, latitude]

cloud_water_path:
name: cloud_water_path
resolution: 1000
file_key: Cloud_Water_Path
file_type: mod06_hdf
coordinates: [longitude, latitude]
2 changes: 1 addition & 1 deletion satpy/readers/modis_l2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ def get_dataset(self, dataset_id, dataset_info):

# No byte manipulation required
else:
dataset = self.load_dataset(dataset_name)
dataset = self.load_dataset(dataset_name_in_file)

return dataset

Expand Down