Rastervision multiple raster source types and multiple raster files

[mmcs-work]

Following this example here:
if we have multiple raster files how can we create the train_ds train dataset. I am trying to use rastervision as a library following the example.

train_image_uri = [ file1, file2]
train_label_uri   = [label1, label2]

1. What's the way to build dataset in such scenarios?

2. Also, if there are scenarios, where we have sentinel-1 data(2 channels) and sentinel-2 data (3 channels and it has difference scale of numbers from sentinel-1) and NDVI ([-1,+1] value ranges), what is the best way to include them in the dataset so it all comes up as (2+3+1 = ) 6 channel tensor finally with normalized values as per their own data (NDVI normalization is different than sentinel-1 normalization etc) ?

[AdeelH]

In Raster Vision, this is accomplished via a MultiRasterSource. The GeoDataset.from_uris() convenience constructor does not support this, so you'll have to do it the long way.

For normalization, see StatsTransformer tutorial.

Example:

stats_tf_s1 = ... # see the StatsTransformer tutorial
stats_tf_s2 = ... # see the StatsTransformer tutorial

rs_sentinel_1 = RasterioSource(s1_uri, raster_transformers=[stats_tf_s1])
rs_sentinel_2 = RasterioSource(s2_uri, raster_transformers=[stats_tf_s2])
rs_ndvi = RasterioSource(
    ndvi_uri, 
    # just an example, you can also write a custom transformer
    # see https://github.com/azavea/raster-vision/tree/master/rastervision_core/rastervision/core/data/raster_transformer
    raster_transformers=[MinMaxTransformer()],
)

raster_sources = [
    rs_seninel_1,
    rs_sentinel_2,
    rs_ndvi,
]
raster_source_multi = MultiRasterSource(raster_sources=raster_sources)

You will then need to create a LabelSource and a Scene. E.g. if your labels are GeoTIFFs (if instead they are GeoJSONs, see the linked tutorial):

label_raster_source = RasterioSource(label_uri)
label_source = SemanticSegmentationLabelSource(
    label_raster_source, class_config=class_config)
scene = Scene(id='my_scene', raster_source=raster_source, label_source=label_source)

And then finally, create a GeoDataset e.g.:

ds = SemanticSegmentationSlidingWindowGeoDataset(
    scene,
    size=256,
    stride=256,
    # ... etc.
)

[mmcs-work]

Thanks @AdeelH . One more point: since we have multiple files (file1, file2 and label1, label2),

train_image_uri = [ file1, file2]
train_label_uri   = [label1, label2]

Do we create separate SemanticSegmentationSlidingWindowGeoDataset for all of these file,label pair and then concatenate them? What's the best rastervision way of doing it?

[AdeelH]

Sorry, I missed that part of your question earlier. Yes, in that case, you should build one dataset for each image-label pair and then concatenate them using torch.utils.data.ConcatDataset.

An alternative workflow is to mosaic multiple images images together by passing multiple URIs to RasterioSource and merge multiple GeoJSON label files together by passing multiple URIs to GeoJSONVectorSource. In this case, you could then create just a single dataset. But this is not a common workflow and there might be bugs.