MemoryError: Unable to allocate 8.45 GiB for an array with shape (300, 300, 12600) and data type float64

[geokefe]

I'm trying to create velocity model of small size; I keep getting Memory error, even after reducing the model size significantly. Any suggestion about how to resolve this would be greatly appreciated. Here is the error message:

---

MemoryError Traceback (most recent call last)
Cell In[13], line 69
67 # Initialize and build the complex velocity model
68 velocity_model = ComplexVelocityModel(config)
---> 69 model_data = velocity_model.build_model()
71 # Plot the velocity model
72 velocity_model.plot_velocity_model()

Cell In[13], line 18, in ComplexVelocityModel.build_model(self)
15 os.makedirs(self.cfg['model_folder'], exist_ok=True)
16 os.makedirs(self.cfg['temp_folder'], exist_ok=True)
---> 18 self.model = mn.build_model(user_json=self.cfg['config_file'], run_id=self.cfg['run_id'])
20 return self.model

File ~\Desktop\seis\high_resolution_ML\synthoseis-master\main.py:31, in build_model(user_json, run_id, test_mode, rpm_factors)
29 # Build un-faulted geological models
30 geo_models = Geomodel(p, depth_maps, onlap_list, facies)
---> 31 geo_models.build_unfaulted_geomodels()
33 # Build Faults
34 f = Faults(p, depth_maps, onlap_list, geo_models, fan_list, fan_thicknesses)

File ~\Desktop\seis\high_resolution_ML\synthoseis-master\datagenerator\Geomodels.py:111, in Geomodel.build_unfaulted_geomodels(self)
86 def build_unfaulted_geomodels(self):
87 """
88 Build unfaulted geomodels.
89 --------------------------
(...)
109 None
110 """
--> 111 self.geologic_age[:] = self.create_geologic_age_3d_from_infilled_horizons(
112 self.depth_maps
113 )
114 self.onlap_segments[:] = self.insert_onlap_surfaces()

File ~\Desktop\seis\high_resolution_ML\synthoseis-master\datagenerator\Geomodels.py:196, in Geomodel.create_geologic_age_3d_from_infilled_horizons(self, depth_maps, verbose)
194 # create geologic age cube
195 age_range = np.linspace(0.0, float(cube_shape[2] - 1), cube_shape[2])
--> 196 age = np.zeros(cube_shape, "float")
197 for i in range(cube_shape[0]):
198 for j in range(cube_shape[1]):

MemoryError: Unable to allocate 8.45 GiB for an array with shape (300, 300, 12600) and data type float64

[gavargas22]

It looks like there is not enough memory in your machine, ultimately we need to convert the whole system to use zarr and so it would not use RAM memory.

An ongoing effort is here #12. In the meantime, we could try to reduce the bitsize of the numpy arrays to float16. It looks like we have float64 by default. Let me have a look to see how we might be able to change that.

Essentially, we need to set the dtype whenever there is a numpy array being defined np.array(list, dtype="float32")

If you have access to any other machine with larger RAM capacity, that would also resolve the issue for now.

[geokefe]

Correct me if I'm wrong, I think the reduction of bit size should be done from the source code, not in the application. Please keep me posted as the development progresses. Thanks.

[gavargas22]

That's correct @geokefe we can look into doing that. In the meantime, I would say reduce the size of the output volume so that it can fit in your memory