220 config file csv - initial chem species are moved to CSV, and also specified with JSON data #299

carl-drews · 2025-01-02T21:28:59Z

my_config.json will now initialize chemical species like this JSON specifying one or more CSV files:

  "initial conditions": {
    "filepaths": ["initial_conditions.csv"],
    "data": [
      ["ENV.temperature [K]", "ENV.pressure [Pa]", "CONC.A [mol m-3]", "CONC.B [mol m-3]"],
      [200, 70000, 0.8, 0.2]
    ]
  },

Evolving conditions are also specified in the same manner:

  "evolving conditions": {
    "filepaths": [
      "conditions_Boulder.csv"
    ]
  },

The implementation provides warnings when a later condition overrides an earlier one.

…ia initial conditions.

…th JSON values.

…CSV files.

…ON configuration.

…ple lists.

…list on the basis of time.

K20shores · 2025-01-07T15:55:04Z

src/acom_music_box/conditions.py

+    @classmethod
+    def retrieve_initial_conditions_from_JSON(
+            self,
+            path_to_json,
+            json_object,
+            reaction_types):


Suggested change

@classmethod

def retrieve_initial_conditions_from_JSON(

self,

path_to_json,

json_object,

reaction_types):

@classmethod

def retrieve_initial_conditions_from_JSON(

cls,

path_to_json,

json_object,

reaction_types):

I only recently learned about classmethod and staticmethod. But, apparently it's common in python to use cls instead of self for classmethods.

K20shores · 2025-01-07T15:59:19Z

src/acom_music_box/conditions.py

+        for header, value in zip(header_row, value_row):
+            # are we looking for this type?
+            if (react_types):
+                header_type = header.split('.')[0]
+                if (header_type not in react_types):
+                    continue
+
+            data_values[header] = float(value)


Suggested change

for header, value in zip(header_row, value_row):

# are we looking for this type?

if (react_types):

header_type = header.split('.')[0]

if (header_type not in react_types):

continue

data_values[header] = float(value)

df = pd.DataFrame(value_row, columns=header_row)

I think pandas may be able to handle this. In general I think we should prefer to push parsing into pandas as much as possible

K20shores · 2025-01-07T16:01:30Z

src/acom_music_box/examples/configs/carbon_bond_5/my_config.json

+        "filepaths": [
+            "initial_reaction_rates.csv",
+            "initial_concentrations.csv"
+        ]


Love that we can do this now!

K20shores · 2025-01-07T16:04:52Z

src/acom_music_box/examples/configs/ts1/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 60.0,
    "simulation length [hour]": 1.0
  },
-  "chemical species": {
+  "chemical species deprecated": {


Suggested change

"chemical species deprecated": {

If we are doing to deprecate the thing, I think we should remove it entirely and we will update music box interactive accordingly to work with this new configuration format. We only have one active user whose configurations we will need to update.

K20shores · 2025-01-07T16:05:12Z

tests/integration/configs/mixing_ratio/mol mol-1/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 1,
    "simulation length [sec]": 100
  },
-  "chemical species": {
+  "chemical species deprecated": {


same as above

K20shores · 2025-01-07T16:05:16Z

tests/integration/configs/mixing_ratio/ppb/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 1,
    "simulation length [sec]": 100
  },
-  "chemical species": {
+  "chemical species deprecated": {


same as above

K20shores · 2025-01-07T16:05:20Z

tests/integration/configs/mixing_ratio/ppm/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 1,
    "simulation length [sec]": 100
  },
-  "chemical species": {
+  "chemical species deprecated": {


same as above

K20shores · 2025-01-07T16:05:24Z

tests/integration/configs/mixing_ratio/ppt/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 1,
    "simulation length [sec]": 100
  },
-  "chemical species": {
+  "chemical species deprecated": {


same as above

K20shores · 2025-01-07T16:05:28Z

tests/integration/configs/mixing_ratio/ppth/my_config.json

@@ -6,7 +6,7 @@
    "output time step [sec]": 1,
    "simulation length [sec]": 100
  },
-  "chemical species": {
+  "chemical species deprecated": {


same as above

K20shores · 2025-01-07T16:10:05Z

tests/integration/test_analytical.py

+            C_conc += initial_A * (
                1.0
                + (k1 * math.exp(-k2 * curr_time) - k2 * math.exp(-k1 * curr_time))
                / (k2 - k1)
-            )
+            ) + initial_B
+
+            if (curr_time >= last_out_time + out_time_step):
+                analytical_concentrations.append([A_conc, B_conc, C_conc])
+                last_out_time += out_time_step

-            analytical_concentrations.append([A_conc, B_conc, C_conc])
-            curr_time += time_step
+            curr_time += chem_time_step


This seems to be accumulating rather than transforming the mass. I don't think that is correct, and I don't understand the new time stuff. Can you explain that a bit more?

mattldawson

looks good! just a few questions and suggestions

mattldawson · 2025-01-08T16:54:30Z

src/acom_music_box/conditions.py

+        species_concentrations = Conditions.retrieve_initial_conditions_from_JSON(
+            path_to_json, json_object, {"ENV", "CONC"})
+        reaction_rates = Conditions.retrieve_initial_conditions_from_JSON(
+            path_to_json, json_object, {"EMIS", "PHOTO", "LOSS"})


I would separate environmental conditions from species concentrations here, just for clarity

mattldawson · 2025-01-08T16:57:15Z

src/acom_music_box/examples/configs/flow_tube/my_config.json

+        "simulation length [sec]": 100.0,
+        "real simulation length [hr]": 1.0


What is the difference between simulation length and real simulation length?

mattldawson · 2025-01-08T20:20:34Z

tests/integration/test_analytical.py

            A_conc = initial_A * math.exp(-(k1) * curr_time)
            B_conc = (
                initial_A
                * (k1 / (k2 - k1))
                * (math.exp(-k1 * curr_time) - math.exp(-k2 * curr_time))
            )
-            C_conc = initial_A * (
+            C_conc += initial_A * (


Suggested change

C_conc += initial_A * (

C_conc = initial_A * (

The analytical examples can be calculated from the initial concentrations and the current time, so no need to accumulate the values.

mattldawson · 2025-01-08T20:21:30Z

tests/integration/test_analytical.py

                1.0
                + (k1 * math.exp(-k2 * curr_time) - k2 * math.exp(-k1 * curr_time))
                / (k2 - k1)
-            )
+            ) + initial_B


why was this added?

mattldawson · 2025-01-08T20:21:46Z

tests/integration/test_mixing_ratios.py

            A_conc = initial_A * math.exp(-(k1) * curr_time)
            B_conc = (
                initial_A
                * (k1 / (k2 - k1))
                * (math.exp(-k1 * curr_time) - math.exp(-k2 * curr_time))
            )
-            C_conc = initial_A * (
+            C_conc += initial_A * (


Suggested change

C_conc += initial_A * (

C_conc = initial_A * (

mattldawson · 2025-01-08T20:21:54Z

tests/integration/test_mixing_ratios.py

                1.0
                + (k1 * math.exp(-k2 * curr_time) - k2 * math.exp(-k1 * curr_time))
                / (k2 - k1)
-            )
+            ) + initial_B


same question here

carl-drews added 30 commits December 9, 2024 16:56

Adding initial conditions as CSV or JSON.

8f9ffb2

Handle the case where chem_step != output_step != 1.

945b04c

initial conditions now has filepath.

98a6c8d

New JSON format for initial conditions CSV.

98d83b2

Added retrieval of command-line args musicaDir and template.

5decc90

Avoid overwriting TS1 input with WACCM test output.

798482f

Headers in CSV file need reactionType prefix.

c40a670

Misleading typo in comment.

f0164da

Changed values to be more suitable for testing.

9486ca8

Reclassified alleged Warnings to be Info or Debug message.

77eaaf5

Number density is calculated from pressure and temperature, not set v…

8596984

…ia initial conditions.

Removed chemical species section. Better testing values.

47a143e

Support multiple initial_conditions.csv files. Override CSV values wi…

6bc3449

…th JSON values.

New input configuration for intial conditions in CSV files.

35da4bf

Added LOSS to reactions.

3175b5b

Species names differ in format from reaction names. Support multiple …

4dc4b84

…CSV files.

Initial check-in of CSV initial concentrations.

05302da

Initial species concentrations moved from JSON to CSV file.

9133563

Allow space after comma for human readability.

fe0a335

Allow time steps != 1 second. Take initial concentrations from the JS…

ec2fc5a

…ON configuration.

Converted JSON intial species concentrations JSON to CSV.

dbff66e

Converted JSON species concentrations to CSV.

81f75aa

Removed unnecessary debug logging.

c4c3dd4

Give warning when one CSV file overrides a previous one.

a38a3b9

filepath is now plural: filepaths.

00270fe

Removed deprecated section chemical species from JSON.

5a668a6

There is no longer a chemical species section in my_config.json.

897fa90

Read multiple CSV evolution conditions. Still need to merge the multi…

06a7955

…ple lists.

Keep evolving conditions sorted in order of time. Merge the override …

99ecd9a

…list on the basis of time.

Corrected call to non-implemented member evolvingConditions.

c176728

carl-drews added 2 commits January 2, 2025 14:07

Added warning for overrides when multiple files of evolving conditions.

7cad82f

Allow multiple files of evolving conditions.

3d6a202

carl-drews requested review from mattldawson, K20shores, boulderdaze and montythind January 2, 2025 21:28

K20shores requested changes Jan 7, 2025

View reviewed changes

mattldawson requested changes Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

220 config file csv - initial chem species are moved to CSV, and also specified with JSON data #299

220 config file csv - initial chem species are moved to CSV, and also specified with JSON data #299

carl-drews commented Jan 2, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

K20shores Jan 7, 2025

mattldawson left a comment

mattldawson Jan 8, 2025

mattldawson Jan 8, 2025

mattldawson Jan 8, 2025

mattldawson Jan 8, 2025

mattldawson Jan 8, 2025

mattldawson Jan 8, 2025

		"simulation length [sec]": 100.0,
		"real simulation length [hr]": 1.0

220 config file csv - initial chem species are moved to CSV, and also specified with JSON data #299

Are you sure you want to change the base?

220 config file csv - initial chem species are moved to CSV, and also specified with JSON data #299

Conversation

carl-drews commented Jan 2, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattldawson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment