Network-based interactions (#24)

* Bugfix: Set the phase duration for initially infected individuals so they are not permanently infected

* Add initial household networks generation and interaction

* WIP: initial home/network split, reference model doesn't behave as expected, need to dig more/ rething things

* Implement per individual file and demo plotting script

This is useful for double checking household and workplace sizes

* Switch to GCC 8 compatible inclusive_scan

* fixup: port another std::inclusive_scan

* Implement GCC 8 compatible std::recduce(first, last, init)

* Tests: tests for GCC 8 compatible reduce

* Refactor GCC 8 compatible inclusive_scan methods to be more like C++17 stdlib

* fixup lint

* Visualisation: Group agents by household, reduce impact on non-vis builds

* Performance: Switch to bucket messaging to improve performance

The maximum bucket index will need changing in future implementations

Due to flamegpu limitations, the max value will have to be a worst-case value, i.e. the number of individuals rather than the actual number of households

For individual simualtions with some restructuring we could make this the correct value, but for ensembles it would not be possible, as the maximum value is shared for all instances of the ensemble

* Tests: expand util testing

* Add missing struct members to cli::print

* wip: refactoring

* Refactor: Move per-agent visualisation variable initialisation to visualisation.h/cu

* Refactor: Change how household/populations are generated and add tests

This will simplify future household genration improvement, improves legibility and adds some tests

* Fixup: fixup vis build after refactor

* Readme: remove rogue backtick

* Readme: Markdown fix

* Wip: random network

* Fix initial id-based pair-wise random interactions

* Tests: Fix std::TestPopulation.generateHouseholdStructures for older gcc

* Random Interactions: Initiial fully functional random daily interactions

- Per day generation of random interactions via host layer function(s)
- Target number per individual generated once at start of simulation, from a normal distribution from model parameters
- Not all target amounts will lead to valid interaction combinations, leading to less actual interactions than intended
- Does not exclude self interactions yet
- Does not exclude repeat interactions on a given day (but this would increase the likelihood of exposure)
- Implemented in serial on the CPU - will not scale well, but GPU implementation non-trivial.
- Possible encountered a FLAME GPU 2 bug / limitation, tobe investigated and reported
- Not implemented binomeal sampling (yet)
- Added CMAKE configuraiton option for the maximum number
- Could improve perf by changing this implementation, to be investigated

* sample data file fixup - accidentally commeted error triggering entry

* Random interaction workaround based on flamegpu version

* Fixup: Remove debuguing exit statement realted to flamegpu bug

* Small world network implementation. WIP commit prior to FLAMEGPU_INIT_FUNC refactor

* Refactoring, small world networks & workarounds

- Finishes implementing small world networks and uses the resulting graph for interactions within the workplace
  - Includes several new parameters to influence workplace network generation
  - Now (and in prev commits) 5 workplace networks
- Refactors code, including the graph data structure once again. More refactoring to come.
- Expands test suite
- Switches to agent generation in an init function, but workarounds are required due to bugs in flame gpu 2, of which a PR is already in place to address at least one of.
- Vis adjustments

CMakeLists.txt

@@ -2,7 +2,7 @@
 cmake_minimum_required(VERSION 3.18...3.25 FATAL_ERROR)
 
 # Optionally set the version of flamegpu which should be used, ideally a tag (i.e. `v2.0.0-rc`) or branch name, or potentially a commit hash.
-set(FLAMEGPU_VERSION "v2.0.0-rc.1" CACHE STRING "FLAMEGPU/FLAMEGPU2 git branch or tag to use")
+set(FLAMEGPU_VERSION "v2.0.0-rc.2" CACHE STRING "FLAMEGPU/FLAMEGPU2 git branch or tag to use")
 # If the above version is a hash instead, also set FLAMEGPU_VERSION_ALLOW_HASH to ON
 # set(FLAMEGPU_VERSION_ALLOW_HASH "ON")
 
@@ -29,6 +29,11 @@ option(BUILD_TESTING "Build the testing tree." OFF)
 # Option to enable google test test discovery
 cmake_dependent_option(ENABLE_GTEST_DISCOVER "Enable GTEST_DISCOVER for more detailed ctest output without -VV. This dramatically increases test suite runtime to CUDA context initialisation." OFF "BUILD_TESTING" OFF)
 
+# CMAke Cache option for the maximum number of per-agent random interactions.
+# If the value is changed here, CMakeCache.txt / the build dir will need deleting
+# Or reconfigure with -DEXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS=<new>
+set(EXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS "20" CACHE STRING "Maximum number of random daily interactions per agent")
+
 # Include common rules from the FLAMEGPU/FLAMEGPU2 repositories CMake
 include(${FLAMEGPU_ROOT}/cmake/common.cmake)
 

data/inputs/sample.csv

@@ -1,4 +1,7 @@
-rng_seed,param_id,duration,n_total,population_0_9,population_10_19,population_20_29,population_30_39,population_40_49,population_50_59,population_60_69,population_70_79,population_80,n_seed_infection,p_interaction_susceptible_to_exposed,mean_time_to_infected,sd_time_to_infected,mean_time_to_recovered,sd_time_to_recovered,mean_time_to_susceptible,sd_time_to_susceptible,relative_susceptibility_0_9,relative_susceptibility_10_19,relative_susceptibility_20_29,relative_susceptibility_30_39,relative_susceptibility_40_49,relative_susceptibility_50_59,relative_susceptibility_60_69,relative_susceptibility_70_79,relative_susceptibility_80
-12,0,365,16384,4,4,4,4,4,4,3,2,1,8,0.10,3.5,1.0,7.0,1.0,60,5.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
-12,0,365,16384,4,4,4,4,4,4,4,4,4,8,0.10,3.5,1.0,7.0,1.0,180,5.0,0.0,0.0,0.0,0.01,0.05,0.01,0.05,1.0,5.0
-
+rng_seed,param_id,duration,n_total,n_seed_infection,population_0_9,population_10_19,population_20_29,population_30_39,population_40_49,population_50_59,population_60_69,population_70_79,population_80,household_size_1,household_size_2,household_size_3,household_size_4,household_size_5,household_size_6,p_interaction_susceptible_to_exposed,mean_time_to_infected,sd_time_to_infected,mean_time_to_recovered,sd_time_to_recovered,mean_time_to_susceptible,sd_time_to_susceptible,relative_susceptibility_0_9,relative_susceptibility_10_19,relative_susceptibility_20_29,relative_susceptibility_30_39,relative_susceptibility_40_49,relative_susceptibility_50_59,relative_susceptibility_60_69,relative_susceptibility_70_79,relative_susceptibility_80,child_network_adults,elderly_network_adults,relative_transmission_household,relative_transmission_occupation,relative_transmission_random,mean_work_interactions_child,mean_work_interactions_adult,mean_work_interactions_elderly,daily_fraction_work,work_network_rewire,mean_random_interactions_0_19,sd_random_interactions_0_19,mean_random_interactions_20_69,sd_random_interactions_20_69,mean_random_interactions_70plus,sd_random_interactions_70plus
+12,0,60,32,4,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3
+12,1,365,32,4,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3
+12,2,365,1024,8,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3
+12,3,365,4096,8,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3
+12,4,365,4096,8,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3
+12,5,365,65536,8,331,370,374,371,361,415,377,270,142,442,492,208,165,51,27,0.10,3.5,2.0,7.0,2.0,28,10.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.2,0.2,2.0,0.5,0.5,10,7,3,0.5,0.1,2,2,4,4,3,3

readme.md

@@ -2,7 +2,7 @@
 
 A GPU accelerated Agent-Based Model (ABM) of infection disease spread within a population.
 
-Implemented using [FLAMEGPU/FLAMEGPU2](https://github.com/FLAMEGPU/FLAMEGPU2), with model functionality inspired by [https://github.com/BDI-pathogens/OpenABM-Covid19](BDI-pathogens/OpenABM-Covid19).
+Implemented using [FLAMEGPU/FLAMEGPU2](https://github.com/FLAMEGPU/FLAMEGPU2), with model functionality inspired by [BDI-pathogens/OpenABM-Covid19](https://github.com/BDI-pathogens/OpenABM-Covid19).
 
 ## Requirements / Dependencies
 
@@ -51,6 +51,7 @@ cmake --build . --target exatepp_abm -j 8
 | `CMAKE_CUDA_ARCHITECTURES`        | `"50;60;70;80;90"` | ` Specify which CUDA [Compute Capability](https://developer.nvidia.com/cuda-gpus) Architectures to build for |
 | `CMAKE_BUILD_TYPE`                | `Release` | CMake build configuration, setting optimisation levels etc. Choose from [`Release`, `RelWithDebInfo`, `MinSizeRel`, `Debug`] |
 | `BUILD_TESTING`                   | `OFF` | Enable / disable the test suite |
+| `EXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS` | `20` | The maximum number of per agent random daily interactions for the build. If this is exceeded due to model parameters an error will be reported, and you must reconfigure and rebuild with a higher value. | 
 | `FLAMEGPU_VISUALISATION`          | `OFF` | If FLAME GPU's 3D interactive visualisation should be enabled. Requires OpenGL and local execution. |
 | `FLAMEGPU_SEATBELTS`              | `ON` | Enable / Disable additional runtime checks which harm performance but increase usability |
 | `FLAMEGPU_SHARE_USAGE_STATISTICS` | `ON` | Enable / Disable FLAME GPU 2 telemetry which helps evidence use/impact of FLAME GPU 2. See the [FLAME GPU 2 user guide for more information](https://docs.flamegpu.com/guide/telemetry/) |
@@ -121,7 +122,7 @@ Source code linting required `cpplint` to be installed and on your path at CMake
 Linting can be initiated via cmake, i.e from a build directory:
 
 ```bash
-cmake --build . --target lint`
+cmake --build . --target lint
 ```
 
 ## Documentation

src/CMakeLists.txt

@@ -21,9 +21,11 @@ SET(LIBRARY_SRC
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/disease/SEIR.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/exatepp_abm.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/input.h
+    ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/network.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/OutputFile.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/PerformanceFile.h
+    ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/PerIndividualFile.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/TimeSeriesFile.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/person.h
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/population.h
@@ -35,9 +37,12 @@ SET(LIBRARY_SRC
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/disease/SEIR.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/exatepp_abm.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/input.cu
+    ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/network.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/PerformanceFile.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/TimeSeriesFile.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/PerIndividualFile.cu
+    ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/output/TimeSeriesFile.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/person.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/population.cu
     ${CMAKE_CURRENT_SOURCE_DIR}/exateppabm/util.cu
@@ -83,10 +88,16 @@ endif()
 target_compile_options(${LIBRARY_NAME} PUBLIC "$<$<COMPILE_LANGUAGE:CUDA>:--expt-relaxed-constexpr>")
 
 # Forward on the cmake build type as a macro definition, for performance output files. There's prolly a cleaner way to do this.
-target_compile_definitions(${LIBRARY_NAME} PRIVATE "$<$<CONFIG:Release>:CMAKE_BUILD_TYPE=\"Release\">")
-target_compile_definitions(${LIBRARY_NAME} PRIVATE "$<$<CONFIG:RelWithDebInfo>:CMAKE_BUILD_TYPE=\"RelWithDebInfo\">")
-target_compile_definitions(${LIBRARY_NAME} PRIVATE "$<$<CONFIG:MinSizeRel>:CMAKE_BUILD_TYPE=\"MinSizeRel\">")
-target_compile_definitions(${LIBRARY_NAME} PRIVATE "$<$<CONFIG:Debug>:CMAKE_BUILD_TYPE=\"Debug\">")
+target_compile_definitions(${LIBRARY_NAME} PUBLIC "$<$<CONFIG:Release>:CMAKE_BUILD_TYPE=\"Release\">")
+target_compile_definitions(${LIBRARY_NAME} PUBLIC "$<$<CONFIG:RelWithDebInfo>:CMAKE_BUILD_TYPE=\"RelWithDebInfo\">")
+target_compile_definitions(${LIBRARY_NAME} PUBLIC "$<$<CONFIG:MinSizeRel>:CMAKE_BUILD_TYPE=\"MinSizeRel\">")
+target_compile_definitions(${LIBRARY_NAME} PUBLIC "$<$<CONFIG:Debug>:CMAKE_BUILD_TYPE=\"Debug\">")
+
+# Set the upper limit of the per agent max random interaction count
+if(EXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS GREATER 0)
+    target_compile_definitions(${LIBRARY_NAME} PUBLIC "EXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS=${EXATEPP_ABM_MAX_RANDOM_DAILY_INTERACTIONS}")
+endif()
+
 
 # Add src directory to include path, publicly so that the target library inherits this dependency (for now).
 target_include_directories("${LIBRARY_NAME}" PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")

src/exateppabm/cli.cpp

@@ -13,16 +13,18 @@ void setup(CLI::App& app, std::shared_ptr<exateppabm::cli::params> params) {
     app.add_flag("-v, --verbose", params->verbosity, "Verbosity of simulation output, forwarded to FLAME GPU 2");
     app.add_option("-i, --input-file", params->inputParamFile, "Path to input parameters file");
     app.add_option("-n, --param-number", params->inputParamLine, "The line from the parameters file to use. 1 indexed (assuming there is a header)");
-
     app.add_option("-o, --output-dir", params->outputDir, "Path to output directory");
+    app.add_flag("--individual-file", params->individualFile, "Enable the creation of the per individual file");
 }
 
 void print(const exateppabm::cli::params params) {
     fmt::print("params {{\n");
     fmt::print("  device = {}\n", params.device);
     fmt::print("  verbosity = {}\n", params.verbosity);
     fmt::print("  inputParamFile = {}\n", params.inputParamFile);
+    fmt::print("  inputParamLine = {}\n", params.inputParamLine);
     fmt::print("  outputDir = {}\n", params.outputDir);
+    fmt::print("  individualFile = {}\n", params.individualFile);
     fmt::print("}}\n");
 }
 

src/exateppabm/cli.h

@@ -37,6 +37,10 @@ struct params {
      * Path to directory for file output
      */
     std::string outputDir = std::filesystem::current_path();
+    /**
+     * bool indicating if the individual file should be written
+     */
+    bool individualFile = false;
 };
 
 /**

src/exateppabm/demographics.cu

@@ -1,9 +1,11 @@
 #include "exateppabm/demographics.h"
 
+#include <array>
 #include <cstdint>
 
 #include "flamegpu/flamegpu.h"
 #include "exateppabm/input.h"
+#include "exateppabm/util.h"
 
 namespace exateppabm {
 namespace demographics {
@@ -27,5 +29,42 @@ void define(flamegpu::ModelDescription& model, const exateppabm::input::config&
     }, true);
 }
 
+std::array<demographics::Age, demographics::AGE_COUNT> getAllAgeDemographics() {
+    std::array<demographics::Age, demographics::AGE_COUNT> all = {{
+        demographics::Age::AGE_0_9,
+        demographics::Age::AGE_10_19,
+        demographics::Age::AGE_20_29,
+        demographics::Age::AGE_30_39,
+        demographics::Age::AGE_40_49,
+        demographics::Age::AGE_50_59,
+        demographics::Age::AGE_60_69,
+        demographics::Age::AGE_70_79,
+        demographics::Age::AGE_80
+    }};
+    return all;
+}
+
+std::array<float, demographics::AGE_COUNT> getAgeDemographicCumulativeProbabilityArray(const exateppabm::input::config& params) {
+    // Prepare a probability matrix for selecting an age demographic for the agent based on the ratio from the configuration.
+    // @todo - this could probably be cleaned up
+    std::uint64_t configDemographicSum = params.population_0_9 + params.population_10_19 + params.population_20_29 + params.population_30_39 + params.population_40_49 + params.population_50_59 + params.population_60_69 + params.population_70_79 + params.population_80;
+
+    std::array<float, demographics::AGE_COUNT> demographicProbabilties =  {{
+        params.population_0_9 / static_cast<float>(configDemographicSum),
+        params.population_10_19 / static_cast<float>(configDemographicSum),
+        params.population_20_29 / static_cast<float>(configDemographicSum),
+        params.population_30_39 / static_cast<float>(configDemographicSum),
+        params.population_40_49 / static_cast<float>(configDemographicSum),
+        params.population_50_59 / static_cast<float>(configDemographicSum),
+        params.population_60_69 / static_cast<float>(configDemographicSum),
+        params.population_70_79 / static_cast<float>(configDemographicSum),
+        params.population_80 / static_cast<float>(configDemographicSum)
+    }};
+    // Perform an inclusive scan to convert to cumulative probability
+    // Using a local method which supports inclusive scans in old libstc++
+    exateppabm::util::inclusive_scan(demographicProbabilties.begin(), demographicProbabilties.end(), demographicProbabilties.begin());
+    return demographicProbabilties;
+}
+
 }  // namespace demographics
 }  // namespace exateppabm

src/exateppabm/demographics.h

@@ -41,5 +41,20 @@ enum Age : AgeUnderlyingType {
  */
 void define(flamegpu::ModelDescription& model, const exateppabm::input::config& params);
 
+/**
+ * Get an array containing one of each age demographic enum
+ * 
+ * This is a workaround for the lack of reflection in c++17, used to simplify code elsewhere
+ * @return std::array which is the inverse of the Age enum.
+ */
+std::array<demographics::Age, demographics::AGE_COUNT> getAllAgeDemographics();
+
+/**
+ * Generate a cumulative probability distribution for age demographic sampling for a given simulation configuration
+ * @param params model configuration parameters
+ * @return per-age demographic cumulative probability, for sampling with a uniform distribution [0, 1)
+ */
+std::array<float, demographics::AGE_COUNT> getAgeDemographicCumulativeProbabilityArray(const exateppabm::input::config& params);
+
 }  // namespace demographics
 }  // namespace exateppabm

src/exateppabm/disease/SEIR.cu

@@ -94,9 +94,6 @@ void define(flamegpu::ModelDescription& model, const exateppabm::input::config&
     env.newProperty<float>("mean_time_to_susceptible", params.mean_time_to_susceptible);
     env.newProperty<float>("sd_time_to_susceptible", params.sd_time_to_susceptible);
 
-    // Define the temporary, hardcoded spatial infection interaction radius?
-    // env.setProperty<float>("INFECTION_INTERACTION_RADIUS", interactionRadius);
-
     // Get a handle for the Person agent type
     flamegpu::AgentDescription person = model.Agent(exateppabm::person::NAME);
 

src/exateppabm/exatepp_abm.cu

@@ -83,18 +83,19 @@ int entrypoint(int argc, char* argv[]) {
     flamegpu::ModelDescription model("ExaTEPP ABM demonstrator");
 
     // Add the person agent to the model description
-    const float env_width = std::ceil(std::sqrt(config->n_total));
-    constexpr float interactionRadius = 1.5f;
-    exateppabm::person::define(model, *config, env_width, interactionRadius);
+    exateppabm::person::define(model, *config);
 
     // Define demographic related variables
     exateppabm::demographics::define(model, *config);
 
     // Define disease related variables and methods
     exateppabm::disease::SEIR::define(model, *config);
 
+    // Define init function for population generation
+    exateppabm::population::define(model, *config, cli_params->verbosity > 0);
+
     // Add init, step and exit functions related to data collection and output. This may want refactoring when multiple output files are supported or collected data becomes more complex.
-    exateppabm::output::define(model, cli_params->outputDir);
+    exateppabm::output::define(model, cli_params->outputDir, cli_params->individualFile);
 
     // Build the model control flow. This will want abstracting more in the future @todo
     // @note - not using the DAG control flow due to bugs encountered in another project when splitting between compilation units.
@@ -113,6 +114,11 @@ int entrypoint(int argc, char* argv[]) {
 
     // Setup simulation configuration options
 
+    // If verbosity is high enough (-vvv or more) then enable flamegpu's verbose output
+    if (cli_params->verbosity > 2) {
+        simulation.SimulationConfig().verbosity = flamegpu::Verbosity::Verbose;
+    }
+
     simulation.SimulationConfig().steps = config->duration;  // @todo - change this to be controlled by an exit condition?
 
     // Seed the FLAME GPU 2 RNG seed. This is independent from RNG on the host, but we only have one RNG engine available in FLAME GPU 2 currently.
@@ -121,15 +127,6 @@ int entrypoint(int argc, char* argv[]) {
     // Set the GPU index
     simulation.CUDAConfig().device_id = cli_params->device;
 
-    // Generate the population of agents.
-    // @todo - this should probably be an in init function for ease of moving to a ensembles, but then cannot pass parameters in.
-    const std::uint64_t pop_seed = config->rng_seed;  // @todo - split seeds
-    auto personPopulation = exateppabm::population::generate(model, *config, cli_params->verbosity > 0, env_width, interactionRadius);
-    if (personPopulation == nullptr) {
-        throw std::runtime_error("@todo - bad population generation function.");
-    }
-    simulation.setPopulationData(*personPopulation);
-
     perfFile.timers.preSimulate.stop();
 
     // Run the simulation