Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft: Train neural network for roman pots momentum reconstruction #10

Draft
wants to merge 31 commits into
base: master
Choose a base branch
from
Draft

Draft: Train neural network for roman pots momentum reconstruction #10

wants to merge 31 commits into from

Commits on Jan 10, 2024

  1. Add initial script for training neural network for roman pots momentu… 

    …m reconstruction. Function create a parametried dense neural network and train it with standardized input data.
    @rahmans1
    rahmans1 committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    4a5c335 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. Simplify layer description code using nn.ModuleList

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    77a117d View commit details
    Browse the repository at this point in the history

  2. Calculate the sample mean and standard deviation to pass on as standa… 

    …rdization parameters during inference
    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    1a5d06e View commit details
    Browse the repository at this point in the history

  3. Use number of epochs and learning rate as training hyper parameters

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    7398a0b View commit details
    Browse the repository at this point in the history

  4. Update number format and precision in training progress message

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    c4e1703 View commit details
    Browse the repository at this point in the history

  5. Functions to run experiments with parametrize inputs to the neural ne… 

    …twork models
    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    45ffea0 View commit details
    Browse the repository at this point in the history

  6. Import argparse and sys python modules

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    92bc46e View commit details
    Browse the repository at this point in the history

  7. Fix typo in call to add_argument function

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    5ad5a79 View commit details
    Browse the repository at this point in the history

  8. Fix typo. Missing '--' in list of arguments.

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    e1b0786 View commit details
    Browse the repository at this point in the history

  9. Input file locations will be passed along with list of hyperparameters

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    c4310f3 View commit details
    Browse the repository at this point in the history

  10. Fetch the number of training inputs from hyperparameter list

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    6b7dae7 View commit details
    Browse the repository at this point in the history

  11. Fix typo in variable name multiplier

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    34eee45 View commit details
    Browse the repository at this point in the history

  12. Cast hyperparameters to the appropriate numerical datatypes

    @rahmans1
    rahmans1 authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    2e25da5 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2024

  1. First commit of snakefile for roman pots neural network training. Gen… 

    …erate model files and assign unique names based on sha512 hash of concatenated hyperparamater values.
    @rahmans1
    rahmans1 committed Jan 19, 2024
    Configuration menu
    Copy the full SHA
    65310c7 View commit details
    Browse the repository at this point in the history

  2. Unhash the detector config and detector version. Only hash the parame… 

    …ter string.
    @rahmans1
    rahmans1 committed Jan 19, 2024
    Configuration menu
    Copy the full SHA
    130ff48 View commit details
    Browse the repository at this point in the history

  3. Parametrise subsystem and model type for readability

    @rahmans1
    rahmans1 committed Jan 19, 2024
    Configuration menu
    Copy the full SHA
    08fa4c5 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2024

  1. Truncate the hash to first 6 hexdigits. 6 hexdigits is equivalent to … 

    …6*4=24 bits. The number of unique hashes that we can generate before the probability of hash collision between any 2 model hashes reaches 50% is sqrt(2*0.5*2^(24))=4096. We are unlikely to run training over hyperparameter space that large on a single iteration.
    @rahmans1
    rahmans1 committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    2cbb87e View commit details
    Browse the repository at this point in the history

  2. Introduce steering file to simulate events for model training purposes

    @rahmans1
    rahmans1 authored Jan 22, 2024
    Configuration menu
    Copy the full SHA
    81bf1d5 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2024

  1. Add rule to generate training events. Use DETECTOR_VERSION inferred f… 

    …rom environment for creating directory structure
    @rahmans1
    rahmans1 committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    1e66076 View commit details
    Browse the repository at this point in the history

  2. Use epic_ip6 geometry for faster processing of far-forward events

    @rahmans1
    rahmans1 committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    570f99d View commit details
    Browse the repository at this point in the history

  3. Add rule to extract hit information necessary for model training

    @rahmans1
    rahmans1 committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    dd7e61b View commit details
    Browse the repository at this point in the history

  4. Use realistic hyperparameter values

    @rahmans1
    rahmans1 committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    ec5d176 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. Add rule to train network and save models and artifacts.

    @rahmans1
    rahmans1 committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    d89bae8 View commit details
    Browse the repository at this point in the history

  2. Unnecessary because directory structure is automatically created by rule

    @rahmans1
    rahmans1 committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    f34159a View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. Variable is an array with one element. So, indexing needs to be used

    @rahmans1
    rahmans1 committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    d81ab52 View commit details
    Browse the repository at this point in the history

  2. Split up default target rule into 3 rules so that proper dependency i… 

    …s obeyed. Execute snakemake all1/2/3 in sequence to avoid errors
    @rahmans1
    rahmans1 committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    805c2cd View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2024

  1. Simplify workflow. Build dependency between generate and process even… 

    …ts rules. Parallelization at generation stage through use of wildcards.
    @rahmans1
    rahmans1 committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    2d1a65e View commit details
    Browse the repository at this point in the history

  2. Use a nested directory approach to uniquely identify models instead o… 

    …f hash following https://waterdata.usgs.gov/blog/snakemake-for-ml-experiments/. Establishes a DAG with greater parallelization of processes.
    @rahmans1
    rahmans1 committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    acbb6c5 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. Changed Snakefile and preprocess_model_training_data.cxx to include t… 

    …he Low Pt model training
    David Ruth committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    5496e3c View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. Extract detector version

    @rahmans1
    rahmans1 authored Jun 4, 2024
    Configuration menu
    Copy the full SHA
    b0c5198 View commit details
    Browse the repository at this point in the history

  2. The default position of the planes updated to reflect current locatio… 

    …ns in epic
    @rahmans1
    rahmans1 authored Jun 4, 2024
    Configuration menu
    Copy the full SHA
    a695c51 View commit details
    Browse the repository at this point in the history