25.1.15rc0

Upcoming

Highlights

• New UI updates:

  • An Event Logs page for seeing the logs generated by your Pioreactors
  • A detailed overview of your cluster's leader-specific duties on the new Leader's page.
    • See the Leader's filesystem, logs, update cluster clocks, and view important running jobs.
  • View different Pioreactors' plugins on the Plugins page, and install to specific Pioreactor vs entire cluster.
  • Manage your calibrations from the UI's new Calibrations page.
    • View existing calibrations, set active calibrations, and download calibration files.

• New calibrations API. A calibration now creates a YAML file as an artifact, stored in ~/.pioreactor/calibrations. This makes editing, creating, sharing, and transferring calibrations much easier.

• There's also a new CLI for calibrations:

  Usage: pio calibrations [OPTIONS] COMMAND [ARGS]...
  
    interface for all calibration types.
  
  Options:
    --help  Show this message and exit.
  
  Commands:
    delete      Delete a calibration file from local storage.
    display     Display the contents of a calibration YAML file.
    list        List existing calibrations for the given device.
    run         Run an interactive calibration assistant for a specific device.
    set-active  Mark a specific calibration as 'active' for that device
    analyze     Analyze the data from a calibration.
  

  For example, to run a pump calibration, use pio calibrations run --device media_pump. View all your media pump calibrations with: pio calibrations list --device media_pump. And to duplicate a

• For now, the actual calibrations are the same protocol as before, but in the near future, we'll be updating them with new features. Adding this unified CLI and YAML format was the first step.

Web API changes

• New API to retrieve and set clocks on Pioreactors
  • GET /api/units/<pioreactor_unit>/system/utc_clock
  • GET /unit_api/system/utc_clock
  • POST /api/system/utc_clock
  • POST /unit_api/system/utc_clock
• New log APIs
  • GET /api/experiments/<experiment>/recent_logs
  • GET /api/experiments/<experiment>/logs
  • GET /api/logs
  • GET /api/workers/<pioreactor_unit>/experiments/<experiment>/recent_logs
  • GET /api/workers/<pioreactor_unit>/experiments/<experiment>/logs
  • GET /api/units/<pioreactor_unit>/logs
  • POST /workers/<pioreactor_unit>/experiments/<experiment>/logs
• New calibrations APIs
  • GET /api/workers/<pioreactor_unit>/calibrations
  • GET /unit_api/calibrations
  • GET /unit_api/active_calibrations
  • GET /api/workers/<pioreactor_unit>/calibrations/<device>
  • GET /unit_api/calibrations/<device>
  • PATCH /api/workers/<pioreactor_unit>/active_calibrations/<device>/<cal_name>
  • PATCH /unit_api/active_calibrations/<device>/<cal_name>
  • DELETE /api/workers/<pioreactor_unit>/active_calibrations/<device>/<cal_name>
  • DELETE /api/workers/<pioreactor_unit>/calibrations/<device>/<cal_name>
  • DELETE /unit_api/active_calibrations/<device>/<cal_name>
  • DELETE /unit_api/calibrations/<device>/<cal_name>
• New API for plugins
  • GET /api/units/<pioreactor_unit>/plugins/installed
  • PATCH /api/units/<pioreactor_unit>/plugins/install
  • PATCH /api/units/<pioreactor_unit>/plugins/uninstall
• Changed the settings API (see docs).
• New /api/units that returns a list of units (this is workers & leader). If leader is also a worker, then it's identical to /api/workers
• New /api/experiments/<experiment>/historical_worker_assignments that stores historical assignments to experiments
• New Path API for getting the dir structure of ~/.pioreactor:
  • /unit_api/system/path/<path>

Enhancements

• new SQL table for historical_experiment_assignments that stores historical assignments to experiments.
• UI performance improvements
• Better terminal plots

Breaking changes

• use_calibration under od_reading.config is deprecated. Use the calibrations "active" state instead.

• Note: by default, all calibrations are not active, even if they were "current" before. You must set them to be active.

• removed Python library diskcache.

• any stirring calibrations needs to be redone. On the command line, run pio calibration run --device stirring to start the calibration assistant.

• fixed typo utils.local_persistant_storage to utils.local_persistent_storage.

• Kalman Filter database table is no longer populated. There is a way to re-add it, lmk.

• moved intermittent cache location to /tmp/pioreactor_cache/local_intermittent_pioreactor_metadata.sqlite. This also determined by your configuration, see [storage].

• removed calibrations export dataset. Use the export option on the /Calibrations page instead.

• persistent storage is now on single sqlite3 database in /home/pioreactor/.pioreactor/storage/local_persistent_pioreactor_metadata.sqlite. This is configurable in your configuration.

• When checking for calibrations in custom Dosing automations, users may have added:

     with local_persistant_storage("current_pump_calibration") as cache:
         if "media" not in cache:
         ...

  This should be updated to (Note the spelling in local_persistant_storage changed, too!):

      with local_persistent_storage("active_calibrations") as cache:
          if "media_pump" not in cache:
          ...

• removed pioreactor.utils.gpio_helpers

Bug fixes

• Fix PWM3 not cleaning up correctly
• Fixed Stirring not updating to best DC % when using a calibration after changing target RPM
• Fixed a bug that could cause OD calibrations to map a small voltage value to a max OD.

25.1.14rc0

Highlights

• New UI updates:

  • An Event Logs page for seeing the logs generated by your Pioreactors
  • A detailed overview of your cluster's leader-specific duties on the new Leader's page.
    • See the Leader's filesystem, logs, update cluster clocks, and view important running jobs.

• View different Pioreactors' plugins on the Plugins page, and install to specific Pioreactor vs entire cluster.

• Manage your calibrations from the UI's new Calibrations page.

  • View existing calibrations, set active calibrations, and download calibration files.

• New calibrations API. A calibration now creates a YAML file as an artifact, stored in ~/.pioreactor/calibrations. This makes editing, creating, sharing, and transferring calibrations much easier.

• There's also a new CLI for calibrations:

  Usage: pio calibrations [OPTIONS] COMMAND [ARGS]...
  
    interface for all calibration types.
  
  Options:
    --help  Show this message and exit.
  
  Commands:
    delete      Delete a calibration file from local storage.
    display     Display the contents of a calibration YAML file.
    list        List existing calibrations for the given device.
    run         Run an interactive calibration assistant for a specific device.
    set-active  Mark a specific calibration as 'active' for that device
  

  For example, to run a pump calibration, use pio calibrations run --device media_pump. View all your media pump calibrations with: pio calibrations list --device media_pump. And to duplicate a

• For now, the actual calibrations are the same protocol as before, but in the near future, we'll be updating them with new features. Adding this unified CLI and YAML format was the first step.

Web API changes

• New API to retrieve and set clocks on Pioreactors
  • GET /api/units/<pioreactor_unit>/system/utc_clock
  • GET /unit_api/system/utc_clock
  • POST /api/system/utc_clock
  • POST /unit_api/system/utc_clock
• New log APIs
  • GET /api/experiments/<experiment>/recent_logs
  • GET /api/experiments/<experiment>/logs
  • GET /api/logs
  • GET /api/workers/<pioreactor_unit>/experiments/<experiment>/recent_logs
  • GET /api/workers/<pioreactor_unit>/experiments/<experiment>/logs
  • GET /api/units/<pioreactor_unit>/logs
  • POST /workers/<pioreactor_unit>/experiments/<experiment>/logs
• New calibrations APIs
  • GET /api/workers/<pioreactor_unit>/calibrations
  • GET /unit_api/calibrations
  • GET /unit_api/active_calibrations
  • GET /api/workers/<pioreactor_unit>/calibrations/<device>
  • GET /unit_api/calibrations/<device>
  • PATCH /api/workers/<pioreactor_unit>/active_calibrations/<device>/<cal_name>
  • PATCH /unit_api/active_calibrations/<device>/<cal_name>
  • DELETE /api/workers/<pioreactor_unit>/active_calibrations/<device>/<cal_name>
  • DELETE /api/workers/<pioreactor_unit>/calibrations/<device>/<cal_name>
  • DELETE /unit_api/active_calibrations/<device>/<cal_name>
  • DELETE /unit_api/calibrations/<device>/<cal_name>
• New API for plugins
  • GET /api/units/<pioreactor_unit>/plugins/installed
  • PATCH /api/units/<pioreactor_unit>/plugins/install
  • PATCH /api/units/<pioreactor_unit>/plugins/uninstall
• Changed the settings API (see docs).
• New /api/units that returns a list of units (this is workers & leader). If leader is also a worker, then it's identical to /api/workers
• New /api/experiments/<experiment>/historical_worker_assignments that stores historical assignments to experiments
• New Path API for getting the dir structure of ~/.pioreactor:
  • /unit_api/system/path/<path>

Enhancements

• new SQL table for historical_experiment_assignments that stores historical assignments to experiments.
• UI performance improvements

Breaking changes

• use_calibration under od_reading.config is deprecated. Use the calibrations "active" state instead.

• Note: by default, all calibrations are not active, even if they were "current" before. You must set them to be active.

• removed Python library diskcache.

• any stirring calibrations needs to be redone. On the command line, run pio calibration run --device stirring to start the calibration assistant.

• fixed typo utils.local_persistant_storage to utils.local_persistent_storage.

• Kalman Filter database table is no longer populated. There is a way to re-add it, lmk.

• moved intermittent cache location to /tmp/pioreactor_cache/local_intermittent_pioreactor_metadata.sqlite. This also determined by your configuration, see [storage].

• removed calibrations export dataset. Use the export option on the /Calibrations page instead.

• persistent storage is now on single sqlite3 database in /home/pioreactor/.pioreactor/storage/local_persistent_pioreactor_metadata.sqlite. This is configurable in your configuration.

• When checking for calibrations in custom Dosing automations, users may have added:

     with local_persistant_storage("current_pump_calibration") as cache:
         if "media" not in cache:
         ...

  This should be updated to (Note the spelling in local_persistant_storage changed, too!):

      with local_persistent_storage("active_calibrations") as cache:
          if "media_pump" not in cache:
          ...

• removed pioreactor.utils.gpio_helpers

Bug fixes

• fix PWM3 not cleaning up correctly
• fixed Stirring not updating to best DC % when using a calibration after changing target RPM

24.12.10

24.12.10

• Hotfix for UI settings bug

24.12.5

24.12.5

Important

We recommend to be on at least 24.10.29 before updating!

Highlights

• New export datasets improvements!
  • new export dataset API. The datasets on the Export Data UI page are now provided via YAML files on the leader's disk. This makes it easy to add new datasets to that UI to be exported. These YAML files can be added to ~/.pioreactor/exportable_datasets.
  • new Export Data page in the UI. Preview datasets before you export them, and new partition options for the exported CSVs.
  • Plugins can now add datasets to the Export Data page. The plugin's datasets are automatically added to the Export Data page when installed.
• Stirring can now pause itself during an OD reading. This is accomplished by "dodging OD readings". You can activate this feature by setting the enable_dodging_od to True in config.ini, under [stirring.config]. The replaces an older, less reliable plugin that was on our forums. Users have wanted this feature to have a very fast RPM between OD measurements (to get more aeration), and avoid noisy OD measurements. There's no reason to believe this will decrease the noise if using a "moderate" RPM though.

Enhancements

• improvements to Dodging background job code, including the ability to initialize the class based on dodging or not.
• better error handling for failed OD blank action.
• better button state management in the UI.
• a job YAMLs' published_settings can have a new field, editable (bool), which controls whether it shows up on the Settings dialog or not. (False means it won't show up since it's not editable!). Default is true. This should align with the published_setting in Python's job classes.
• you can add IPv4 addresses to the (new) [cluster.addresses] section to specify IPs for pioreactors. Example:

  [cluster.addresses]
  pio01=10.42.0.2
  pio02=10.42.0.3
  
  

  Note that the leader's address is automatically added in our software.
• new installs only: updated RPiOS to version 2024-11-19
• improvements to correlation self-tests

Bug fixes

• Fixed "circulate X" actions in the Manage All dialog in the UI.

Breaking changes

• moved all the temporary caches, which previously where their own sqlite3 dbs in /tmp/ to /tmp/local_intermittent_pioreactor_metadata.sqlite. This shouldn't break anything unless you update during an experiment - don't do that!

24.12.3rc0

Upcoming

Highlights

• New export datasets improvements!
  • new export dataset API. The datasets on the Export Data UI page are now provided via YAML files on the leader's disk. This makes it easy to add new datasets to that UI to be exported. These YAML files can be added to ~/.pioreactor/exportable_datasets.
  • new Export Data page in the UI. Preview datasets before you export them, and new partition options for the exported CSVs.
  • Plugins can now add datasets to the Export Data page. The plugin's datasets are automatically added to the Export Data page when installed.
• Stirring can now pause itself during an OD reading. This is accomplished by "dodging OD readings". You can activate this feature by setting the enable_dodging_od to True in config.ini, under [stirring.config]. The replaces an older, less reliable plugin that was on our forums. Users have wanted this feature to have a very fast RPM between OD measurements (to get more aeration), and avoid noisy OD measurements. There's no reason to believe this will decrease the noise if using a "moderate" RPM though.

Enhancements

• improvements to Dodging background job code, including the ability to initialize the class based on dodging or not.
• better error handling for failed OD blanks.
• better button state management in the UI.
• job YAMLs' published_settings can have a new field, editable (bool), which controls whether it shows up on the Settings dialog or not. (False means it won't show up since it's not editable!). Default is true. This should align with the published_setting in Python's job classes.
• you can add IPv4 addresses to the (new) [cluster.addresses] section to specify IPs for pioreactors. Example:

  [cluster.addresses]
  pio01_address=10.42.0.2
  pio02_address=10.42.0.3
  

  Note that the leader's address is automatically added in our software.
• new installs only: updated RPiOS to version 2024-11-19
• improvements to correlation self-tests

Bug fixes

• Fixed "circulate X" actions in the Manage All dialog in the UI.

Breaking changes

• moved all the temporary caches, which previously where their own sqlite3 db in /tmp/ to /tmp/local_intermittent_pioreactor_metadata.sqlite. This shouldn't break anything unless you update during an experiment - don't do that!

24.10.29

24.10.29

Important

We recommend to be on atleast 24.9.19 before updating!

Enhancements

• dosing_automation.vial_volume replaced with dosing_automation.liquid_volume. You can see the values by watching pio mqtt -t "pioreactor/+/+/dosing_automation/liquid_volume" after starting a dosing automation.
• Adding a SQL table for tracking liquid_volume.
• Because we are now storing liquid_volume in the database, you can add charts in the UI that track the volume over time:
  1. Add the following yaml contents to ~/.pioreactor/plugins/ui/contrib/charts/liquid_volume.yaml: https://gist.github.com/CamDavidsonPilon/95eef30189101da69f706d02ef28d972
  2. In your config.ini, under ui.overview.charts, add the line liquid_volume=1.
• New dataset exports from the Export data page in the UI: calibrations and liquid-volumes.
• Added a "partition by unit" option to the Export data page that will create a csv per Pioreactor in the export, instead of grouping them all together.
• od calibrations can use the --json-file to edit calibration polynomial coefficients. In the json file, specify curve_data_ fields with values of the curve's polynomial coefficients (leading term first), and set curve_type as "poly". The routine will begin with that calibration curve displayed.
• faster UI response times when starting jobs.
• faster syncing configs.
• faster copying files across cluster via pio cp.
• faster clean up of jobs using PWMs.
• new installs only: updated base RPiOS to 2024-10-22.
• new database table in /tmp/local_intermittent_pioreactor_metadata.sqlite called pio_job_published_settings that stores the published settings for each job. This powers the next API endpoints:
• New API endpoints for getting the current settings of a running job:
  • Per pioreactor:
    • GET: /unit_api/jobs/settings/job_name/<job_name>
    • GET: /unit_api/jobs/settings/job_name/<job_name>/setting/<setting>
  • Across the cluster:
    • GET: /api/jobs/settings/job_name/<job_name>/setting/<setting>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>/setting/<setting>
    • GET: /api/jobs/settings/workers/<pioreactor_unit>/job_name/<job_name>
    • GET: /api/jobs/settings/workers/<pioreactor_unit>/job_name/<job_name>/setting/<setting>
      Ex: query the temperature of a Pioreactor: curl http://pio01.local/unit_api/jobs/settings/job_name/temperature_automation/setting/temperature

Breaking changes

• pio kill --name x is now pio kill --job-name x
• removed publishing published_settings metadata to mqtt. Ex $properties, $settable, $unit, $datatype are no longer being sent mqtt. This was never used, and just a bandwidth suck.

Bug fixes

• fix for OD calibration graph showing "two lines" in the terminal display
• fix for updating over the internet when a Pioreactor is on a A.devX or B.rcY release
• pio kill --all-jobs will no longer kill long-running jobs from plugins (specifically, logs2x jobs.)
• updating the UI software won't prematurely stop any currently running activities
• correct ethernet mac address on RPi5s
• We weren't passing all the OS environment variables when jobs were started from the UI. This is fixed now.
• Fixed circulate media / alt. media in the UI.
• Fixed manual dosing updates in the UI.

24.10.28rc0

Upcoming

Enhancements

• dosing_automation.vial_volume replaced with dosing_automation.liquid_volume. You can see the values by watching pio mqtt -t "pioreactor/+/+/dosing_automation/liquid_volume" after starting a dosing automation.
• Adding a SQL table for tracking liquid_volume.
• Because we are now storing liquid_volume in the database, you can add charts in the UI that track the volume over time:
  1. Add the following yaml contents to ~/.pioreactor/plugins/contrib/charts/lqiuid_volume.yaml: https://gist.github.com/CamDavidsonPilon/95eef30189101da69f706d02ef28d972
  2. In your config.ini, under ui.overview.charts, add the line liquid_volume=1.
• New dataset exports from the Export data page in the UI: calibrations and liquid-volumes.
• Added a "partition by unit" option to the Export data page that will create a csv per Pioreactor in the export, instead of grouping them all together.
• od calibrations can use the --json-file to edit calibration polynomial coefficients. In the json file, specify curve_data_ fields with values of the curve's polynomial coefficients (leading term first), and set curve_type as "poly". The routine will begin with that calibration curve displayed.
• faster UI response times when starting jobs.
• faster syncing configs.
• faster copying files across cluster via pio cp.
• faster clean up of jobs using PWMs.
• New installs only: updated base RPiOS to 2024-10-22.
• new database table in /tmp/local_intermittent_pioreactor_metadata.sqlite called pio_job_published_settings that stores the published settings for each job. This powers the next API endpoints:
• New API endpoints for getting the current settings of a running job:
  • Per pioreactor:
    • GET: /unit_api/jobs/settings/job_name/<job_name>
    • GET: /unit_api/jobs/settings/job_name/<job_name>/setting/<setting>
  • Across the cluster:
    • GET: /api/jobs/settings/job_name/<job_name>/setting/<setting>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>/setting/<setting>
    • GET: /api/jobs/settings/workers/<unit>/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/workers/<unit>/job_name/<job_name>/experiments/<experiment>/setting/<setting>
      Ex: query the temperature of a Pioreactor: curl http://pio01.local/unit_api/jobs/settings/job_name/temperature_automation/setting/temperature

Breaking changes

• pio kill --name x is now pio kill --job-name x

Bug fixes

• fix for OD calibration graph showing "two lines" in the terminal display
• fix for updating over the internet when a Pioreactor is on a A.devX or B.rcY release
• pio kill --all-jobs will no longer kill long-running jobs from plugins (specifically, logs2x jobs.)
• updating the UI software won't prematurely stop any currently running activities
• correct ethernet mac address on RPi5s
• We weren't passing all the OS environment variables when jobs were started from the UI. This is fixed now.
• Fixed circulate media / alt. media in the UI.

24.10.20rc0

Enhancements

• dosing_automation.vial_volume replaced with dosing_automation.liquid_volume. You can see the values by watching pio mqtt -t "pioreactor/+/+/dosing_automation/liquid_volume" after starting a dosing automation.
• Adding a SQL table for tracking liquid_volume.
• Because we are now storing liquid_volume in the database, you can add charts in the UI that track the volume over time:
  1. Add the following yaml contents to ~/.pioreactor/plugins/contrib/charts/lqiuid_volume.yaml: https://gist.github.com/CamDavidsonPilon/95eef30189101da69f706d02ef28d972
  2. In your config.ini, under ui.overview.charts, add the line liquid_volume=1.
• New dataset exports from the Export data page in the UI: calibrations and liquid-volumes.
• Added a "partition by unit" option to the Export data page that will create a csv per Pioreactor in the export, instead of grouping them all together.
• od calibrations can use the --json-file to edit calibration polynomial coefficients. In the json file, specify curve_data_ fields with values of the curve's polynomial coefficients (leading term first), and set curve_type as "poly". The routine will begin with that calibration curve displayed.
• faster UI response times when starting jobs.
• faster syncing configs.
• faster copying files across cluster via pio cp.
• New API endpoints for getting the current settings of a running job:
  • Per pioreactor:
    • GET: /unit_api/jobs/settings/job_name/<job_name>
    • GET: /unit_api/jobs/settings/job_name/<job_name>/setting/<setting>
  • Across the cluster:
    • GET: /api/jobs/settings/job_name/<job_name>/setting/<setting>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>/setting/<setting>
    • GET: /api/jobs/settings/workers/<unit>/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/workers/<unit>/job_name/<job_name>/experiments/<experiment>/setting/<setting>
      Ex: query the temperature of a Pioreactor: curl http://pio01.local/unit_api/jobs/settings/job_name/temperature_automation/setting/temperature

Breaking changes

• pio kill --name x is now pio kill --job-name x

Bug fixes

• fix for OD calibration graph showing "two lines" in the terminal display
• fix for updating over the internet when a Pioreactor is on a A.devX or B.rcY release
• pio kill --all-jobs will no longer kill long-running jobs from plugins (specifically, logs2x jobs.)
• updating the UI software won't prematurely stop any currently running activities

24.10.13rc0

Upcoming

Enhancements

• dosing_automation.vial_volume replaced with liquid_volume.
• Adding a SQL table for tracking liquid_volume
• New dataset exports from the Export data page in the UI: calibrations and liquid-volumes.
• Added a "partition by unit" option to the Export data page that will create a csv per Pioreactor in the export, instead of grouping them all together.
• od calibrations can use the -f to edit calibration polynomial coefficients.
• faster UI response times when starting jobs
• faster syncing configs
• faster copying files across cluster via pio cp
• Because we are now storing liquid_volume in the database, you can add charts in the UI that track the volume over time:
  1. Add the following yaml contents to ~/.pioreactor/plugins/contrib/charts/lqiuid_volume.yaml: https://gist.github.com/CamDavidsonPilon/95eef30189101da69f706d02ef28d972
  2. In your config.ini, under ui.overview.charts, add the line liquid_volume=1.
• New API endpoints for getting the current settings of a running job:
  • Per pioreactor:
    • GET: /unit_api/jobs/settings/job_name/<job_name>
    • GET: /unit_api/jobs/settings/job_name/<job_name>/setting/<setting>
  • Across the cluster:
    • GET: /api/jobs/settings/job_name/<job_name>/setting/<setting>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>
    • GET: /api/jobs/settings/job_name/<job_name>/experiments/<experiment>/setting/<setting>
      This is the first time a web API has allowed one to query basic Pioreactor data, ex: like the temperature of a Pioreactor: curl pio01.local/unit_api/jobs/settings/job_name/temperature_automation/setting/temperature

24.10.1

24.10.1

Important

This is an important bug release update to regressions introduced in 24.9.26. We highly recommend you be on version 24.9.19 or above before updating!

Enhancements

• amount of data shown on charts is now a function of the OD sampling rate
• allow for showing more than 16 workers in a chart.

Bug fixes

• Bug fix for "Manage all" that would start activities in all Pioreactors, whether they were in the experiment or not.
• Fix for bug when clicking a legend element it not hiding
• led_intensity (i.e. changes to LEDs) now respect whether a worker is active or not.
• Fix bug for UI crashing with "colors" error.
• If a worker is referenced in a profile, but is not part of the current experiment, the actions will not be schedualed for it.