Skip to content

Commit

Permalink
updates to ls
Browse files Browse the repository at this point in the history
  • Loading branch information
amandarichardsonn committed Oct 31, 2024
1 parent 441579c commit 98e4590
Showing 1 changed file with 94 additions and 75 deletions.
169 changes: 94 additions & 75 deletions doc/launch_settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ LaunchSettings
========
Overview
========
The ``LaunchSettings`` class manages launcher configuration settings to enable the injection of
launcher-specific behavior into jobs. SmartSim ``LaunchSettings`` supports several launchers and allows users to
configure launch arguments and environment variables. Additionally, the object provides methods to access and modify
launch arguments, environment variables and retrieve information associated with the ``LaunchSettings`` object.
The ``LaunchSettings`` class manages launcher configuration settings to enable the injection of launcher-specific behavior into jobs.
SmartSim’s ``LaunchSettings`` supports multiple launchers, allowing users to set both launch arguments and environment variables.
Additionally, this class provides methods to access and configure these settings, as well as retrieve information associated with
the ``LaunchSettings`` object.

**Dragon** is the fastest and most versatile distributed runtime available for HPC workflows. It can function as a
launcher within a Slurm allocation, providing rapid, interactive, and customizable execution of complex workflows
on large HPC systems. As a scheduler-agnostic solution, Dragon allows the same SmartSim script to run seamlessly
on both Slurm and PBS systems, with future support for additional schedulers.
on large HPC systems. As a scheduler-agnostic solution, Dragon allows the same SmartSim script to run on both Slurm and
PBS systems, with future support for additional schedulers.

===================
Supported Launchers
Expand All @@ -21,71 +21,73 @@ The ``LaunchSettings`` class supports multiple launchers, each customized for di
**Dragon** stands out as the fastest and most versatile option, ideal for large-scale HPC jobs. The following launchers
are categorized based on their specific use cases and the systems they are designed for.

HPE Cray Specific Launchers
===========================
SLURM and PALS launchers are specifically designed for HPE Cray systems, providing efficient management and
execution of parallel applications.
HPE Cray
========
SmartSim provides support for two HPE Cray system launchers: **ALPS** and **PALS**. ALPS is designed for legacy Cray platforms
to address both future Cray platform needs and the limitations of older software components. PALS acts as a launcher for third-party
workload managers (WLMs) that do not have their own launchers.

.. list-table:: HPE Cray Specific Launchers
:header-rows: 1

* - **Run Command**
- **Description**
* - ``aprun``
- Application Level Placement Scheduler for HPE Cray systems.
* - ``palsrun``
- Parallel Application Launch Service for distributed computing.
- Application Level Placement Scheduler (ALPS).
* - ``mpiexec``
- Parallel Application Launch Service (PALS).


IMB Specific Launchers
======================
The LSF launcher is tailored for use with the IBM Spectrum Load Sharing Facility (LSF), which is commonly used in
enterprise environments to distribute and manage workloads.
IBM
===
Smartsim provides support for an IBM launcher: **LSF**. IBM's LSF is a platform for managing workloads and scheduling jobs
in distributed high-performance computing (HPC) environments.

.. list-table:: IMB Specific Launchers
:header-rows: 1

* - **Run Command**
- **Description**
* - ``jsrun``
- Load Sharing Facility for launching parallel jobs.
- Load Sharing Facility (LSF).

Standard MPI Launchers
======================
These launchers are used for running MPI (Message Passing Interface) applications, providing flexibility
and control over parallel job execution.
Standard MPI
============
SmartSim supports three commands to launch Message Passing Interface (MPI) applications: ``mpiexec``, ``mpirun`` and ``orterun``.
Each execute serial and parallel jobs in Open MPI.

.. list-table:: MPI Launchers
:header-rows: 1

* - **Run Command**
- **Description**
* - ``mpiexec``
- Standard MPI launcher for parallel jobs.
- Used command to start MPI applications.
* - ``mpirun``
- Another MPI launcher, often used with Open MPI.
- Alternative command for launching MPI programs.
* - ``orterun``
- Open MPI's runtime environment launcher.
- Backend launcher for Open MPI's runtime environment.

General HPC Launchers
=====================
The SLURM and Dragon launchers are suitable for general high-performance computing (HPC) environments, offering robust solutions
for job scheduling and resource management.
General
=======
SmartSim provides support for two general launchers: **Dragon** and **SLURM**. Dragon is the most rapid and adaptable distributed
runtime for HPC workflows. SLURM is a open-source job scheduler and resource manager for Linux clusters.

.. list-table:: General HPC Launchers
:header-rows: 1

* - **Run Command**
- **Description**
* - ``srun``
- Open-source job scheduler and resource manager for Linux clusters.
- Simple Linux Utility for Resource Management (SLURM).
* - ``dragonrun``
- High-performance computing launcher for large-scale jobs.
- High-performance computing launcher for large-scale jobs (Dragon).

Other
Local
=====
SmartSim provides support to launch applications on the local machine.

.. list-table:: General HPC Launchers
.. list-table:: Local Launcher
:header-rows: 1

* - **Run Command**
Expand All @@ -99,9 +101,9 @@ various environments.
==========
Initialize
==========
The ``LaunchSettings`` class provides a way to configure and manage the execution environment for your
applications. This section outlines the steps to configure the essential parameters that determine how
your application launches.
The ``LaunchSettings`` class allows you to set your application’s execution
environment. This section details the steps to set up the input parameters for
launching your application.

**Step 1: Import LaunchSettings**

Expand All @@ -114,8 +116,8 @@ After installing Smartsim, ``LaunchSettings`` may be imported in Python code lik
**Step 2: Set the Launcher Type**

Set the launcher type using either a string or a ``LauncherType`` enum. This step is crucial as it
determines the specific launcher configuration that will be applied. The following table lists the
supported strings and their corresponding enums:
determines the applied launcher. The following table lists the supported launcher strings and
their corresponding ``LauncherType`` enums:

.. list-table:: Supported Launcher Strings and Enums
:header-rows: 1
Expand Down Expand Up @@ -151,9 +153,15 @@ supported strings and their corresponding enums:
- ``"lsf"``
- ``LauncherType.Lsf``

To use an enum, import ``LauncherType`` into Python code like:

.. code-block:: python
from smartsim import LauncherType
**Step 3: Provide Launch Arguments and Environment Variables**
Optionally, you can provide ``launch_args`` and ``env_vars`` to customize the job execution environment:

Optionally, you can provide ``launch_args`` and ``env_vars`` to customize the application's execution environment:

* ``launch_args``: A dictionary where keys are argument names (strings) and values are argument values (strings).
These arguments are specific to the launcher being used. Example:
Expand All @@ -163,17 +171,17 @@ Optionally, you can provide ``launch_args`` and ``env_vars`` to customize the jo
launch_args = {"--time": "01:00:00", "--nodes": "2"}
* ``env_vars``: A dictionary where keys are environment variable names (strings) and values are environment
variable values (strings). These variables set the environment for the job execution. Example:
variable values (strings). These variables set the environment for the application execution. Example:

.. code-block:: python
env_vars = {"MY_VAR": "my_value", "ANOTHER_VAR": "another_value"}
Here's how you can initialize ``LaunchSettings`` with these parameters:
Here's how you can initialize ``LaunchSettings`` with input parameters:

**Example using a launcher String:**
Once you have imported ``LaunchSettings`` using ``from smartsim import LaunchSettings``, use a
launcher string such as `"slurm"`. For example:
Once you have imported ``LaunchSettings`` using ``from smartsim import LaunchSettings``, set the input
varaible ``launcher`` to a launcher string such as `"slurm"`. For example:

.. code-block:: python
Expand All @@ -184,8 +192,8 @@ launcher string such as `"slurm"`. For example:
)
**Example using an Enum:**
Once you have imported ``LaunchSettings`` using ``from smartsim import LaunchSettings``, use a
``LauncherType`` enum such as ``LauncherType.Slurm``. For example:
Once you have imported ``LaunchSettings`` and ``LauncherType`` using ``from smartsim import LaunchSettings, LauncherType``,
set the input variable ``launcher`` to a ``LauncherType`` enum such as ``LauncherType.Slurm``. For example:

.. code-block:: python
Expand All @@ -199,22 +207,17 @@ Once you have imported ``LaunchSettings`` using ``from smartsim import LaunchSet
Configure
=========
After initializing a ``LaunchSettings`` object, you might want to go back and configure ``launch_args`` or
``env_vars`` for several reasons. Customizing these settings allows you to fine-tune the execution environment
to meet the specific needs of different jobs, optimize performance, and achieve better resource utilization and
faster execution times. Additionally, modifying these settings can aid in debugging by enabling additional
logging, setting debug flags, or changing the execution environment. Experimentation with different configurations
can also help you understand how changes affect performance or behavior, which is particularly useful in research
and development settings. Finally, adjusting these settings allows for a more personalized and efficient workflow,
accommodating different user preferences and requirements.
``env_vars``. Configuring these settings allows you to fine-tune the execution environment
to meet the specific needs of different jobs.

Launch Arguments
================
Configure `launch_args` in two ways:
There are two methods to configure ``launch_args``:

1. Use ``LaunchSettings.launch_args.set``.
2. Use custom methods specific to each launcher.
2. Use custom methods specific to each HPC launcher.

These functions allow you to customize `launch_args` after initializing the ``LaunchSettings`` object.
These functions allow you to customize launch arguments after initializing the ``LaunchSettings`` object.

**Option 1: Use LaunchSettings.launch_args.set**

Expand All @@ -225,11 +228,10 @@ use the `set` method on `launch_args` as shown below:
launch_settings.launch_args.set("--nodes", "2")
**Option 2: Use custom methods specific to each launcher**
**Option 2: Use custom HPC launcher methods**

The ``LaunchSettings`` class provides custom methods to set launch arguments for different supported launchers.
These methods tailor the job execution environment to meet specific requirements of each launcher.
Below are examples of how to use these custom methods for various launchers.
The ``LaunchSettings`` class provides custom methods to set launch arguments for different launchers.
Below are examples of how to use the custom methods for the supported launchers.

.. tabs::

Expand All @@ -239,76 +241,93 @@ Below are examples of how to use these custom methods for various launchers.

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set the nodes for a Dragon job
launch_settings.launch_args.set_nodes(128)
.. tab:: SLURM

**Set Launch Arguments for SLURM:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set the nodes for a SLURM job
launch_settings.launch_args.set_nodes(128)
.. tab:: PALS

**Set Launch Arguments for PALS:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks per node for a PALS job
launch_settings.launch_args.set_tasks_per_node(8)
.. tab:: ALPS

**Set Launch Arguments for ALPS:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
.. tab:: Local

**Set Launch Arguments for Local:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks per node for a ALPS job
launch_settings.launch_args.set_tasks_per_node(8)
.. tab:: Mpiexec

**Set Launch Arguments for Mpiexec:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks per node for a Mpiexec job
launch_settings.launch_args.set_tasks_per_node(8)
.. tab:: Mpirun

**Set Launch Arguments for Mpirun:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks per node for a Mpirun job
launch_settings.launch_args.set_tasks_per_node(8)
.. tab:: Orterun

**Set Launch Arguments for Orterun:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks per node for a Orterun job
launch_settings.launch_args.set_tasks_per_node(8)
.. tab:: LSF

**Set Launch Arguments for LSF:**

.. code-block:: python
launch_settings.launch_args.set("--nodes", "2")
# Set tasks for a LSF job
launch_settings.launch_args.set_tasks(8)
For detailed information on these methods, refer to the API reference page.

Environment Variables
=====================
There are two methods to configure ``env_vars``:

1. Use ``LaunchSettings.env_vars`` to overwrite the existing executable arguments.
2. Use ``LaunchSettings.update_env`` to add to the existing environment variables.

**Option 1: Use LaunchSettings.env_vars**

To overwrite the ``env_vars`` after initializing the ``LaunchSettings`` object, set a dictionary where
each key and value are strings to ``LaunchSettings.env_vars``. This method overwrites the existing dictionary
of environment settings. For example:

.. code-block:: python
launch_settings.env_vars = {"MY_VAR": "new_value", "ANOTHER_VAR": "another_value"}
**Option 1: Use LaunchSettings.update_env**

To update the ``env_vars`` after initializing the ``LaunchSettings`` object, pass in a dictionary where
each key and value are strings to ``LaunchSettings.update_env``. This function updates the existing dictionary
of environment settings without overwriting it. For example:
Expand Down

0 comments on commit 98e4590

Please sign in to comment.