Skip to content

Tutorial

Jump to bottom
Joris Borgdorff edited this page Aug 30, 2016 · 3 revisions

There are some examples provided within the MUSCLE bundle. Example code for kernels can be found in the directories src/java/examples and src/cpp/examples. There are also prepared configurations for coupled examples CxA (MUSCLE simulations), which can be found in the src/cxa directory.

For more information, please contact < muscle at jorisborgdorff.nl >

Part 1: installing and testing

If you have a Windows machine, first log in to a Linux or Mac OS X machine.

  1. Follow the MUSCLE installation

  2. To add MUSCLE to the path and set a few environment variables, run the following (don't forget the dot at the start!):

 . [MUSCLE_DIR]/etc/muscle.profile
  1. Now you can run muscle2. We will show its functionality with an example model that is packaged with MUSCLE. First go to the unzipped MUSCLE folder, and try to read src/cxa/SimpleSubmodelExample.cxa.rb and see what it does. To show the submodels that the example contains, run:
 cd ..
 muscle2 --cxa src/cxa/SimpleSubmodelExample.cxa.rb

This example has a writer submodel w and a reader submodel r. The writer sends data to the reader, and the reader prints that data to screen. You can run the simple example by specifying that the current MUSCLE command will run the Simulation Manager with the --main flag or its shorthand -m, and specifying all submodels that should run in this command:

muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w r

Usually, if you want to run all the submodels instances with the current MUSCLE command, specify the --allinstances or its shorthand -a:

muscle2 -amc src/cxa/SimpleSubmodelExample.cxa.rb

So what do we observe? MUSCLE detects that it should start the Simulation Manager and a Simulation. It first starts the manager, which shows its TCP/IP address. Then the Simulation starts and it executes the submodel instances w and r. These start their respective conduits, and start 'computing'. On local machines, these commands are sufficient to run any MUSCLE model. You can try the other examples in the src/cxa directory, just run

muscle2 -amc src/cxa/[...].cxa.rb
  1. Scientific applications that are started remotely generally do not have a direct feedback to a user interface, but we will run one that does. To get a user interface with SSH, use its -X flag.
muscle2 -amc src/cxa/LaplaceExample.cxa.rb

It will open two windows that compute temperature dissipation, that are coupled on the boundary. Red is a +1 temparature, blue a -1 temperature, and green a zero temperature. The initial condition, far right and far left correspond to zero temperature, the top to a sine function and the bottom to a cosine. Temperature is dissipated during each time step by taking the average of the four neighboring cells. The adjacent boundary is transmitted each iteration.

  1. Take a look at all the options by running
muscle2

Part 2: distributed execution

In this part we will do an inter-process and an inter-machine run of MUSCLE.

  1. Open a second terminal, and log in to another machine, or try it on the same machine. When logged in do
 . [MUSCLE_DIR]/etc/muscle.profile

Then go to the unzipped MUSCLE directory. Now we can run muscle on the same site. In the first terminal window, start the main MUSCLE with the writing submodel w. This is done by typing:

 muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w

So what do you observe? One of the most important messages is Started the connection handler, listening on [IPADDRESS:PORT]. Copy the IP address and port number. In the second terminal, start the reading submodel r, and at the end put the -M or --manager flag with the IP address and port:

 muscle2 -c src/cxa/SimpleSubmodelExample.cxa.rb r -M [IPADDRESS:PORT]

What happens in both terminals?

  1. To isolate the output from the Simulation Manager, we can run it separately. In the first terminal window, start the Simulation Manager:
 muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb

In the second terminal, start the submodels. Again, replace the port number with the one that is printed by the Simulation Manager.

 muscle2 -ac src/cxa/SimpleSubmodelExample.cxa.rb -M [IPADDRESS:PORT]

What happens in both terminals?

  1. To do distributed computing, also install MUSCLE on another node
ssh user@node2
[INSTALL MUSCLE...]
. MUSCLE_DIR/etc/muscle.profile

And go to the source directory of MUSCLE. We will perform the same experiment, but now on different hosts. So on the first node, for instance run only w

muscle2 -mc src/cxa/SimpleSubmodelExample.cxa.rb w

On the other node, we now also need to specify which host and port the other MUSCLE instance is running on

 muscle -c src/cxa/SimpleSubmodelExample.cxa.rb -M [IPADDRESS:PORT] r

What is the output in node one? And on the other?

Related: inter-cluster executions can be done with the --intercluster tag, with which the MUSCLE transport overlay (MTO) is used. We will not be doing this directly here. In the MTO documentation this is explained more thoroughly.

Part 3: The power of coupling

First, download an extension of the LaplaceExample that was shown in item 1.4.

wget https://github.com/psnc-apps/muscle2/wiki/resources/laplace.tar.gz
tar xzf laplace.tar.gz
cd laplace
ant

The source of the submodels can be found in src/laplace. This example code shows how the same code can be reused in different scenarios:

  • Coupling as in example 1.4 - run muscle2 -amc laplace.cxa.rb
  • Doing some simple transformation with filters after sending the data - muscle2 -amc laplaceFilter.cxa.rb
  • Setting one boundary explicitly using terminals - muscle2 -amc laplaceTerminal.cxa.rb
  • Using a periodic boundary condition - muscle2 -amc laplacePeriodic.cxa.rb
  • Doing domain decomposition using mappers - muscle2 -amc laplaceMapper.cxa.rb

Their coupling is shown in the following diagrams:

Scenarios

  1. Try the different scenarios and see what the differences are between the configuration files.
  2. Modify the configuration files to get different geometries or transformations. For example (pick one):
  • add a submodel instance so that you get a coupling between West - Middle - East;
  • change the filters and see how it affects the computations (see Filter documentation); or
  • modify the file src/laplace/filter/InvertFilter to do something additional (see Java filter documentation)
  • modify the file src/laplace/terminal/DoubleArraySource.java to do something additional, such as computing a function instead of sending a fixed number (see Java terminal documentation).

Part 4: a diagram of your tightly coupled model

To apply this programming paradigm to your own multiscale model, you can apply the methodology as presented on day 1.

  1. Identify the processes in your problem, and their scales. Draw them on the same diagram, taking time as a x-axis, and space as the y-axis. Each process should have a granularity (step size), which is the lower bound for the rectangle, and a problem size (total size), which is the upper bound for the rectangle.
  2. Draw arrows between interacting processes. Now your scale separation map is more or less complete.
  3. When you think about implementation of that scale separation map, could you use any of the coupling types shown in part 3? Would it be tightly or loosely coupled?
  4. Work out coupling diagram of your own application.

Part 5: modifying the simple example (optional)

To start on using MUSCLE for your own applications, you can start by modifying the examples. Depending on the programming language of your application, you can start with (Java documentation) the files in src/java/examples/simplesubmodel or with (C++/C documentation) src/cpp/examples/simplecpp or (Fortran documentation) src/cpp/examples/simplefortran. The last two only contain the Sender (the submodel w) of the simple example.

  1. Modify src/java/examples/simplesubmodel/ConsoleWriter.java so that it sends a message back to Sender.java; modify Sender.java to receive this message. In this example, call both the sending and the receiving port "messages", for example. Test your implementation by running
  cd build
./build.sh $MUSCLE_HOME
 cd ..
 muscle2 -amc src/cxa/SimpleSubmodelExample.cxa.rb

Spoiler:

  • In ConsoleWriter.java
    • create the method protected void intermediateObservation()
    • send some data over it in execute() by adding 
         double[] msg = {1.0, 0.0, 1.0};
  out("messages").send(msg);
  • In Sender.java
    • Create the method protected void solvingStep()
    • Receive the data
    double[] msg = (double[])in("messages").receive();
    • Print the data with
    for (double d : msg) {
  log("Message from ConsoleWriter: " + d);
}
  • In the configuration file src/cxa/SimpleSubmodelExample.cxa.rb
    • Append the lines 
      r.couple(w, {'messages' => 'messages'})
    • Since the names of the ports are equal (messages), you can also use the abbreviated form: 
      r.couple(w, 'messages')
  • Build it and execute
  1. Modify src/cpp/examples/simplecpp/Sender.cpp to receive a message from ConsoleWriter
cd build
./build.sh $HOME/muscle
muscle2 -amc ../src/cxa/NativeExample2.cxa.rb

Spoiler:

  • In Sender.cpp

    • Receive some data after sending by coding in the main() function
     size_t sz;
 double* msg = (double *)
  muscle::env::receive("messages", (void *)0, sz, MUSCLE_DOUBLE);
    • Print the data with
     for (size_t j = 0; j < sz; j++) {
  logger::info("Message from ConsoleWriter: %f", msg[j]);
 }
    • Free the data with
     muscle::env::free_data(msg, MUSCLE_DOUBLE);

  • In the configuration file src/cxa/NativeExample2.cxa.rb

    • Change the line
     r = Instance.new('r', 'examples.simplejava.ConsoleWriter')

    to

     r = Instance.new('r', 'examples.simplesubmodel.ConsoleWriter')

  • Append the lines

 r.couple(w, 'messages')

If you would prefer to use the free-form API of MUSCLE, take a look at the SimpleExample, this doesn't implement the different formal operators of MML, but just an execute() method.

Running SimpleExample

  • Sender: in src/java/examples/simplejava/Sender.java we provide a minimalist kernel written in Java, which sends data (an array of double).

  • Receiver: in src/java/examples/simplejava/ConsoleWriter.java we provide a simple kernel written in Java. It receives data (an array of double) and prints its content to the standard output.

  • CxA (coupling schema): the configuration src/cxa/SimpleExample.cxa.rb configures a minimal CxA which couples the examples.simplejava.Sender and examples.simplejava.ConsoleWriter kernels.

  • Execution: to run the simulation defined in src/cxa/SimpleExample.cxa.rb use the following command:

    muscle2 --cxa src/cxa/SimpleExample.cxa.rb --main --allkernels

    or in shorthand notation

    muscle2 -c src/cxa/SimpleExample.cxa.rb -am

    It will first start the Simulation Manager, of which one per simulation is necessary, and then the local manager, which manages the kernels that are started with one muscle2 invocation.

Clone this wiki locally