This repository has been archived by the owner on Aug 20, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 55
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #11 from MeteoSwiss-APN/master
Push latest changes from MCH-APN for a 0.2 release
- Loading branch information
Showing
43 changed files
with
1,341 additions
and
1,045 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,3 @@ | ||
build | ||
install | ||
buildenv |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,28 +4,41 @@ | |
|
||
# Introduction | ||
|
||
The SerialBox is a set of tools around serialization / deserialization of data. | ||
The Serialbox is a set of tools around serialization / deserialization of data. | ||
The box is used in several projects for building validation frameworks against reference runs. | ||
This is useful in the scope of rewrite of large codes, or when porting codes to multiple computing architectures. | ||
As an example, porting scientific codes to graphical processing units, that require continuous validation against the existing x86 code. | ||
|
||
# Structure | ||
|
||
* src/ : It the main core of the SerialBox, written in C++, which provides | ||
* src/ : It the main core of the Serialbox, written in C++, which provides | ||
* C++ functionality that can be used to serialize data from a reference run. | ||
* Interfaces for deserializing into memory data structures | ||
* Control the metadata of the fields being serialized/deserialized. | ||
* src/utils: logger and other C++ utilities | ||
* wrapper: a C wrapper that allows to interoperate the core C++ functionality from other languages like Fortran | ||
* doc/ : latex documentation, tutorials and presentations | ||
* [python/](python/README.md) : contains various python tools based on the SerialBox format, namely | ||
* pp_ser: a tiny DSL/parser that facilitates inserting SerialBox statements into your Fortran Code. | ||
* [python/](python/README.md) : contains various python tools based on the Serialbox format, namely | ||
* pp_ser: a tiny DSL/parser that facilitates inserting Serialbox statements into your Fortran Code. | ||
* Visualizer: a python tool that helps visualizing serialized data | ||
* tools/ : various tools used to validation, dumping, converting or comparing of multiple serialized runs, etc. | ||
* fortran/ : fortran interfaces to the SerialBox C++ functionality | ||
* fortran/ : fortran interfaces to the Serialbox C++ functionality | ||
|
||
# Building | ||
|
||
simply ```mkdir build; cd build; cmake ../; make [install]``` | ||
|
||
for building the documentation ```make doc``` | ||
|
||
## CSCS Environment | ||
|
||
On daint, greina, kesch and lema a Serialbox installation is provided by jenkins (daily checkout of MeteoSwiss-APN master). If you want to compile and install Serialbox manually you can do so using the build environment. | ||
|
||
Go to your checkout of Serialbox, then execute: | ||
|
||
git clone [email protected]:C2SM-RCM/buildenv.git | ||
./buildenv/package_builder/build_apkg.sh -p serialbox -d . --local -c $COMPILER | ||
|
||
This will build and install serialbox into the `./install` folder with the compiler `$COMPILER` (usually gnu or cray). If the `-c $COMPILER` option is omitted the script will build against all available targets. | ||
|
||
Refer to the script for further configuration options. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
CMAKE_MINIMUM_REQUIRED(VERSION 2.8) | ||
PROJECT(PERTURB_SERIALBOX_FORTRAN_SAMPLE NONE) | ||
ENABLE_LANGUAGE(Fortran) | ||
|
||
set(SERIALBOX_DIR "${CMAKE_SOURCE_DIR}/../../install/") | ||
set(SERIALBOXI "${SERIALBOX_DIR}/${COMPILER}/include/SerialBox/") | ||
set(SERIALBOXL "${SERIALBOX_DIR}/lib") | ||
SET(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} -cpp -ffree-form -ffree-line-length-180 -J.") | ||
add_definitions(-DSERIALIZE) | ||
|
||
file(MAKE_DIRECTORY pp) | ||
|
||
include_directories(${SERIALBOXI}) | ||
include_directories("${SERIALBOX_DIR}/include/fortran") | ||
link_directories(${SERIALBOXL}) | ||
|
||
set( | ||
PERTURB_SAMPLE_SOURCES | ||
"m_ser.f90" | ||
"main_producer.f90" | ||
"main_consumer.f90" | ||
"main_consumer_perturb.f90" | ||
) | ||
|
||
# generate preprocessed files with pp_ser | ||
ADD_CUSTOM_COMMAND( | ||
OUTPUT pp/m_ser.f90 pp/main_consumer.f90 pp/main_consumer_perturb.f90 pp/main_producer.f90 | ||
COMMAND ${CMAKE_SOURCE_DIR}/../../python/pp_ser.py -d pp ${CMAKE_SOURCE_DIR}/m_ser.f90 | ||
COMMAND ${CMAKE_SOURCE_DIR}/../../python/pp_ser.py -d pp ${CMAKE_SOURCE_DIR}/main_consumer.f90 | ||
COMMAND ${CMAKE_SOURCE_DIR}/../../python/pp_ser.py -d pp ${CMAKE_SOURCE_DIR}/main_consumer_perturb.f90 | ||
COMMAND ${CMAKE_SOURCE_DIR}/../../python/pp_ser.py -d pp ${CMAKE_SOURCE_DIR}/main_producer.f90 | ||
DEPENDS m_ser.f90 main_consumer.f90 main_consumer_perturb.f90 main_producer.f90 | ||
) | ||
|
||
ADD_EXECUTABLE( | ||
main_producer | ||
pp/m_ser.f90 | ||
pp/main_producer.f90 | ||
) | ||
|
||
ADD_EXECUTABLE( | ||
main_consumer | ||
pp/m_ser.f90 | ||
pp/main_consumer.f90 | ||
) | ||
|
||
ADD_EXECUTABLE( | ||
main_consumer_perturb | ||
pp/m_ser.f90 | ||
pp/main_consumer_perturb.f90 | ||
) | ||
|
||
target_link_libraries(main_producer FortranSer.a) | ||
target_link_libraries(main_producer SerialboxWrapper.a) | ||
target_link_libraries(main_producer Serialbox.a) | ||
target_link_libraries(main_producer Utils.a) | ||
target_link_libraries(main_producer sha256.a) | ||
target_link_libraries(main_producer json.a) | ||
target_link_libraries(main_producer stdc++) | ||
|
||
target_link_libraries(main_consumer FortranSer.a) | ||
target_link_libraries(main_consumer SerialboxWrapper.a) | ||
target_link_libraries(main_consumer Serialbox.a) | ||
target_link_libraries(main_consumer Utils.a) | ||
target_link_libraries(main_consumer sha256.a) | ||
target_link_libraries(main_consumer json.a) | ||
target_link_libraries(main_consumer stdc++) | ||
|
||
target_link_libraries(main_consumer_perturb FortranSer.a) | ||
target_link_libraries(main_consumer_perturb SerialboxWrapper.a) | ||
target_link_libraries(main_consumer_perturb Serialbox.a) | ||
target_link_libraries(main_consumer_perturb Utils.a) | ||
target_link_libraries(main_consumer_perturb sha256.a) | ||
target_link_libraries(main_consumer_perturb json.a) | ||
target_link_libraries(main_consumer_perturb stdc++) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Serialbox Fortran example | ||
|
||
This small example shows how to use Serialbox in a Fortran code using the | ||
serialization directives. | ||
|
||
The serialization happens in the `m_ser.f90` module. Two small programs use | ||
this module to serialize and deserialize data. | ||
|
||
|
||
It is important to use the same Fortran compiler than the one used to build | ||
Serialbox. | ||
|
||
### Build the example | ||
```bash | ||
cmake . | ||
make | ||
``` | ||
|
||
### Run the example | ||
```bash | ||
./main_producer | ||
./main_consumer | ||
./main_consumer_perturb | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
MODULE m_ser | ||
|
||
IMPLICIT NONE | ||
|
||
CONTAINS | ||
|
||
SUBROUTINE serialize(a) | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(:,:,:) :: a | ||
|
||
!$ser init directory='.' prefix='SerialboxTest' | ||
!$ser savepoint sp1 | ||
!$ser mode write | ||
!$ser data ser_a=a | ||
|
||
END SUBROUTINE serialize | ||
|
||
SUBROUTINE deserialize(a) | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(:,:,:) :: a | ||
|
||
!$ser init directory='.' prefix='SerialboxTest-output' prefix_ref='SerialboxTest' | ||
!$ser savepoint sp1 | ||
!$ser mode read | ||
!$ser data ser_a=a | ||
!$ser mode write | ||
!$ser data ser_a=a | ||
|
||
END SUBROUTINE deserialize | ||
|
||
SUBROUTINE deserialize_with_perturb(a) | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(:,:,:) :: a | ||
REAL(KIND=8) :: rprecision | ||
rprecision = 10.0**(-PRECISION(1.0)) | ||
|
||
!$ser init directory='.' prefix='SerialboxTest-output' prefix_ref='SerialboxTest' rprecision=rprecision rperturb=1.0e-5_8 | ||
!$ser savepoint sp1 | ||
!$ser mode read-perturb | ||
!$ser data ser_a=a | ||
|
||
END SUBROUTINE deserialize_with_perturb | ||
|
||
END MODULE m_ser |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
PROGRAM main_consumer | ||
USE m_ser | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(5,5,5) :: a | ||
|
||
a = 0.0 | ||
PRINT*,'Before read from serializer: sum(a)=', sum(a) | ||
CALL deserialize(a) | ||
PRINT*,'After read from serializer: sum(a)=', sum(a) | ||
END PROGRAM main_consumer |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
PROGRAM main_consumer_perturb | ||
USE m_ser | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(5,5,5) :: a | ||
|
||
a = 0.0 | ||
PRINT*,'Before read from serializer: sum(a)=', sum(a) | ||
CALL deserialize_with_perturb(a) | ||
PRINT*,'After read from serializer: sum(a)=', sum(a) | ||
END PROGRAM main_consumer_perturb |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
PROGRAM main_producer | ||
USE m_ser | ||
IMPLICIT NONE | ||
REAL(KIND=8), DIMENSION(5,5,5) :: a | ||
|
||
a = 5.0 | ||
PRINT *, 'CALL serialize with sum(a)=', sum(a) | ||
CALL serialize(a) | ||
END PROGRAM main_producer |
Oops, something went wrong.