Skip to content

Conference call notes 20210203

Jump to bottom
Kenneth Hoste edited this page Feb 3, 2021 · 5 revisions

(back to Conference calls)

Notes on the 166th EasyBuild conference call, Wednesday February 3rd 2021 (16:00 UTC)

Attendees

Alphabetical list of attendees (16):

  • Sebastian Achilles (JSC, Germany)
  • Simon Branford (University of Birmingham, UK)
  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Fotis Georgatos (SDSC, Switzerland)
  • Alexander Grund (TU Dresden, Germany)
  • Victor Holando Rusu (CSCS, Switzerland)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Terje Kvernes (University of Oslo, Norway)
  • Kurt Lust (Univ. of Antwerp, Belgium + LUMI User Support Team)
  • Robert Mijakovic (LuxProvide)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (Compute Canada)
  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)
  • Åke Sandgren (Umeå University, Sweden)
  • Robert Schade (Univ. of Paderborn, Germany)

Agenda

  • update on recent developments + outlook to next release
  • review of EasyBuild User Meeting
  • switching from OpenBLAS to BLIS in foss/2021a
  • Q&A

Recent developments

  • next release (v4.3.3): before EUM'21? during EUM week? hopefully next week...
  • recent changes
    • framework
      • bug fixes
        • (nothing)
      • enhancements
        • also consider toolchain deps when defining template values (PR #3541)
          • added so %(cudaver)s template is defined when using fosscuda toolchain
          • => follow-up PR #3560 to add missing test + avoid early Toolchain initialization
      • changes
        • rename EasyBlock._skip_step to EasyBlock.skip_step, to make it part of the public API (PR #3561)
          • required for letting PythonPackage easyblock use skip_step method (PR #2290)
    • easyblocks
      • bug fixes
        • Use library search paths of compiler for RPATH when building binutils with system compiler + enhance sanity check by running --version for binutils commands (PR #2323)
          • avoids use of hardcoded paths like /usr/lib64 when building binutils with system toolchain
        • Fix OpenFOAM sanity check on POWER (PR #2320)
      • enhancements
        • improve Bazel easyblock: add support for running tests, enable static linking, use build dir rather than tmpdir, verbose output (PR #2285)
        • add check for accepted EULA in custom easyblock for NVHPC (PR #2311)
        • update Clang easyblock to add support for building extra tools + leveraging hwloc and Z3 as optional dependencies (PR #2310)
        • update QuantumESPRESSO easyblock to handle v6.7 (PR #2319)
      • new software
        • (none)
      • changes
        • create less temporary directories for TensorFlow by (only) using --output_user_root (PR #2293)
    • easyconfigs
      • ~50 merged easyconfig PRs since last conf call!
      • bug fixes
        • fix for "error: 'runtime_error' is not a member of 'std'" in qtlocation (Qt5) (PR #12012)
        • fix download URL for UDUNITS 2.2.26 (PR #12020)
          • this raises the need again for an EasyBuild source mirror (for disappearing sources)
          • only for software we're allow to redistribute...
          • what about export restrictions?
          • see GNU mirrors?
        • add patch to fix version for bam-readcount 0.8.0 (PR #12075)
        • fix vector mul and div with broadcasts in -masm=intel mode in GCCcore v9.3.0 (PR #12065)
          • broken avx512f instructions, affect PyTorch
      • enhancements
        • enable tests for most recent Bazel versions (PR #11894)
      • new software
        • (various, nothing 'major')
      • software updates
      • changes
        • orca renamed to plotly-orca (to avoid confusion with ORCA) (PR #12015)
        • update Java/1.8 wrapper to Java 1.8.0_281 (PR #12088)
  • to merge/fix/tackle soon
    • framework (v4.3.3 milestone)
      • bug fixes
        • (BLOCKER for v4.3.3) avoid initializing Toolchain instance when taking into account toolchain dependencies for templates (PR #3560)
        • Performance improvements for easyconfig parsing (PR #3555)
        • Avoid module use in Lmod if possible to allow faster execution (PR #3557)
        • Avoid metadata greedy behaviour (mostly relevant for Cray systems) (PR #3559)
      • enhancements
        • (NICE TO HAVE for v4.3.3) add support for using customized HTTP headers in download_file (PR #2472)
          • important for ITER.org (see EUM'21 talk)
        • support additional features in easystack files (see issues #3468, #3512, #3513, #3516)
        • add Generation module naming scheme (PR #3547)
        • Allow use of alternate envvar(s) to $HOME for user modules (PR #3558)
    • easyblocks (v4.3.3 milestone)
      • bug fixes
        • correctly determine path to active binutils in TensorFlow easyblock (PR #2218) [EESSI, RPATH]
        • fix taking into account --sysroot when installing/using CMake [EESSI] (PR #2247 or PR #2248, latter is best option?)
        • treat files/directories of unpacked sources equally in PackedBinary generic easyblock (PR #2306)
        • make sure the installation of libiberty.a in the binutils easyblock goes into a populated directory (PR #2308)
        • Fix for building GCC with --sysroot on ppc64le (PR #2315)
        • RPackages: Change to using R_LIBS_SITE when installing RPackages (PR #2326)
        • pass down compilation flags from build environment for ESMF (PR #2325)
      • enhancements
        • enhance OpenBLAS easyblock to make it aware of optarch (PR #1946)
        • add support for skipping steps in Python packages installed as extension + print progress on individual steps for installing Python packages as extensions (PR #2290)
        • (BLOCKER for v4.3.3) add support for running TensorFlow CPU and GPU tests separately and enhance test failure reporting (PR #2312)
        • (NICE TO HAVE for v4.3.3) update impi easyblock for impi 2021.x (oneAPI) (PR #2313)
        • enhance test and install step of CMakePythonPackage easyblock (PR #2318)
        • pass $CXXFLAGS to PDT's configure script via '-useropt' (PR #2324)
        • Add sanity check commands to GCC (including LTO support) (PR #2322)
        • Added support for OpenFOAM v2012 layout (PR #2321)
        • Update optional feature support of TensorFlow (PR #2314)
        • Add support for including PTX code in PyTorch (PR #2328)
      • changes
        • (nothing)
      • new software
        • add custom easyblock for:
          • ADIOS (PR #2070)
          • JAX (PR #2262)
          • RELION (PR #2274)
            • does adding an EasyBuild configuration option to specify the system job scheduler used make sense?
            • yes, if it can help easyblocks to make proper decisions
            • can also be useful for OpenMPI to opt-in to Slurm integration
        • (NICE TO HAVE for v4.3.3) new generic easyblock for Intel oneAPI compilers (PR #2305)
          • using intel-compilers as software name, including icc/icpc/ifort (classic compilers) + icx/icpx/ifx/dpcpp (oneAPI compilers)
    • easyconfigs (v4.3.3 milestone)
      • bug fixes
        • fix libxml2 dep for Clang (part of PR #12013)
        • p4est-2.2: Add patches to fix test problems and use a source_url that contains a complete tar-file (PR #12028
      • enhancements *
      • new software
        • (NICE TO HAVE FOR v4.3.3) Intel oneAPI
          • there will be no further updates to Intel Parallel Studio (apparently)
          • intel-compilers v2020.1 (PR #11982)
      • software updates
        • (NICE TO HAVE FOR v4.3.3) TensorFlow 2.4.1 (PR #11637)
        • Bazel v3.7.1 with GCCcore/10.2.0 (PR #11810)
          • requires patch?
          • see Alexander's patch for Bazel v3.7.1 included in (PR #11637)
        • (NICE TO HAVE FOR v4.3.3) impi v2021.1 (PR #12026)
        • TensorFlow v1.15.5 (PR #12037)

Review of EasyBuild User Meeting

  • (preliminary) EUM'21 evaluation results available at https://users.ugent.be/~kehoste/eum21_survey_20210203.pdf
  • very positive feedback overall!
  • interesting suggestion w.r.t a "hackathon" meeting
    • proposed issues to work on, then group together
    • maybe some time in summer (August?)
    • relatively limited set of topics to work on, to keep effort focused
    • try to have at least one maintainer per "working group"

Switching from OpenBLAS to BLIS in foss/2021a

  • Collect low-level performance results on a variety of CPUs
  • Also test with real-world applications which heavily rely on BLAS
    • CP2K, VASP, QuantumESPRESSO, ABINIT, Amber (FFTW), GROMACS (FFTW), NWChem, SciPy-bundle, Octave, finite-element codes (OpenFOAM?), ...
    • using gobff toolchain (currently based on foss/2020a)
      • should be updated to be in line with foss/2020b
  • "working group" for properly evaluating BLIS
    • Åke, Jörg (CP2K), Bart, Sebastian, Robert Schade (CP2K, VASP)
    • Robert Schade already has experience with benchmarking CP2K with BLIS
    • depends a lot on the calculations whether CP2K uses BLAS (for some stuff it uses libxsmm)
    • Sebastian ran into some trouble with SciPy-bundle that includes matplotlib
    • OpenFOAM is mostly memory bandwidth limited, so not a good test case to evaluate BLIS
    • Kenneth will set up a doodle to plan a focused meeting w.r.t. BLIS evaluation

Q&A

  • Bart: anyone working on Trilinos (other than Maxime)?
    • Åke tried "recently" (2019-ish), but gave up eventually
    • dies after hours of compiling, bundles lots of stuff...
    • required dep for FENICS, deal.II (optional, could also be PETSc)
Clone this wiki locally