Skip to content

Conference call notes 20210414

Jump to bottom
ocaisa edited this page Apr 13, 2021 · 12 revisions

(back to Conference calls)

Notes on the 170th EasyBuild conference call, Wednesday April 14th 2021 (08:00 UTC)

Attendees

Alphabetical list of attendees ():

  • Mikael Öhman (Chalmers University of Technology, Sweden)

  • Sebastian Achilles (Jülich Supercomputing Centre, Germany)

  • Simon Branford (University of Birmingham, UK)

  • Steve Davis (Smith College, US)

  • Fotis Georgatos (SDSC, Switzerland)

  • Alexander Grund (TU Dresden, Germany)

  • Kenneth Hoste (HPC-UGent, Belgium)

  • Adam Huffman (Big Data Institute, Oxford, UK)

  • Terje Kvernes (University of Oslo, Norway)

  • Kurt Lust (Univ. of Antwerp, Belgium + LUMI User Support Team)

  • Bart Oldeman (Compute Canada)

  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

Agenda

  • update on recent developments
  • 2021a update of common toolchains
    • outlook to component versions
    • BLAS/LAPACK component: OpenBLAS vs BLIS, maybe FlexiBLAS?
    • collapsing foss and fosscuda toolchains
  • bintray
  • Q&A

Recent developments

  • next release: a month or so since we just released last week
    • project for next release: (not created yet)
      • to maintainers: add issues/PRs you consider important there!
  • recent changes
    • framework
      • bug fixes
        • Catch problems early on if --github-user is not specified for --new-pr & co (PR #3644)
      • enhancements
        • Avoid module call for unuse() for Lmod and set $MODULEPATH directly (PR #3633)
        • update validate_github_token function to accept GitHub token in new format (PR #3632)
        • mention easyblocks PR in gist when uploading test report for it + fix clean_gists.py script (PR #3622
        • add templates for architecture independent Python wheels (PR #3618
      • changes
        • test bootstrap script in separate workflow, and limit test configurations a bit (PR #3646)
        • deprecate --accept-eula, rename to --accept-eula-for + also accept regular expression value (PR #3630
    • easyblocks
      • bug fixes
        • (nothing major)
      • enhancements
        • make WRF and WPS easyblocks aware of (pre)configopts (PR #2361)
        • enhance BWA easyblock: pass compiler flags + use filetools functions (PR #2368)
        • enhance PythonPackage easyblock to catch faulty version (0.0.0) for installed Python packages (PR #2367)
      • changes
        • use PYPI_SOURCE as the default for source_urls of PythonPackage (PR #2364 + PR #2370)
          • easyconfig files were updated accordingly (see for example PR #12451)
          • tests were enhanced to fail when [PYPI_SOURCE] is used in source_urls for Python packages (PR #12456)
    • easyconfigs
      • over 50 merged easyconfig PRs since last conf call
      • bug fixes
        • (nothing major)
      • enhancements
        • (nothing major)
      • new software
        • (nothing major)
      • noteworthy software updates
      • noteworthy changes
        • (none)
  • to merge/fix/tackle soon
    • framework
      • bug fixes
        • performance improvements for easyconfig parsing (PR #3555)
        • avoid module use in Lmod if possible to allow faster execution (PR #3557)
      • enhancements
        • support additional features in easystack files
          • support for filtering via labels (PR 3620)
        • add templates for architecture independent Python wheels (PR #3618)
        • show the easyblocks repo and PR number when uploading a test report against that (PR #3622)
      • changes
        • refactor the CI configuration to use inclusion instead of exclusion (PR #3616)
          • nice cleanup, but will require changing which test configuration must pass...
          • implies that tests for all framework PRs will need to be re-triggered after this is merged
    • easyblocks
      • bug fixes
        • treat files/directories of unpacked sources equally in PackedBinary (PR #2306)
        • make the CUDA stub libs take preference over system libs when linking (PR #2373)
        • also set $TORCH_CUDA_ARCH_LIST for PyTorch tests (PR #2363)
      • enhancements
        • enhance CUDA support in CP2K easyblock (WIP) (PR #2349)
          • this could use a review
          • currently requires a single value in --cuda-compute-capabilities EasyBuild configuration option or cuda_compute_capabilities easyconfig parameter
          • do we need custom easyconfig parameters to easily enable/disable the different GPU capabilities supported by CP2K?
        • enhance cuDNN, CUDA, and Java easyblocks to support aarch64 (PR #2356)
        • add Java wrapper support to OpenMPI (PR #2360)
          • still missing a matching easyconfig PR that leverages this?
        • enable installation of samples for CUDA > 10.1 (PR #2374)
        • don't unpack *.whl files by default in generic PythonPackage easyblock (PR #2366)
      • changes
        • set $R_LIBS_SITE rather than $R_LIBS when installing R packages (PR #2326)
      • new software
        • new easyblock for NCCL (built from source) (PR #2337)
        • custom easyblock for FlexiBLAS (PR #2369)
    • easyconfigs
      • bug fixes
        • make sure that VTK 9.0.1 links to libpython by configuring with -DVTK_PYTHON_OPTIONAL_LINK=OFF (PR #12480)
        • fix numpy tests for recent SciPy-bundle easyconfig on POWER (PR #12481)
      • enhancements
        • require sanity_pip_check for all Python package/bundles touched in PRs (PR #12464)
          • also for stuff installed with python setup.py install
          • because pip check will also see those packages
          • if we don't enforce sanity_pip_check always, pip check may fail if dependency was installed via setup.py and was broken
      • new software
      • software updates
        • PyTorch 1.8.0 (PR #12347)
          • PyTorch only test against MKL which is probably connected to failures in their testsuite
          • should probably be bumped to 1.8.1
        • goblf/2020b (PR #12381)

2021a update of common toolchains

  • outlook to component versions
    • GCC 10.3 (ready to go?)
    • OpenMPI 4.1.1 (out soon)
    • Intel oneAPI versions of compilers, MPI, MKL?
    • Python version?
  • BLAS/LAPACK component: OpenBLAS vs BLIS, maybe FlexiBLAS?
  • collapsing foss and fosscuda toolchains
  • HMNS should be updated to be aware of intel-compilers component

New download URLs for bintray

Q&A

  • (none)
Clone this wiki locally