build instructions for M2 mac

[samuelpmishLLNL]

I was trying to install serac's dependencies on a new M2 MacBook Pro and this is just a record of the issues I ran into, and what eventually worked (for me).

The machine is running Ventura, and using homebrew as a package manager (rather than MacPorts). Our installation instructions are given in terms of using MacPorts as a package manager, but I didn't want to install both and risk having them step on each other's toes.

There is no spack.yaml for ventura, so I have to create one. Since I didn't want to describe where all the existing libraries are installed, I opted to use a minimal file (and let spack build practically everything). I couldn't figure out how to get spack to use the default apple-clang compiler, so I did

brew install llvm@15

to install llvm + clang-15, and made a spack.yaml file:

spack:
  packages:
    all:
      compiler: [clang, gcc]
      providers:
        blas: [netlib-lapack]
        lapack: [netlib-lapack]
        mpi: [openmpi]

  compilers:
  - compiler:
      environment: {}
      extra_rpaths: []
      flags: {}
      modules: []
      operating_system: ventura
      paths:
        cc: /path/to/homebrew/llvm@15/bin/clang
        cxx: /path/to/homebrew/llvm@15/bin/clang++
        f77: /path/to/homebrew/bin/gfortran
        fc: /path/to/homebrew/bin/gfortran
      spec: [email protected]
      target: aarch64

When invoking

./scripts/uberenv/uberenv.py --prefix=../serac_libs --spack-env-file=spack.yaml --spec="%[email protected]"

two of the packages failed to build:

Problem with raja

  >> 40    CMake Error at /Users/mish2/code/serac_libs/spack/opt/spack/darwin-ventura-m2/clang-15.0.7/cmake-3.25.2-5bwpbfkbpqgk52jkmjn3u472qt7qwloj/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:23
           0 (message):
     41      Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)

I thought the RAJA problem was related to not having openmp headers / libs installed, since Apple is notoriously anti-OpenMP. I tried brew install libomp, but that didn't fix the problem. Instead, I ended up appending ~openmp to the spec to turn this feature off.

Problem with hdf5

  >> 434    /var/folders/p_/35zffh5d0ns1zpr0ddx1fn4r000trx/T/mish2/spack-stage/spack-stage-hdf5-1.8.21-ybehpka6zktzpjoa2esyzryufispfwaw/spack-src/src/H5Aint.c:2102:21: error: incompatible integer to pointer conv
            ersion assigning to 'H5A_t *' (aka 'struct H5A_t *') from 'int' [-Wint-conversion]
     435                        HGOTO_ERROR(H5E_ATTR, H5E_CANTALLOC, FAIL, "memory allocation failed")
     436                        ^                                    ~~~~
     437    /var/folders/p_/35zffh5d0ns1zpr0ddx1fn4r000trx/T/mish2/spack-stage/spack-stage-hdf5-1.8.21-ybehpka6zktzpjoa2esyzryufispfwaw/spack-src/src/H5Eprivate.h:67:4: note: expanded from macro 'HGOTO_ERROR'
     438       HGOTO_DONE(ret_val)   

Without patching the sources, I don't know how to avoid this compilation error, so I ended up doing brew install [email protected] and adding an entry in the spack.yaml file so that it now looks like:

spack:
  packages:
    all:
      compiler: [clang, gcc]
      providers:
        blas: [netlib-lapack]
        lapack: [netlib-lapack]
        mpi: [openmpi]

    hdf5:
      buildable: false
      externals:
      - spec: [email protected]
        prefix: /path/to/homebrew/[email protected]/

  compilers:
  - compiler:
      environment: {}
      extra_rpaths: []
      flags: {}
      modules: []
      operating_system: ventura
      paths:
        cc: /path/to/homebrew/llvm@15/bin/clang
        cxx: /path/to/homebrew/llvm@15/bin/clang++
        f77: /path/to/homebrew/bin/gfortran
        fc: /path/to/homebrew/bin/gfortran
      spec: [email protected]
      target: aarch64

Note: homebrew actually installed [email protected], but if I put that version in the spack.yaml, spack produced errors about mismatched versions, so I lied in the spack.yaml and said it was actually 1.8.21.

Now, when running

./scripts/uberenv/uberenv.py --prefix=../serac_libs --spack-env-file=spack.yaml --spec="%[email protected]~openmp"

everything eventually built successfully (albeit with openmp disabled)