Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dev): setup devcontainer for developers #4263

Open
wants to merge 5 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM mcr.microsoft.com/devcontainers/cpp:1-ubuntu-24.04

COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
35 changes: 35 additions & 0 deletions .devcontainer/READMD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# DeePMD-kit devcontainer environment

This [devcontainer](https://vscode.js.cn/docs/devcontainers/devcontainer-cli) environment setups Python and C++ environment to develop DeePMD-kit.
One can setup locally or use [GitHub Codespaces](https://docs.github.com/en/codespaces) by clicking the Code button on the DeePMD-kit repository page.
The whole setup process requires about 10 minutes, so one needs to be patient.

## Python environment

The following packages are installed into the Python environment `.venv`:

- DeePMD-kit (in edit mode)
- Backends including TensorFlow, PyTorch, JAX
- LAMMPS
- MPICH
- CMake
- pre-commit (including hooks)
- Test packages including pytest
- Doc packages including sphinx

## C++ interface

The C++ interface with TensorFlow and PyTorch support is installed into `dp` directory.

When calling and debuging LAMMPS with DeePMD-kit, use the following scripts instead of the regular `lmp`:

- `.devcontainer/lmp`
- `.devcontainer/gdb_lmp`

## Rebuild

Usually the Python package does not need to reinstall.
But when one wants to recompile the C++ code, the following scripts can be executed.

- `.devcontainer/build_cxx.sh`
- `.devcontainer/build_py.sh`
21 changes: 21 additions & 0 deletions .devcontainer/build_cxx.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash
set -ev

NPROC=$(nproc --all)
SCRIPT_PATH=$(dirname $(realpath -s $0))

njzjz marked this conversation as resolved.
Show resolved Hide resolved
export CMAKE_PREFIX_PATH=${SCRIPT_PATH}/../libtorch
njzjz marked this conversation as resolved.
Show resolved Hide resolved
TENSORFLOW_ROOT=$(python -c 'import importlib,pathlib;print(pathlib.Path(importlib.util.find_spec("tensorflow").origin).parent)')
njzjz marked this conversation as resolved.
Show resolved Hide resolved

mkdir -p ${SCRIPT_PATH}/../buildcxx/
cd ${SCRIPT_PATH}/../buildcxx/
cmake -D ENABLE_TENSORFLOW=ON \
-D ENABLE_PYTORCH=ON \
-D CMAKE_INSTALL_PREFIX=${SCRIPT_PATH}/../dp/ \
-D LAMMPS_VERSION=stable_29Aug2024_update1 \
-D CMAKE_BUILD_TYPE=Debug \
-D BUILD_TESTING:BOOL=TRUE \
-D TENSORFLOW_ROOT=${TENSORFLOW_ROOT} \
${SCRIPT_PATH}/../source
njzjz marked this conversation as resolved.
Show resolved Hide resolved
cmake --build . -j${NPROC}
cmake --install .
njzjz marked this conversation as resolved.
Show resolved Hide resolved
8 changes: 8 additions & 0 deletions .devcontainer/build_py.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
set -ev
njzjz marked this conversation as resolved.
Show resolved Hide resolved

SCRIPT_PATH=$(dirname $(realpath -s $0))
cd ${SCRIPT_PATH}/..
njzjz marked this conversation as resolved.
Show resolved Hide resolved

uv sync --dev --python 3.12 --extra cpu --extra torch --extra jax --extra lmp --extra test --extra docs
pre-commit install
njzjz marked this conversation as resolved.
Show resolved Hide resolved
17 changes: 17 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "DeePMD-kit",
"build": {
"dockerfile": "Dockerfile"
},
"features": {
"ghcr.io/devcontainers/features/github-cli:1": {}
},
njzjz marked this conversation as resolved.
Show resolved Hide resolved
"postCreateCommand": ".devcontainer/build_py.sh && .devcontainer/download_libtorch.sh && .devcontainer/build_cxx.sh && pre-commit install-hooks",
njzjz marked this conversation as resolved.
Show resolved Hide resolved
"remoteEnv": {
"PATH": "${containerEnv:PATH}:${containerWorkspaceFolder}/.venv/bin",
"DP_ENABLE_PYTORCH": "1",
"DP_VARIANT": "cpu",
"LMP_CXX11_ABI_0": "1",
"UV_EXTRA_INDEX_URL": "https://download.pytorch.org/whl/cpu"
}
njzjz marked this conversation as resolved.
Show resolved Hide resolved
}
8 changes: 8 additions & 0 deletions .devcontainer/download_libtorch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
set -ev
njzjz marked this conversation as resolved.
Show resolved Hide resolved

SCRIPT_PATH=$(dirname $(realpath -s $0))
cd ${SCRIPT_PATH}/..
njzjz marked this conversation as resolved.
Show resolved Hide resolved

wget https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.5.0%2Bcpu.zip -O ~/libtorch.zip
njzjz marked this conversation as resolved.
Show resolved Hide resolved
unzip ~/libtorch.zip
njzjz marked this conversation as resolved.
Show resolved Hide resolved
9 changes: 9 additions & 0 deletions .devcontainer/gdb_lmp
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
SCRIPT_PATH=$(dirname $(realpath -s $0))
njzjz marked this conversation as resolved.
Show resolved Hide resolved

export CMAKE_PREFIX_PATH=${SCRIPT_PATH}/../libtorch
TENSORFLOW_ROOT=$(python -c 'import importlib,pathlib;print(pathlib.Path(importlib.util.find_spec("tensorflow").origin).parent)')
njzjz marked this conversation as resolved.
Show resolved Hide resolved

env LAMMPS_PLUGIN_PATH=${SCRIPT_PATH}/../dp/lib/deepmd_lmp \
LD_LIBRARY_PATH=${SCRIPT_PATH}/../dp/lib:${CMAKE_PREFIX_PATH}/lib:${TENSORFLOW_ROOT} \
gdb ${SCRIPT_PATH}/../.venv/lib/python3.12/site-packages/lammps/lmp "$@"
njzjz marked this conversation as resolved.
Show resolved Hide resolved
9 changes: 9 additions & 0 deletions .devcontainer/lmp
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
SCRIPT_PATH=$(dirname $(realpath -s $0))

export CMAKE_PREFIX_PATH=${SCRIPT_PATH}/../libtorch
TENSORFLOW_ROOT=$(python -c 'import importlib,pathlib;print(pathlib.Path(importlib.util.find_spec("tensorflow").origin).parent)')
njzjz marked this conversation as resolved.
Show resolved Hide resolved

env LAMMPS_PLUGIN_PATH=${SCRIPT_PATH}/../dp/lib/deepmd_lmp \
LD_LIBRARY_PATH=${SCRIPT_PATH}/../dp/lib:${CMAKE_PREFIX_PATH}/lib:${TENSORFLOW_ROOT} \
${SCRIPT_PATH}/../.venv/bin/lmp "$@"
njzjz marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,8 @@ build_c_tests
build_c/
libdeepmd_c/
.uv/
libtorch/
uv.lock
buildcxx/
node_modules/
*.bib.original
20 changes: 20 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -136,16 +136,28 @@ cu12 = [
"nvidia-cuda-nvcc-cu12",
]
jax = [
# below is a funny workaround for
# https://github.com/astral-sh/uv/issues/8601
'jax>=0.4.33;python_version>="3.10"',
'jax>=0.4.33;python_version>="3.10"',
'flax>=0.10.0;python_version>="3.10"',
'flax>=0.10.0;python_version>="3.10"',
'orbax-checkpoint;python_version>="3.10"',
'orbax-checkpoint;python_version>="3.10"',
# The pinning of ml_dtypes may conflict with TF
# 'jax-ai-stack;python_version>="3.10"',
]

[tool.deepmd_build_backend.scripts]
dp = "deepmd.main:main"

[dependency-groups]
dev = [
"pre-commit",
"cmake",
"mpich",
]

[tool.setuptools_scm]

[tool.scikit-build]
Expand Down Expand Up @@ -428,3 +440,11 @@ select = [
"TOR1",
"TOR2",
]

[tool.uv.sources]
mpich = { index = "mpi4py" }

[[tool.uv.index]]
name = "mpi4py"
url = "https://pypi.anaconda.org/mpi4py/simple"
explicit = true