Skip to content

Commit

Permalink
resources: updated pannotia documentation for compile commands (#15)
Browse files Browse the repository at this point in the history
The docker command for compiling pannotia tests needs to have visibility
to m5ops.
  • Loading branch information
mattsinc authored Nov 27, 2023
2 parents c8f01b7 + 7ba26c5 commit e0da918
Show file tree
Hide file tree
Showing 24 changed files with 398 additions and 218 deletions.
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,7 @@ git clone https://github.com/gem5/gem5-resources.git

```
cd src/gpu/square
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make gfx8-apu
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make gfx8-apu
```

The compiled binary can be found in `src/gpu/square/bin`
Expand Down Expand Up @@ -409,7 +409,7 @@ To compile:

```
cd src/gpu/hsa-agent-pkt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make gfx8-apu
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make gfx8-apu
```

The compiled binary can be found in `src/gpu/hsa-agent-pkt/bin`
Expand All @@ -431,7 +431,7 @@ Certain apps aren't included due to complexities with either ROCm or Docker

```
cd src/gpu/hip-samples
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make
```

Individual programs can be made by specifying the name of the program
Expand Down Expand Up @@ -469,7 +469,7 @@ and the other command-line arguments for use with heterosync.
## Compilation
```
cd src/gpu/heterosync
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make release-gfx8
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make release-gfx8
```

The release-gfx8 target builds for gfx801, a GCN3-based APU, and gfx803, a
Expand All @@ -489,7 +489,7 @@ provided is for use with the gpu-compute model of gem5.
## Compilation and Running
```
cd src/gpu/lulesh
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make
```

By default, the Makefile builds for gfx801, and is placed in the
Expand All @@ -500,7 +500,7 @@ architecture. To build GCN3_X86:

```
# Working directory is your gem5 directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
```

The following command shows how to run lulesh
Expand All @@ -513,7 +513,7 @@ are equivalent to `--options="1.0e-2 10"`.

```
# Assuming gem5 and gem5-resources are in your working directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --mem-size=8GB --benchmark-root=gem5-resources/src/gpu/lulesh/bin -clulesh
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --mem-size=8GB --benchmark-root=gem5-resources/src/gpu/lulesh/bin -clulesh
```

## Pre-built binary
Expand Down Expand Up @@ -584,8 +584,8 @@ commands do this by using `-v ${PWD}:${PWD}` in the docker run commands

```
cd src/gpu/DNNMark
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu ./setup.sh HIP
docker run --rm -v ${PWD}:${PWD} -w ${PWD}/build -u $UID:$GID gcr.io/gem5-test/gcn-gpu make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu ./setup.sh HIP
docker run --rm -v ${PWD}:${PWD} -w ${PWD}/build -u $UID:$GID ghcr.io/gem5/gcn-gpu make
```

DNNMark uses MIOpen kernels, which are unable to be compiled on-the-fly in gem5.
Expand All @@ -595,7 +595,7 @@ benchmarks for a gfx801 GPU with 4 CUs by default
To generate the MIOpen kernels:
```
cd src/gpu/DNNMark
docker run --rm -v ${PWD}:${PWD} -v${PWD}/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} gcr.io/gem5-test/gcn-gpu python3 generate_cachefiles.py cachefiles.csv [--gfx-version={gfx801,gfx803}] [--num-cus=N]
docker run --rm -v ${PWD}:${PWD} -v${PWD}/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} ghcr.io/gem5/gcn-gpu python3 generate_cachefiles.py cachefiles.csv [--gfx-version={gfx801,gfx803}] [--num-cus=N]
```

Due to the large amounts of memory that need to be set up for DNNMark, we have
Expand All @@ -614,13 +614,13 @@ GCN3_X86 architecture.
To build GCN3_X86:
```
# Working directory is your gem5 directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
```

To run one of the benchmarks (fwd softmax) in gem5:
```
# Assuming gem5 and gem5-resources are sub-directories of the current directory
docker run --rm -v ${PWD}:${PWD} -v ${PWD}/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} gcr.io/gem5-test/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_softmax -cdnnmark_test_fwd_softmax --options="-config gem5-resources/src/gpu/DNNMark/config_example/softmax_config.dnnmark -mmap gem5-resources/src/gpu/DNNMark/mmap.bin"
docker run --rm -v ${PWD}:${PWD} -v ${PWD}/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} ghcr.io/gem5/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_softmax -cdnnmark_test_fwd_softmax --options="-config gem5-resources/src/gpu/DNNMark/config_example/softmax_config.dnnmark -mmap gem5-resources/src/gpu/DNNMark/mmap.bin"
```


Expand All @@ -635,7 +635,7 @@ a sample of the typical memory access patterns of FLAG.

```
cd src/gpu/pennant
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu make
```

By default, the binary is built for gfx801 and is placed in `src/gpu/pennant/build`.
Expand All @@ -648,7 +648,7 @@ command shows how to run the sample `noh`:

```
# Assuming gem5 and gem5-resources are in your working directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/pennant/build -cpennant --options="gem5-resources/src/gpu/pennant/test/noh/noh.pnt"
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/pennant/build -cpennant --options="gem5-resources/src/gpu/pennant/test/noh/noh.pnt"
```

The output gets placed in `src/gpu/pennant/test/noh/`, and the file `noh.xy`
Expand Down Expand Up @@ -841,7 +841,7 @@ We provide a docker image with a pre-loaded SPARC cross compiler. To use:

```
cd src/insttest
docker run --volume $(pwd):$(pwd) -w $(pwd) --rm gcr.io/gem5-test/sparc64-gnu-cross:latest make
docker run --volume $(pwd):$(pwd) -w $(pwd) --rm ghcr.io/gem5/sparc64-gnu-cross:latest make
```

The compiled binary can be found in `src/insttest/bin`.
Expand Down
6 changes: 3 additions & 3 deletions src/gpu-fs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,9 @@ You can obtain the `vmlinux-5.4.0-105-generic` kernel using the following path f
It is sometimes useful to build your own application and run in gem5. A docker is provided to allow users to build applications without needing to install ROCm locally. A pre-built docker image is available on gcr.io. This image can be pulled then used to build as follows:

```sh
docker pull gcr.io/gem5-test/gpu-fs:latest
docker pull ghcr.io/gem5/gpu-fs:latest
cd /path/to/gem5-resources/src/gpu/square
docker run --rm -v ${PWD}:${PWD} -w ${PWD} gcr.io/gem5-test/gpu-fs:latest bash -c 'make clean; HCC_AMDGPU_TARGET=gfx900 make'
docker run --rm -v ${PWD}:${PWD} -w ${PWD} ghcr.io/gem5/gpu-fs:latest bash -c 'make clean; HCC_AMDGPU_TARGET=gfx900 make'
```

Currently only Vega (gfx900) is supported for full system GPU simulation in gem5. It is therefore required to tell the compiler to build for this ISA using the HCC_AMDGPU_TARGET environment variable. Otherwise, the command to build the application is the same as if you were building locally.
Expand Down Expand Up @@ -130,4 +130,4 @@ You need to run something like the following (as root).
# echo -1 > /proc/sys/kernel/perf_event_paranoid
```

- If you encounter `Qemu stderr: qemu-system-x86_64: failed to initialize KVM: Permission denied`, the issue is likely related to permission on /dev/kvm
- If you encounter `Qemu stderr: qemu-system-x86_64: failed to initialize KVM: Permission denied`, the issue is likely related to permission on /dev/kvm
10 changes: 5 additions & 5 deletions src/gpu/DNNMark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ won't be able to link against the library. The example commands do this by using
`-v ${PWD}:${PWD}` in the docker run commands
```
cd src/gpu/DNNMark
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 ./setup.sh HIP
docker run --rm -v ${PWD}:${PWD} -w ${PWD}/build -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 ./setup.sh HIP
docker run --rm -v ${PWD}:${PWD} -w ${PWD}/build -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 make
```

DNNMark uses MIOpen kernels, which are unable to be compiled on-the-fly in gem5.
Expand All @@ -38,7 +38,7 @@ benchmarks for a gfx801 GPU with 4 CUs by default
To generate the MIOpen kernels:
```
cd src/gpu/DNNMark
docker run --rm -v ${PWD}:${PWD} -v${PWD}/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} gcr.io/gem5-test/gcn-gpu:v22-1 python3 generate_cachefiles.py cachefiles.csv [--gfx-version={gfx801,gfx803}] [--num-cus=N]
docker run --rm -v ${PWD}:${PWD} -v${PWD}/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} ghcr.io/gem5/gcn-gpu:v22-1 python3 generate_cachefiles.py cachefiles.csv [--gfx-version={gfx801,gfx803}] [--num-cus=N]
```

Due to the large amounts of memory that need to be set up for DNNMark, we have
Expand All @@ -56,13 +56,13 @@ DNNMark is a GPU application, which requires that gem5 is built with the GCN3_X8
To build GCN3_X86:
```
# Working directory is your gem5 directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
```

To run one of the benchmarks (fwd softmax) in gem5:
```
# Assuming gem5 and gem5-resources are sub-directories of the current directory
docker run --rm -v ${PWD}:${PWD} -v ${PWD}/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} gcr.io/gem5-test/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_softmax -cdnnmark_test_fwd_softmax --options="-config gem5-resources/src/gpu/DNNMark/config_example/softmax_config.dnnmark -mmap gem5-resources/src/gpu/DNNMark/mmap.bin"
docker run --rm -v ${PWD}:${PWD} -v ${PWD}/gem5-resources/src/gpu/DNNMark/cachefiles:/root/.cache/miopen/2.9.0 -w ${PWD} ghcr.io/gem5/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --benchmark-root=gem5-resources/src/gpu/DNNMark/build/benchmarks/test_fwd_softmax -cdnnmark_test_fwd_softmax --options="-config gem5-resources/src/gpu/DNNMark/config_example/softmax_config.dnnmark -mmap gem5-resources/src/gpu/DNNMark/mmap.bin"
```

Information from the original DNNMark README included below.
Expand Down
2 changes: 1 addition & 1 deletion src/gpu/halo-finder/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM gcr.io/gem5-test/gcn-gpu:latest
FROM ghcr.io/gem5/gcn-gpu:latest
RUN apt-get update && apt-get -y install libopenmpi-dev libomp-dev

ENV HCC_AMDGPU_TARGET="gfx801,gfx803,gfx900"
Expand Down
4 changes: 2 additions & 2 deletions src/gpu/heterosync/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ command-line arguments for use with heterosync.
## Compilation
```
cd src/gpu/heterosync
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 make release-gfx8
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 make release-gfx8
```

The release-gfx8 target builds for gfx801, a GCN3-based APU, and gfx803, a
Expand All @@ -33,7 +33,7 @@ that are currently unsupported in gem5.
HeteroSync has multiple applications that can be run (see below). For example, to run sleepMutex with 10 ld/st per thread, 16 WGs, and 4 iterations of the critical section:

```
docker run -u $UID:$GID --volume $(pwd):$(pwd) -w $(pwd) gcr.io/gem5-test/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n 3 -c bin/allSyncPrims-1kernel --options="sleepMutex 10 16 4"
docker run -u $UID:$GID --volume $(pwd):$(pwd) -w $(pwd) ghcr.io/gem5/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n 3 -c bin/allSyncPrims-1kernel --options="sleepMutex 10 16 4"
```

## Pre-built binary
Expand Down
2 changes: 1 addition & 1 deletion src/gpu/hip-samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Compiling the HIP samples, compiling the GCN3_X86 gem5, and running the HIP samp

```
cd src/gpu/hip-samples
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 make
```

Individual programs can be made by specifying the name of the program
Expand Down
2 changes: 1 addition & 1 deletion src/gpu/hsa-agent-pkt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ To compile:

```
cd src/gpu/hsa-agent-pkt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 make gfx8-apu
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 make gfx8-apu
```

The compiled binary can be found in `src/gpu/hsa-agent-pkt/bin`
6 changes: 3 additions & 3 deletions src/gpu/lulesh/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Compiling LULESH, compiling the GCN3_X86 gem5, and running LULESH on gem5 is dep
## Compilation and Running
```
cd src/gpu/lulesh
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 make
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 make
```

By default, the makefile builds for gfx801, and is placed in the `src/gpu/lulesh/bin` folder.
Expand All @@ -30,7 +30,7 @@ To build GCN3_X86:

```
# Working directory is your gem5 directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 scons -sQ -j$(nproc) build/GCN3_X86/gem5.opt
```

The following command shows how to run lulesh
Expand All @@ -42,7 +42,7 @@ to the run command. The default arguments are equivalent to `--options="1.0e-2 1

```
# Assuming gem5 and gem5-resources are in your working directory
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID gcr.io/gem5-test/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --mem-size=8GB --benchmark-root=gem5-resources/src/gpu/lulesh/bin -clulesh
docker run --rm -v ${PWD}:${PWD} -w ${PWD} -u $UID:$GID ghcr.io/gem5/gcn-gpu:v22-1 gem5/build/GCN3_X86/gem5.opt gem5/configs/example/apu_se.py -n3 --mem-size=8GB --benchmark-root=gem5-resources/src/gpu/lulesh/bin -clulesh
```

## Pre-built binary
Expand Down
7 changes: 7 additions & 0 deletions src/gpu/pannotia/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,10 @@ shortdoc: >
---

This folder and its subfolders contain each of the 9 Pannotia benchmarks (there are 6 folders because Color, and PageRank, SSSP each have 2 versions). All of these benchmarks have been ported from the prior CUDA and OpenCL variants to HIP, and validated on a Vega-class AMD GPU. See each application's README for details on how to compile and run them in gem5 using the GCN3 GPU model.

## Compiling m5ops
To compile the m5ops:
```
cd gem5/util/m5
scons build/x86/out/m5
```
6 changes: 3 additions & 3 deletions src/gpu/pannotia/bc/Makefile.gem5-fusion
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ HIP_PATH ?= /opt/rocm/hip
HIPCC = $(HIP_PATH)/bin/hipcc

# these are needed for m5ops
GEM5_PATH ?= /path/to/gem5
CFLAGS += -I$(GEM5_PATH)/include -I../graph_parser
LDFLAGS += -L$(GEM5_PATH)/util/m5/build/x86/out -lm5
GEM5_ROOT ?= /path/to/gem5
CFLAGS += -I$(GEM5_ROOT)/include -I../graph_parser
LDFLAGS += -L$(GEM5_ROOT)/util/m5/build/x86/out -lm5

BIN_DIR ?= ./bin

Expand Down
Loading

0 comments on commit e0da918

Please sign in to comment.