resources: Add OpenSBI Firmware as a Bootloader resource (#13)

This patch documents how to compile OpenSBI's `FW_JUMP` and use it as a
bootloader in gem5.
The document also includes steps to build a linux kernel that works with
gem5.

---------

Co-authored-by: Bobby R. Bruce <[email protected]>

src/riscv-opensbi-alt/Dockerfile-linux

@@ -0,0 +1,4 @@
+FROM --platform=$BUILDPLATFORM alpine:3.18.4
+
+# The Linux dependencies
+RUN apk add --no-cache build-base bc bison flex openssl paholelibssl perl

src/riscv-opensbi-alt/Dockerfile-opensbi

@@ -0,0 +1,11 @@
+FROM --platform=$BUILDPLATFORM ubuntu:22.04 as riscv-toolchain
+
+# The RISCV Tool Chain dependencies (not correct... needs work...).
+RUN apt update && apt install -y git autoconf automake autotools-dev curl python3 python3-pip libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev ninja-build git cmake libglib2.0-dev
+
+RUN git clone -b 2023.10.12 --depth 1 --single-branch https://github.com/riscv/riscv-gnu-toolchain
+
+WORKDIR /riscv-gnu-toolchain
+RUN ./configure --prefix=/opt/riscv
+RUN make -j$(nproc) linux
+

src/riscv-opensbi-alt/README.md

@@ -0,0 +1,18 @@
+# An alternative way to build opensbi for RISC-V
+
+`docker-compose up` is a command that is used to start and run multi-container Docker applications.
+It reads the `docker-compose.yml` file and starts the services defined in it.
+This command is useful for managing complex applications that require multiple containers.
+In this case we are creating services to build the two binaries: linux and opensbi.
+
+On the other hand, `docker buildx` is a Docker CLI plugin that extends the `docker build` command with the ability to build images for multiple platforms.
+It allows you to create and manage builder instances, which are used to build images for different architectures and operating systems.
+This is useful for building images that can run on different platforms, such as ARM and x86. In this case we use it to build to RISC-V.
+
+When doing multiplatform builds, `docker-compose up` can still be used to manage the containers, but `docker buildx` must be used to build the images for multiple platforms.
+This involves creating a builder instance with `docker buildx create`, setting it as the default builder with `docker buildx use`, and then running `docker buildx build` with the `--platform` flag to specify the target platforms.
+
+This example is not complete but should provide a good example of how this could be done.
+I cannot get opensbi to build, but this is mostly due to not knowing how to setup the correct environment in "Dockerfile-opensbi" (my build command in "docker-compose.yaml" may also be incorrect).
+
+Once complete all a user would need to do is run `docker-compose up` and the two binaries would be built and available for use.

src/riscv-opensbi-alt/docker-compose.yaml

@@ -0,0 +1,62 @@
+
+version: '3'
+
+services:
+  obtain-opensbi:
+    image: alpine/git
+    volumes:
+      - ./:/work
+    working_dir: /work/
+    command: |
+      clone -c advice.detachedHead=false -b v1.3.1 --depth 1 --single-branch https://github.com/riscv-software-src/opensbi
+
+  build-opensbi:
+    depends_on:
+       - obtain-opensbi
+    build:
+      context: .
+      dockerfile: Dockerfile-opensbi
+      platforms: 
+        - linux/riscv64
+    platform: linux/riscv64
+    volumes:
+      - ./opensbi:/work
+    working_dir: /work/
+    # Can't figure out how to pass `nproc` to make, so using 10, change or fix.
+    command: make PLATFORM=generic FW_JUMP=y FW_JUMP_ADDR=0x80200000 FW_JUMP_FDT_ADDR=0x87e00000 -j 10
+
+  obtain-linux:
+    image: alpine/git
+    volumes:
+      - ./:/work
+    working_dir: /work/
+    command: |
+      clone -c advice.detachedHead=false -b v6.5-rc5 --depth 1 --single-branch https://github.com/torvalds/linux.git
+
+  config-linux:
+    depends_on:
+      - obtain-linux
+    build:
+      context: .
+      dockerfile: Dockerfile-linux
+      platforms:  [linux/riscv64]
+    volumes:
+      - ./linux:/work
+    working_dir: /work/
+    platform: linux/riscv64
+    command: make defconfig
+
+  build-linux:
+    depends_on:
+      - obtain-linux
+      - config-linux
+    build:
+      context: .
+      dockerfile: Dockerfile-linux
+      platforms:  [linux/riscv64]
+    volumes:
+      - ./linux:/work
+    working_dir: /work/
+    platform: linux/riscv64
+    # Couldn't get -j`nproc` to work, so using 10, change or fix.
+    command: make -j 10

src/riscv-opensbi/README.md

@@ -0,0 +1,117 @@
+---
+title: RISC-V Full System with OpenSBI bootloader
+tags:
+    - fullsystem
+    - bootloader
+    - riscv
+layout: default
+permalink: resources/riscv-opensbi
+shortdoc: >
+    Resources to build OpenSBI bootloader that works with gem5 full system simulations.
+author: ["Hoa Nguyen"]
+---
+
+# RISCV Full System with OpenSBI Firmware as a Bootloader
+
+This document provides instructions to create an (OpenSBI)[https://github.com/riscv-software-src/opensbi] bootloader binary and a Linux kernel binary that work with gem5 full system simulations.
+The bootloader and the kernel binaries are completely independent; however, we'll provide instructions to build both binaries for completeness.
+
+In gem5, this bootloader is supposed to work with the default RISC-V Linux kernel configuration from the Linux project, as well as with any disk RISC-V image provided by `gem5-resources`.
+Different from the `riscv-fs` resource, the bootloader and the Linux kernel are separate inputs to gem5.
+
+**Note:** The bootloader and the Linux kernel are compiled using a docker container as specified in `build-env.Dockerfile`.
+The docker image is derived from Ubuntu 22.04 LTS and uses the cross compilers provided by Ubuntu.
+
+We assume the following directory structure while following the instructions in this README file:
+
+```
+riscv-opensbi/
+  |___ opensbi/                                # The source of OpenSBI cloned from https://github.com/riscv-software-src/opensbi
+  |
+  |___ linux/                                  # Linux source code
+  |
+  |___ build-env.dockerfile                    # A docker file to build cross compilation environment
+  |
+  |___ README.md                               # This README file
+```
+
+## Overview
+
+`OpenSBI` is a reference implementation of RISC-V SBI (Supervisor Binary Interface), which provides an interface to the M-mode (machine mode) or HS-mode (hypervisor mode).
+Despite the name, `OpenSBI` itself can act as a first-stage bootloader setting up the environment before jumping to the start of a payload, a Linux kernel in this case.
+`OpenSBI` also offers handles for events requiring M-mode interventation, e.g., interrupts.
+
+`OpenSBI` offers 3 different boot flows: `FW_JUMP`, `FW_DYNAMIC`, and `FW_PAYLOAD`.
+The definition for each can be found in the official documentation.
+In this document, we use `FW_JUMP` as a bootloader.
+
+The `FW_JUMP` bootloader assumes that the device tree blob and the payload (in this case, the linux kernel) are written to memory before `FW_JUMP` bootloader is executed.
+In gem5 simulation, the `FW_JUMP` bootloader itself, the device tree blob, and the linux kernel are written to memory before simulation.
+
+During simulation, the `FW_JUMP` bootloader performs the first stage of booting then jumps to a specific address hardcoded in the bootloader.
+Therefore, you need to make sure that the linux kernel is written to memory starting from that specific address.
+
+## Building the Docker Image
+
+The following command builds a Docker image as specified in the `build-env.dockerfile` file.
+The image name is `gem5/riscv-gcc`.
+
+In the `riscv-opensbi/` folder,
+
+```sh
+docker build -f build-env.dockerfile -t gem5/riscv-gcc .
+```
+
+## Downloading and Building the OpenSBI `FW_JUMP` Bootloader
+
+The following commands download the official `OpenSBI` repository and checkout the `v1.3.1` release.
+
+In the `riscv-opensbi/` folder,
+
+```sh
+git clone https://github.com/riscv-software-src/opensbi
+cd opensbi
+git checkout v1.3.1
+```
+
+The following commands use the `gem5/riscv-gcc` docker image to build the `FW_JUMP` firmware, which we will use as a bootloader later on.
+
+In the `riscv-opensbi/` folder,
+
+```sh
+cd opensbi
+docker run -u $UID:$GID --volume $PWD:/workdir -w /workdir --rm -it gem5/riscv-gcc
+(docker) make PLATFORM=generic FW_JUMP=y FW_JUMP_ADDR=0x80200000 FW_JUMP_FDT_ADDR=0x87e00000 CROSS_COMPILE=riscv64-linux-gnu- -j`nproc`
+(docker) exit
+```
+
+The above `make` command explicitly specifies that,
+
+- `FW_JUMP_ADDR=0x80200000`: after the first stage booting is done by OpenSBI, it will jump to the instruction at the address `0x80200000`.
+As we are interesting in booting the Linux kernel, we should write to memory the linux kernel (the vmlinux file) at the physical address of `0x80200000` before starting gem5 simulation.
+- `FW_JUMP_FDT_ADDR=0x87e00000`: the device tree is located at `0x87e00000`.
+This means, in gem5, we should write to memory the device tree blob at the physical address of `0x87e00000` before starting gem5 simulation.
+
+The bootloader is located at `opensbi/build/platform/generic/firmware/fw_jump.elf`.
+
+## Downloading and Building the Linux kernel
+
+The following commands download the linux kernel version 6.5.5 from `kernel.org`, and build the linux kernel using the default configuration.
+
+In the `riscv-opensbi/` folder,
+
+```sh
+wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.5.tar.xz
+tar xf linux-6.5.5.tar.xz
+mv linux-6.5.5 linux
+cd linux
+docker run -u $UID:$GID --volume $PWD:/workdir -w /workdir --rm -it gem5/riscv-gcc
+(docker) yes "" | make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- oldconfig
+(docker) make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- -j$(nproc)
+(docker) exit
+```
+
+The `yes "" | make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- oldconfig` chooses all default options in the linux kernel configuration for RISC-V.
+To change the configuration after that step, you can use `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- menuconfig`.
+
+The linux kernel is located at `linux/vmlinux`.

src/riscv-opensbi/build-env.dockerfile

@@ -0,0 +1,24 @@
+FROM ubuntu:22.04
+
+ENV DEBIAN_FRONTEND=noninteractive
+RUN apt -y update
+RUN apt -y upgrade
+RUN apt -y install \
+  binutils build-essential libtool texinfo gzip zip unzip patchutils curl git \
+  make cmake ninja-build automake bison flex gperf grep sed gawk bc \
+  zlib1g-dev libexpat1-dev libmpc-dev libglib2.0-dev libfdt-dev libpixman-1-dev \
+  vim tmux python-is-python3 \
+  libncurses-dev gawk openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev \
+  gcc-12-riscv64-linux-gnu g++-12-riscv64-linux-gnu
+
+RUN ln -s /usr/bin/riscv64-linux-gnu-cpp-12 /usr/bin/riscv64-linux-gnu-cpp & \
+    ln -s /usr/bin/riscv64-linux-gnu-cpp-12 /usr/bin/riscv64-linux-gnu-cpp & \
+    ln -s /usr/bin/riscv64-linux-gnu-g++-12 /usr/bin/riscv64-linux-gnu-g++ & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcc-12 /usr/bin/riscv64-linux-gnu-gcc & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcc-ar-12 /usr/bin/riscv64-linux-gnu-gcc-ar & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcc-nm-12 /usr/bin/riscv64-linux-gnu-gcc-nm & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcc-ranlib-12 /usr/bin/riscv64-linux-gnu-gcc-ranlib & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcov-12 /usr/bin/riscv64-linux-gnu-gcov & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcov-dump-12 /usr/bin/riscv64-linux-gnu-gcov-dump & \
+    ln -s /usr/bin/riscv64-linux-gnu-gcov-tool-12 /usr/bin/riscv64-linux-gnu-gcov-tool & \
+    ln -s /usr/bin/riscv64-linux-gnu-lto-dump-12 /usr/bin/riscv64-linux-gnu-lto-dump