Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to latest cxx-common version 0.4.1 + llvm 16 + ubuntu 22.04 #329

Merged
merged 6 commits into from
Jul 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/anghabench-after-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
strategy:
fail-fast: false
matrix:
llvm: [ '15' ]
llvm: [ '16' ]
run_size: [ '1k' ]

name: Run AnghaBench CI (AMD64)
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/anghabench-cron-jobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
strategy:
fail-fast: false
matrix:
llvm: [ '15' ]
llvm: [ '16' ]
run_size: [ '1k' ]

name: Run AnghaBench CI (AMD64)
Expand Down
28 changes: 14 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,25 @@ on:
branches:
- '*'

env:
CC: clang
CXX: clang++

jobs:
build_linux:
strategy:
fail-fast: false
matrix:
image:
- { name: 'ubuntu', tag: '20.04' }
- { name: 'ubuntu', tag: '22.04' }
llvm: [
'16'
]

name: Rellic CI
runs-on: "gha-ubuntu-32"
container:
image: docker.pkg.github.com/lifting-bits/cxx-common/vcpkg-builder-${{ matrix.image.name }}:${{ matrix.image.tag }}
image: ghcr.io/lifting-bits/cxx-common/vcpkg-builder-${{ matrix.image.name }}-v2:${{ matrix.image.tag }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -47,12 +51,8 @@ jobs:
shell: bash
run: |
# TODO some of these should probably live in the Docker build image
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -
echo "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-${{ matrix.llvm }} main" >> /etc/apt/sources.list
echo "deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-${{ matrix.llvm }} main" >> /etc/apt/sources.list
apt-get update
apt-get install -y ninja-build pixz xz-utils make rpm python3.8
update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 100
apt-get install -y ninja-build pixz xz-utils make rpm python3

- name: Build with build script
shell: bash
Expand Down Expand Up @@ -173,7 +173,7 @@ jobs:
release_packages:
# Do not run the release procedure if any of the builds has failed
needs: [ build_linux, build_mac ]
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')

steps:
Expand Down Expand Up @@ -212,22 +212,22 @@ jobs:

- name: Group the packages by platform
run: |
zip -r9 rellic_ubuntu-20.04_packages.zip \
ubuntu-20.04*
zip -r9 rellic_ubuntu-22.04_packages.zip \
ubuntu-22.04*

zip -r9 rellic_macos-12_packages.zip \
macos-12*

- name: Upload the Ubuntu 20.04 packages
- name: Upload the Ubuntu 22.04 packages
uses: actions/upload-release-asset@v1

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: rellic_ubuntu-20.04_packages.zip
asset_name: rellic_ubuntu-20.04_packages.zip
asset_path: rellic_ubuntu-22.04_packages.zip
asset_name: rellic_ubuntu-22.04_packages.zip
asset_content_type: application/gzip

- name: Upload the macOS 12 packages
Expand All @@ -247,7 +247,7 @@ jobs:
strategy:
matrix:
llvm: ["16"]
ubuntu: ["20.04"]
ubuntu: ["22.04"]
steps:
- uses: actions/checkout@v2
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/diff_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
image:
- { name: 'ubuntu', tag: '20.04', codename: 'focal' }
llvm: [ '16' ]
common_base: [ 'https://github.com/lifting-bits/cxx-common/releases/latest/download' ]
common_base: [ 'https://github.com/lifting-bits/cxx-common/releases/download/v0.4.1' ]

env:
CC: clang-${{ matrix.llvm }}
Expand All @@ -22,7 +22,7 @@ jobs:
name: Diff in ouput between old and new rellic
runs-on: gha-ubuntu-32
container:
image: docker.pkg.github.com/lifting-bits/cxx-common/vcpkg-builder-${{ matrix.image.name }}:${{ matrix.image.tag }}
image: ghcr.io/lifting-bits/cxx-common/vcpkg-builder-${{ matrix.image.name }}-v2:${{ matrix.image.tag }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
Expand Down
13 changes: 5 additions & 8 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ FROM ${BUILD_BASE} as base
# Build-time dependencies go here
# See here for full list of those dependencies
# https://github.com/lifting-bits/cxx-common/blob/master/docker/Dockerfile.ubuntu.vcpkg
FROM trailofbits/cxx-common-vcpkg-builder-ubuntu:${UBUNTU_VERSION} as deps
FROM ghcr.io/lifting-bits/cxx-common/vcpkg-builder-ubuntu-v2:${UBUNTU_VERSION} as deps
ARG UBUNTU_VERSION
ARG ARCH
ARG LLVM_VERSION
Expand All @@ -30,19 +30,16 @@ ARG LLVM_VERSION
ARG LIBRARIES
ENV TRAILOFBITS_LIBRARIES="${LIBRARIES}"
ENV PATH="${LIBRARIES}/llvm/bin/:${LIBRARIES}/cmake/bin:${PATH}"
ENV CC=clang
ENV CXX=clang++

WORKDIR /rellic
COPY ./ ./
# The reason we don't use --install
# is so that container has the same exact code as the packages
RUN ./scripts/build.sh \
--llvm-version ${LLVM_VERSION} \
--prefix /opt/trailofbits \
--extra-cmake-args "-DCMAKE_BUILD_TYPE=Release"

RUN cd rellic-build && \
CTEST_OUTPUT_ON_FAILURE=1 cmake --build . --verbose --target test && \
cmake --build . --target install
--extra-cmake-args "-DCMAKE_BUILD_TYPE=Release" \
--install

# Small installation image
FROM base as install
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ If you are experiencing undocumented problems with Rellic then ask for help in t

## Supported Platforms

Rellic is supported on Linux platforms and has been tested on Ubuntu 20.04.
Rellic is supported on Linux platforms and has been tested on Ubuntu 22.04.

## Dependencies

Expand Down Expand Up @@ -236,7 +236,7 @@ Finally, we build and package Rellic. This script will create another directory,

```shell
cd rellic
./scripts/build.sh --llvm-version 14
./scripts/build.sh --llvm-version 16
# to install the deb package, then do:
sudo dpkg -i rellic-build/*.deb
```
Expand Down Expand Up @@ -271,10 +271,10 @@ make -j8

The Docker image should provide an environment which can set-up, build, and run rellic. The Docker images are parameterized by Ubuntu verison, LLVM version, and architecture.

To build the docker image using LLVM 16 for Ubuntu 20.04 on amd64 you can run the following command:
To build the docker image using LLVM 16 for Ubuntu 22.04 on amd64 you can run the following command:

```sh
ARCH=amd64; UBUNTU=20.04; LLVM=16; docker build . \
ARCH=amd64; UBUNTU=22.04; LLVM=16; docker build . \
-t rellic:llvm${LLVM}-ubuntu${UBUNTU}-${ARCH} \
-f Dockerfile \
--build-arg UBUNTU_VERSION=${UBUNTU} \
Expand All @@ -286,13 +286,13 @@ To run the decompiler, the entrypoint has already been set, but make sure the bi

```sh
# Get the bc file
clang-14 -emit-llvm -c ./tests/tools/decomp/issue_4.c -o ./tests/tools/decomp/issue_4.bc
clang-16 -emit-llvm -c ./tests/tools/decomp/issue_4.c -o ./tests/tools/decomp/issue_4.bc

# Decompile
docker run --rm -t -i \
-v $(pwd):/test -w /test \
-u $(id -u):$(id -g) \
rellic:llvm14-ubuntu20.04-amd64 --input ./tests/tools/decomp/issue_4.bc --output /dev/stdout
rellic:llvm16-ubuntu22.04-amd64 --input ./tests/tools/decomp/issue_4.bc --output /dev/stdout
```

To explain the above command more:
Expand Down
10 changes: 7 additions & 3 deletions scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ CURR_DIR=$( pwd )
BUILD_DIR="${CURR_DIR}/rellic-build"
INSTALL_DIR=/usr/local
LLVM_VERSION=llvm-16
CXX_COMMON_VERSION=v0.3.2
CXX_COMMON_VERSION=v0.4.1
OS_VERSION=unknown
ARCH_VERSION=unknown
BUILD_FLAGS=
Expand All @@ -38,6 +38,10 @@ function GetUbuntuOSVersion
OS_VERSION=ubuntu-20.04
return 0
;;
jammy)
OS_VERSION=ubuntu-22.04
return 0
;;
focal)
OS_VERSION=ubuntu-20.04
return 0
Expand All @@ -62,7 +66,7 @@ function GetUbuntuOSVersion
return 0
;;
*)
echo "[x] Ubuntu ${DISTRIB_CODENAME} is not supported. Only focal (20.04) and bionic (18.04) are pre-compiled."
echo "[x] Ubuntu ${DISTRIB_CODENAME} is not supported. Only jammy (22.04) are pre-compiled."
echo "[x] Please see https://github.com/lifting-bits/cxx-common to build dependencies from source."
return 1
;;
Expand Down Expand Up @@ -153,7 +157,7 @@ function GetOSVersion
;;

*debian*)
OS_VERSION=ubuntu-20.04
OS_VERSION=ubuntu-22.04
return 0;
;;

Expand Down
Loading