Skip to content

Commit

Permalink
Merge with squash from stable-v2.0
Browse files Browse the repository at this point in the history
I assume that the project will develop in the direction of supporting
the the in-tree Linux kernel blksnap module.

The standalone blksnap module distributed as part of the Veeam Agent
for Linux product will be located in branches with the prefix 'VAL'.

The history of patches for the kernel was saved in the 'patch'
directory.
  • Loading branch information
SergeiShtepa committed Nov 16, 2023
1 parent ba76dde commit 0837313
Show file tree
Hide file tree
Showing 119 changed files with 3,523 additions and 13,350 deletions.
77 changes: 77 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1

shared: &shared
steps:
- checkout
- run:
name: Prepare environment
command: |
echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
apt-get update
apt-get dist-upgrade -y
- run:
name: Install dependencies
command: |
apt-get install -y g++ cmake uuid-dev libboost-program-options-dev libboost-filesystem-dev libssl-dev debhelper
- run:
name: Build dev, tools and tests packages
working_directory: pkg/deb
command: ./build-blksnap.sh
- run:
name: Save generate packages as artifacts
working_directory: ..
command: |
mkdir /tmp/artifacts
mv *.deb /tmp/artifacts
- store_artifacts:
path: /tmp/artifacts

jobs:
debian10:
<<: *shared
docker:
- image: library/debian:buster
debian11:
<<: *shared
docker:
- image: library/debian:bullseye
debian12:
<<: *shared
docker:
- image: library/debian:bookworm
ubuntu1404:
<<: *shared
docker:
- image: library/ubuntu:trusty
ubuntu1604:
<<: *shared
docker:
- image: library/ubuntu:xenial
ubuntu1804:
<<: *shared
docker:
- image: library/ubuntu:bionic
ubuntu2004:
<<: *shared
docker:
- image: library/ubuntu:focal
ubuntu2204:
<<: *shared
docker:
- image: library/ubuntu:jammy
ubuntu2310:
<<: *shared
docker:
- image: library/ubuntu:mantic

workflows:
build-deb:
jobs:
- debian10
- debian11
- debian12
- ubuntu2004
- ubuntu2204
- ubuntu2310
8 changes: 1 addition & 7 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,5 @@ end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true

[module/**.{c,h}]
indent_style = tab
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[Makefile]
indent_style = tab
indent_style = tab
65 changes: 65 additions & 0 deletions .github/workflows/build-other-archs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Test build on other archs

on:
push:
branches:
- "*"
paths:
- "include/**"
- "lib/**"
- "tests/**"
- "tools/**"
pull_request:
branches:
- "*"
paths:
- "include/**"
- "lib/**"
- "tests/**"
- "tools/**"
workflow_dispatch:

jobs:
build_job:
# The host should always be linux
runs-on: ubuntu-22.04
name: ${{ matrix.arch }}

strategy:
fail-fast: false
matrix:
include:
- arch: aarch64
- arch: ppc64le
- arch: s390x
- arch: armv7
steps:
- uses: actions/checkout@v4
- uses: uraimo/run-on-arch-action@v2
name: Build
id: build
with:
arch: ${{ matrix.arch }}
distro: bullseye

# Not required, but speeds up builds
githubToken: ${{ github.token }}

# The shell to run commands with in the container
shell: /bin/sh

# Install some dependencies in the container. This speeds up builds if
# you are also using githubToken. Any dependencies installed here will
# be part of the container image that gets cached, so subsequent
# builds don't have to re-install them. The image layer is cached
# publicly in your project's package repository, so it is vital that
# no secrets are present in the container state or logs.
install: |
apt-get update -q -y
apt-get install -q -y g++ cmake uuid-dev libboost-program-options-dev libboost-filesystem-dev libssl-dev
# Build blksnap-dev, blksnap-tools and blksnap-tests
run: |
cmake .
make
92 changes: 92 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
name: Build

on:
push:
branches:
- "*"
paths-ignore:
- "README.md"
- "doc/**"
pull_request:
branches:
- "*"
paths-ignore:
- "README.md"
- "doc/**"
workflow_dispatch:

jobs:
amd64:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
name: [Ubuntu-20, Ubuntu-22, Debian-10, Debian-11, Debian-12, Debian-Testing, Debian-Experimental]
cpp_compiler: [g++]
include:
- name: Ubuntu-20
# Uses gcc 9.3.0, clang 10.0.0, cmake 3.16.3
image: "ubuntu:20.04"
ubuntu: 20
- name: Ubuntu-22
# Uses gcc 12.2.0, clang 15.0.7, cmake 3.24.2
image: "ubuntu:22.04"
ubuntu: 22
- name: Debian-10
# Uses gcc 8.3.0, clang 7.0.1, cmake 3.13.4
image: "debian:buster"
- name: Debian-11
# Uses gcc 10.2.1, clang 11.0.1, cmake 3.18.4
image: "debian:bullseye"
- name: Debian-11
image: "debian:bullseye"
c_compiler: clang
cpp_compiler: clang++
- name: Debian-12
# Uses gcc 12.2.0, clang 15.0.6, cmake 3.25.1
image: "debian:bookworm"
- name: Debian-12
image: "debian:bookworm"
c_compiler: clang
cpp_compiler: clang++
- name: Debian-Testing
image: "debian:testing"
- name: Debian-Testing
image: "debian:testing"
c_compiler: clang
cpp_compiler: clang++
- name: Debian-Experimental
image: "debian:experimental"
- name: Debian-Experimental
image: "debian:experimental"
c_compiler: clang
cpp_compiler: clang++
container:
image: ${{ matrix.image }}
env:
LANG: en_US.UTF-8
BUILD_TYPE: ${{ matrix.build_type }}
CC: ${{ matrix.c_compiler }}
CXX: ${{ matrix.cpp_compiler }}
WITH_PROJ: ON
APT_LISTCHANGES_FRONTEND: none
DEBIAN_FRONTEND: noninteractive
steps:
- name: Install packages required
shell: bash
run: |
apt-get update -qq
apt-get install -yq \
clang \
cmake \
g++ \
uuid-dev \
libboost-program-options-dev \
libboost-filesystem-dev \
libssl-dev
- uses: actions/checkout@v4
- name: Build blksnap-dev, blksnap-tools and blksnap-tests
working-directory: .
run: |
cmake .
make
2 changes: 1 addition & 1 deletion .github/workflows/codespell.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: codespell
on: [pull_request]

jobs:
check:
codespell:
name: codespell
runs-on: ubuntu-latest
steps:
Expand Down
6 changes: 3 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ add_subdirectory(${CMAKE_SOURCE_DIR}/tests/cpp)

if(EXISTS ${CMAKE_SOURCE_DIR}/cmake/cmake_uninstall.cmake.in)
configure_file(${CMAKE_SOURCE_DIR}/cmake/cmake_uninstall.cmake.in
${CMAKE_SOURCE_DIR}/cmake/cmake_uninstall.cmake @ONLY
${CMAKE_CURRENT_BINARY_DIR}/cmake_uninstall.cmake @ONLY
)
add_custom_target(uninstall "${CMAKE_COMMAND}" -P "${CMAKE_SOURCE_DIR}/cmake/cmake_uninstall.cmake")
endif()
add_custom_target(uninstall "${CMAKE_COMMAND}" -P "${CMAKE_CURRENT_BINARY_DIR}/cmake_uninstall.cmake")
endif()
63 changes: 6 additions & 57 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,9 @@
| :warning: Important note |
|:---------------------------|
| Master branch was recently updated only about readme and patches posted for upstream kernel |
| Latest work for upstream is in [stable-2.0](https://github.com/veeam/blksnap/tree/stable-v2.0) branch, for more details check [Upstream kernel integration](https://github.com/veeam/blksnap/blob/master/doc/README-upstream-kernel.md#work-in-progress-and-news) |
| For older blksnap version based on external module (actually used in production) see these branches: [VAL-6.1](https://github.com/veeam/blksnap/tree/VAL-6.1), [VAL-6.0](https://github.com/veeam/blksnap/tree/VAL-6.0), [stable-1.0](https://github.com/veeam/blksnap/tree/stable-v1.0) |
| :information_source: To Veeam agent for linux users: |
| If you need only kernel module updated with latest kernel versions support and latest fixes for it [build kernel module](#how-to-build) from [VAL-6.0](https://github.com/veeam/blksnap/tree/VAL-6.0) or [VAL-6.1](https://github.com/veeam/blksnap/tree/VAL-6.1) based on your Veeam agent for linux version |

# BLKSNAP - Block Devices Snapshots Module
# BLKSNAP - Block Devices Snapshots

* [Extended description and features](doc/blksnap.md)
* [Repository structure](#repository-structure)
* [Licensing](#licensing)
* [Kernel module](#kernel-module)
* [Upstream kernel integration](https://github.com/veeam/blksnap/blob/master/doc/README-upstream-kernel.md)
* [Upstream kernel integration](#kernel-integration)
* [Tools](#tools)
* [Library](#library)
* [Tests](#tests)
Expand All @@ -28,8 +19,6 @@
* doc/ - Documentation
* include/ - Libraries public headers
* lib/ - Libraries sources
* module/ - Sources of kernel module
* patches/ - Patches for the upstream linux kernel
* pkg/ - Scripts for building deb and rpm packages
* tests/ - Test scripts and tests source code
* tools/ - Source files of tools for working with blksnap
Expand All @@ -42,48 +31,9 @@ Copyright (C) 2022 Veeam Software Group GmbH

This project use [SPDX License Identifier](https://spdx.dev/ids/) in source files header.


## Kernel module
This kernel module implements snapshot and changed block tracking functionality.
The module is developed with the condition of simply adding it to the upstream.
Therefore, the module is divided into two parts: bdevfilter and blksnap.
bdevfilter provides the ability to intercept I/O units (bio). The main logic
is concentrated in blksnap. The upstream variant does not contain a bdevfilter,
but accesses the kernel to attach and detach the block device filter.

Relating the work in progress for integration in upstream kernel see the specific [README](https://github.com/veeam/blksnap/blob/master/doc/README-upstream-kernel.md)

### How to build
Installing the necessary deb packages.
``` bash
sudo apt install gcc linux-headers-$(uname -r)
```
Or installing the necessary rpm packages.
``` bash
sudo yum install gcc kernel-devel
```
``` bash
cd ./module
mk.sh build
```
In directory current directory you can found bdevfilter.ko and blksnap.ko.

### How to install
``` bash
cd ./module
mk.sh install-flt
mk.sh install
```
### How to create deb package
``` bash
sudo apt install debhelper dkms
# on debian >=12 and ubuntu >= 23.04 is needed dh-dkms, not installed anymore as dkms dep.
sudo apt install dh-dkms
cd ./pkg/deb
./build-blksnap-dkms.sh ${VERSION}
```
### How to create rpm package
There are several variants, look in the ./pkg/rpm directory.
## Kernel integration
Relating the work in progress for integration in upstream kernel see the
specific [README](https://github.com/veeam/blksnap/blob/master/doc/README-upstream-kernel.md)

## Tools
The blksnap tools allows you to manage the module from the command line.
Expand Down Expand Up @@ -152,7 +102,6 @@ cd ./pkg/deb
```

## Compatibility notes
- blksnap kernel module support kernel versions >= 5.10, support only X86 archs, blksnap for upstream instead can support any arch (other archs need to be tested)
- blksnap kernel module for upstream can support any arch (other archs beyond X86 archs needs more testing)
- all supported debian and ubuntu supported versions are supported but with some notes:
- not all have debian/ubuntu versions have official packages of kernel >= 5.10, so an unofficial or custom ones more updated are needed, with blksnap-dkms should be still possible easy/fast build/install blksnap module on them (is also possible build/install it manually without dkms)
- debian 8 and ubuntu 14.04 needs to install cmake 3 from backports to build
Loading

0 comments on commit 0837313

Please sign in to comment.