This is a repo I use to support my own Linux kernel development. I put scripts/tools here instead of inside the actual kernel tree. It is expected that this repo will live on my machine, and any kernel trees will live under this directory, but won't be committed to this repo.
Here is a typical workflow with bells and whistles:
-
(Optional) Update the kernel source tree or get a new one
$ cd linux && git pull upstream master # or $ ./scripts/fetch-kernel-tarball.sh 5.18.6
-
Configure the kernel, using a QEMU configuration with
./qemu/minimal-qemu-kernel-config.sh linux
-
Create a QEMU image using
./qemu/create-qemu-image.sh <debian|nixos>
-
Compile the kernel with
cd linux/ && make -j32
-
Run the kernel with QEMU image using something like:
$ ./qemu/run-qemu-kernel.sh linux nixos.img /path/to/shared-files
/path/to/shared-files
here is a directory that will be packaged up into the QEMU.img
file and mounted at/shared
in the VM. This arg is optional, but it is useful for e.g. adding compiled, out-of-tree kernel modules.
The kernel has a script to generate a compile-commands.json
usable with LSP:
$ cd linux/
# Make a config and build the kernel
$ make mrproper
$ make defconfig
# Build just for good measure
$ make -j14
# Generate compile-commands.json
$ ./scripts/clang-tools/gen_compile_commands.py
$ make mrproper
$ make CC=clang allnoconfig defconfig rust.config
$ make CC=clang -j14
Note: when I use LLVM=1
instead of CC=clang
, I get this error when linking, even when I don't enable Rust:
LD arch/x86/boot/setup.elf
ld.lld: error: section .bsdata file range overlaps with .header
>>> .bsdata range is [0x1092, 0x122B]
>>> .header range is [0x11EF, 0x126B]
Maybe the problem is my LLVM env for nix in general? NixOS/nixpkgs#217724
- Mentorship Session: Setting Up an Environment for Writing Linux Kernel Modules in Rust
- https://github.com/jordanisaacs/kernel-module-flake
- https://github.com/Rust-for-Linux/nix
- https://github.com/jordanisaacs/kernel-module-flake
-
Set up some stupid symlinks that buildroot expects (well, really the problem is
libtool
andautotools
I think.buildroot
can't use host libraries and pretty much compiles everything from scratch).$ sudo ln -s /run/current-system/sw/bin/file /usr/bin/file $ sudo ln -s /run/current-system/sw/bin/true /bin/true $ sudo ln -s /run/current-system/sw/bin/awk /usr/bin/awk $ sudo ln -s /run/current-system/sw/bin/bash /bin/bash $ sudo ln -s /run/current-system/sw/bin/install /usr/bin/install $ sudo ln -s /run/current-system/sw/bin/openssl /usr/bin/openssl
-
Use the external config
$ cd buildroot $ make BR2_EXTERNAL=../buildroot-rpi reaver_rpi_defconfig menuconfig
-
Build
$ make
(no need for-j
since buildroot uses parallelism internally, not for top level targets) -
Copy image to SD card (see below for generic instructions)
(These were the instructions I used before I set up the buildroot-rpi
external infra)
Getting buildroot
working is not too onerous, at least since I figured out
some nix quirks. Here is a general procedure:
-
Set up some stupid symlinks that buildroot expects (well, really the problem is
libtool
andautotools
I think.buildroot
can't use host libraries and pretty much compiles everything from scratch).$ sudo ln -s (which file) /usr/bin/file $ sudo ln -s (which true) /bin/true $ sudo ln -s (which awk) /usr/bin/awk $ sudo ln -s (which bash) /bin/bash
-
git clone git://git.buildroot.net/buildroot
-
cd buildroot
-
make raspberrypi4_64_defconfig
-
make -j32
-
Copy
output/images/sdcard.img
to an SD card (MAKE SURE TO VERIFY DEVICE NAME, REPLACEsdz
)$ sudo dd if=output/images/sdcard.img of=/dev/sdz status=progress
-
Connect Pi serial port to USB serial port connector, use
dmesg
to findtty
device namedmesg | grep 'cp210x converter now attached' [48407.800462] usb 1-4: cp210x converter now attached to ttyUSB0
-
Connect with GNU screen
$ sudo screen /dev/ttyUSB0 115200
-
Boot the Pi! Log in with
root
and no password.
Cool links:
TODO Check out this repo for kernel dev with nix https://github.com/jordanisaacs/kernel-module-flake
QEMU dev env:
- https://nixos.wiki/wiki/Kernel_Debugging_with_QEMU
- https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix
- Alternate NixOS QEMU image https://gist.github.com/tarnacious/f9674436fff0efeb4bb6585c79a3b9ff
- NixOS qcow2 build https://gist.github.com/jahkeup/14c0f35383bf949fdd92fbfa20184b4f
- https://discourse.nixos.org/t/how-to-build-a-standalone-nixos-qemu-vm/5688/4
- https://vccolombo.github.io/cybersecurity/linux-kernel-qemu-setup/
- https://kernel-recipes.org/en/2015/talks/speed-up-your-kernel-development-cycle-with-qemu/
- https://medium.com/@daeseok.youn/prepare-the-environment-for-developing-linux-kernel-with-qemu-c55e37ba8ade
- https://www.collabora.com/news-and-blog/blog/2017/01/16/setting-up-qemu-kvm-for-kernel-development/
- Nice explanation of using -nographic with available options https://web.archive.org/web/20180104171638/http://nairobi-embedded.org/qemu_monitor_console.html
- ClangBuiltLinux has a repo super similar to this repo, including QEMU and buildroot scripts https://github.com/ClangBuiltLinux/boot-utils
Email:
- https://www.kernel.org/doc/html/latest/process/email-clients.html
- https://offlinemark.com/2020/09/26/tips-for-submitting-your-first-linux-kernel-patch/
- https://ane.iki.fi/emacs/patches.html
- https://git-send-email.io
- https://devtut.github.io/git/git-send-email.html#sending-patches-by-mail
First time contributions:
- https://kernelnewbies.org/
- https://kernelnewbies.org/KernelHacking
- Suggests running
checkpatch.pl
ondrivers/staging
directories - Also suggests running Coccinelle
- Suggests running
- https://kernelnewbies.org/KernelHacking
- Very informative patch set with revisions, followups, responses from Greg
K-H's autobot, etc
https://lore.kernel.org/linux-staging/[email protected]/T/
- Versioning patch revisions https://kernelnewbies.org/FirstKernelPatch#Versioning_one_patch_revision
- https://www.linux.com/news/three-ways-beginners-contribute-linux-kernel/
- https://williamdurand.fr/2021/02/22/first-patch-in-the-linux-kernel/
- tpiekarski's comment from discussion "Is reading Linux kernel development helpful in 2020? Is it outdated?"
- How to become a Kernel Developer?
- How should I start kernel development?
- What (not how) to contribute to the kernel
- Recommendations for newer books on kernel development?
Getting started, things to do
- Kernel dev process
- https://github.com/agelastic/eudyptula
- https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project
Embedded:
- Dev setup
- Get booting with NFS or TFTP
- See the bootlin labs or the Mastering Embedded Linux Book
- NFS mounting seems to work with
# mount -v -t nfs 10.42.0.1:/nfs-export /nfs-mnt -o nolock,vers=3
- Consider using buildroot or nix for just a barebones setup to bootstrap NFS and/or SSH, and thereafter just syncing to board
- Get booting with NFS or TFTP
- Buildroot
- (Consider abandoning buildroot and figuring out a kernel dev loop with nix)
- Set up NFS booting, and/or an initramfs for faster booting so we can have a quicker kernel dev inner loop
- https://buildroot.org/downloads/manual/manual.html#customize
- https://buildroot.org/downloads/manual/using-buildroot-development.txt
- Set up my own BR2_EXTERNAL directory for rpi4 + Linux dev