Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

moss: Prime vfs cache on boot via systemd service #548

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ermo
Copy link
Member

@ermo ermo commented Jan 13, 2025

This should serve to make the first blit after boot run against a hot vfs cache, which should improve the performance of the first blit for Serpent OS systems installed to ext4 or f2fs roots in particular.

Neither ext4 nor f2fs blits will be as fast as xfs blits, so xfs currently remains the recommended root filesystem for Serpent OS.

Additionally, include a oneshot .service for testing the time to prime w/ a cold vfs cache.

Status after cold boot on xfs:

󰌽 ermo@blueshift:~/repos/serpent-os/recipes/m/moss [  moss-prime-vfs-cache-at-boot ? ]                                                                                                   16:37
❯ sudo systemctl status moss-prime-asset-store-vfs-cache.service
○ moss-prime-asset-store-vfs-cache.service - Prime moss asset store in vfs cache in the background
     Loaded: loaded (/usr/lib/systemd/system/moss-prime-asset-store-vfs-cache.service; enabled; preset: enabled)
     Active: inactive (dead) since Mon 2025-01-13 16:36:14 CET; 1min 16s ago
   Duration: 6.169s
    Process: 629 ExecStart=/usr/bin/bash -c time nice -n20 ionice -c Idle find /.moss/assets |xargs --max-lines=1000 --max-procs=$(nproc) stat -t >/dev/null (code=exited, status=0/SUCCESS)
   Main PID: 629 (code=exited, status=0/SUCCESS)
        CPU: 6.139s

Jan 13 16:36:08 snekbox systemd[1]: Started Prime moss asset store in vfs cache in the background.
Jan 13 16:36:14 snekbox bash[629]: real        0m6,166s
Jan 13 16:36:14 snekbox bash[629]: user        0m2,996s
Jan 13 16:36:14 snekbox bash[629]: sys        0m3,141s
Jan 13 16:36:14 snekbox systemd[1]: moss-prime-asset-store-vfs-cache.service: Deactivated successfully.
Jan 13 16:36:14 snekbox systemd[1]: moss-prime-asset-store-vfs-cache.service: Consumed 6.139s CPU time.

@ermo ermo force-pushed the moss-prime-vfs-cache-at-boot branch from 87b2c75 to 77f974d Compare January 13, 2025 15:37
@ermo ermo marked this pull request as ready for review January 13, 2025 15:40
@ermo ermo requested a review from a team as a code owner January 13, 2025 15:40
@ermo ermo force-pushed the moss-prime-vfs-cache-at-boot branch from 77f974d to 4897a90 Compare January 13, 2025 15:47
This should serve to make the first blit after boot run against a hot
vfs cache, which should improve the performance of the first blit for
Serpent OS systems installed to ext4 or f2fs roots in particular.

Neither ext4 nor f2fs blits will be as fast as xfs blits, so xfs currently
remains the recommended root filesystem for Serpent OS.

Additionally, include a oneshot .service for testing the time to prime w/
a cold vfs cache.

Signed-off-by: Rune Morling <[email protected]>
@ermo ermo force-pushed the moss-prime-vfs-cache-at-boot branch from 4897a90 to e628d53 Compare January 13, 2025 15:49
ermo added a commit to serpent-os/lichen that referenced this pull request Jan 13, 2025
Note that, in addition to being slow for cold blits, ext4 only supports
65k hardlinks per file, and it has a limit of 4k for combined extended
attribute data as well.

Given we are looking at supporting SELinux, there is a very real chance
that we are artificially (and knowingly) limiting ourselves by offering
ext4 as a rootfs for Serpent OS.

NB: serpent-os/recipes#548 is now up as a way to
    help mitigate the ext4 and f2fs cold blit slowness until we figure
    out a better solution.

Signed-off-by: Rune Morling <[email protected]>
ermo added a commit to serpent-os/lichen that referenced this pull request Jan 13, 2025
Note that, in addition to being slow for cold blits, ext4 only supports
65k hardlinks per file, and it has a limit of 4k for combined extended
attribute data as well.

Given we are looking at supporting SELinux, there is a very real chance
that we are artificially (and knowingly) limiting ourselves by offering
ext4 as a rootfs for Serpent OS.

NB: serpent-os/recipes#548 is now up as a way to
    help mitigate the ext4 and f2fs cold blit slowness until we figure
    out a better solution.

Signed-off-by: Rune Morling <[email protected]>
@ermo
Copy link
Member Author

ermo commented Jan 13, 2025

On an old 4790K w/a SATA-600 SSD with ext4 rootfs, I got the following numbers:

Hot vfs cache install blit (setup+teardown + blit): 13s-ish
Cold vfs cache install blit (setup+teardown + blit): 40s-ish

Video below is dinky, but does the job.

ext4-hot-and-cold-vfs-cache-blit-2025-01-13.webm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant