Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use gradle-build-action in Actions #708

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

syoon2
Copy link
Contributor

@syoon2 syoon2 commented Aug 5, 2023

This PR modifies the Actions workflows to use gradle/gradle-build-action for running gradle instead of invoking it from CLI.

Changes include:

  • Modifying workflows to use gradle/gradle-build-action
  • Disabling caching provided by actions/setup-java to take advantage of finer-grain caching provided by gradle/gradle-build-action.
  • Removing --no-daemon argument in native libraries workflow (handled by gradle/gradle-build-action action - see documentation)

@DevCharly
Copy link
Collaborator

Just tested this on branch test-gradle-build-action.
For testing, temporary enabled cache writing for all branches and did some runs.

The docs for gradle/gradle-build-action claim that it has better cache management than actions/setup-java.
But IMHO it is worse 😢

It creates so many cache files...

And each CI run creates two new cache files for each job. Size is 85 MB per job.
Our CI uses 6 jobs, so each run adds more than 500 MB to cache...
See https://github.com/JFormDesigner/FlatLaf/actions/runs/6051916895

And it is getting even worse: updated Gradle from 8.1.1 to latest version 8.3, but this fails when running in Java 11+, so changed back to Gradle 8.2.1.
The cache now includes files from Gradle 8.1.1, 8.2.1 and 8.3!
Restored cache size is 980 MB and each run adds 260 MB per job to the cache.
For 6 jobs, more than 1500 MB in total. (for each CI run!!!)
See https://github.com/JFormDesigner/FlatLaf/actions/runs/6052546499

In the meantime there are 88 files in cache: https://github.com/JFormDesigner/FlatLaf/actions/caches
Before testing this PR, there were 2 files!

IMO there are two major problems with gradle/gradle-build-action:

  1. keeps files for old Gradle versions in cache when changing Gradle version
  2. each job unnecessary saves new entries to cache, even if changing only a README file

Do you use gradle/gradle-build-action in own projects?
Do you have similar problems?

@syoon2
Copy link
Contributor Author

syoon2 commented Sep 2, 2023

Interesting; I haven't had caches from gradle/gradle-build-action growing that big in my own uses... AFAIK changing only a README does not add new entries to cache (or at least it shouldn't). Plus, I am quite convinced that keeping files for old Gradle versions is not a major issue because GH's policy automatically removes any cache entry that has not been accessed for 7 days anyway (here's a reference to GitHub Docs on this if you are interested 😄). I do agree that it is quite annoying.

Here are some of my observations from https://github.com/JFormDesigner/FlatLaf/actions/runs/6052546499:

  • The cache hit rate itself seems to work as expected; let's take the Gradle binaries that the wrapper is downloading as an example. The cache keys match, action reports exact match found, and content of the cache remains unchanged.
  • What is grabbing my eyes is something else:
    Entry: /home/runner/.gradle/caches/jars-*/*
        Requested Key : instrumented-jars-766e271f679ab63a3ae0bcdb6cbb48f0
        Restored  Key : instrumented-jars-766e271f679ab63a3ae0bcdb6cbb48f0
                  Size: 241 MB (252222873 B)
                  (Entry restored: exact match found)
        Saved     Key : instrumented-jars-bb515984e671c8fe0141d4be317d45a5
                  Size: 241 MB (252401931 B)
                  (Entry saved)
    
    The saved entry is similar in size with the restored one, and the size happens to be roughly the same with all the build artifacts (Javadoc, source JAR, native dlls, etc.) combined for FlatLaf. My hypothesis is that the build artifacts are getting added to the cache after every build for some reason (the snapshot builds receiving different filenames, if I were to guess).

Let me check if there are some dials we could tweak to stop this from happening...

Side note: you might want to consider disabling changes to README triggering the CI regardless of this PR? Just an idea.

@syoon2
Copy link
Contributor Author

syoon2 commented Sep 2, 2023

I believe that 7287ff4 should do the trick? 🤞

@DevCharly
Copy link
Collaborator

Excluding caches/jars-*/* did not help. See:
https://github.com/JFormDesigner/FlatLaf/actions/runs/6057970068

This run added ~1.5 GB and 10 additional files to cache. Coming closer to the 10 GB limit:

grafik

Plus, I am quite convinced that keeping files for old Gradle versions is not a major issue because GH's policy automatically removes any cache entry that has not been accessed for 7 days anyway

But they are accessed. See run 6057970068, which restores cache entries for Gradle 8.1.1, 8.2.1 and 8.3, but runs 8.2.1:

Entry: /home/runner/.gradle/caches/8.1.1/generated-gradle-jars/gradle-api-8.1.1.jar
    Requested Key : generated-gradle-jars-1507980a4fac48022b4011ee35e37935
    Restored  Key : generated-gradle-jars-1507980a4fac48022b4011ee35e37935
              Size: 34 MB (35635448 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)
---
Entry: /home/runner/.gradle/caches/8.1.1/generated-gradle-jars/gradle-kotlin-dsl-extensions-8.1.1.jar
    Requested Key : generated-gradle-jars-bab308473dad344b8635e751b11be532
    Restored  Key : generated-gradle-jars-bab308473dad344b8635e751b11be532
              Size: 0 MB (121613 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)
---
Entry: /home/runner/.gradle/caches/8.2.1/generated-gradle-jars/gradle-api-8.2.1.jar
    Requested Key : generated-gradle-jars-fab89ce2c5682e5ef0317ced9690e5a0
    Restored  Key : generated-gradle-jars-fab89ce2c5682e5ef0317ced9690e5a0
              Size: 36 MB (37883084 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)
---
Entry: /home/runner/.gradle/caches/8.2.1/generated-gradle-jars/gradle-kotlin-dsl-extensions-8.2.1.jar
    Requested Key : generated-gradle-jars-47a7430ddd88985eaeb279fdcd6b6b2f
    Restored  Key : generated-gradle-jars-47a7430ddd88985eaeb279fdcd6b6b2f
              Size: 0 MB (121613 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)
---
Entry: /home/runner/.gradle/caches/8.3/generated-gradle-jars/gradle-api-8.3.jar
    Requested Key : generated-gradle-jars-f7053b13997cd47925adc8f4d7041175
    Restored  Key : generated-gradle-jars-f7053b13997cd47925adc8f4d7041175
              Size: 35 MB (36474638 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)
---
Entry: /home/runner/.gradle/caches/8.3/generated-gradle-jars/gradle-kotlin-dsl-extensions-8.3.jar
    Requested Key : generated-gradle-jars-4ebefdd7992a5d0009d1b258938807f2
    Restored  Key : generated-gradle-jars-4ebefdd7992a5d0009d1b258938807f2
              Size: 0 MB (121612 B)
              (Entry restored: exact match found)
    Saved     Key : 
              Size: 
              (Entry not saved: contents unchanged)

It also does not release storage when files for multiple Gradle versions are in a single cache entry. Take a look at instrumented-jars entry. In earlier run 6051826643, which was before changing Gradle versions, it had only 79 MB:

Entry: /home/runner/.gradle/caches/jars-*/*
    Requested Key : instrumented-jars-b8a11de550279f8a817a0ab2b542beef
    Restored  Key : instrumented-jars-b8a11de550279f8a817a0ab2b542beef
              Size: 79 MB (82554214 B)
              (Entry restored: exact match found)
    Saved     Key : instrumented-jars-b720199ac4e76014aa52d3f30b805146
              Size: 79 MB (82616866 B)
              (Entry saved)

In run 6052414218, which was after moving from Gradle 8.1.1 to 8.3, it had 162 MB:

Entry: /home/runner/.gradle/caches/jars-*/*
    Requested Key : instrumented-jars-814c7b61b7b4fceb2ca07e8c056ed523
    Restored  Key : instrumented-jars-814c7b61b7b4fceb2ca07e8c056ed523
              Size: 162 MB (169476876 B)
              (Entry restored: exact match found)
    Saved     Key : instrumented-jars-7aa223b64269e08ca916c9af8e961681
              Size: 162 MB (169794074 B)
              (Entry saved)

An in run 6052482553, which was after moving from Gradle 8.3 to 8.2.1, it growed to 240 MB:

Entry: /home/runner/.gradle/caches/jars-*/*
    Requested Key : instrumented-jars-7aa223b64269e08ca916c9af8e961681
    Restored  Key : instrumented-jars-7aa223b64269e08ca916c9af8e961681
              Size: 162 MB (169794074 B)
              (Entry restored: exact match found)
    Saved     Key : instrumented-jars-7d4b986d0b1c0d1603abcdbbd123a67b
              Size: 240 MB (252023754 B)
              (Entry saved)

So each time changing Gradle version, this single cache entry grows by 80 MB.

@syoon2 syoon2 marked this pull request as draft September 2, 2023 13:02
@syoon2
Copy link
Contributor Author

syoon2 commented Sep 2, 2023

Um... adding another gigabyte and half of cache was not what I was expecting to happen...
Let me take another look tomorrow. In the meantime, I've changed this PR to a draft for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants