Add Gradle build cache folder

[carlonzo]

Checklist

• I've read and followed the Contribution Guidelines
• step.yml and README.md is updated with the changes (if needed)

Context

Current cache step ignores the build cache. Opening issues here is only for bug reports so I tried to open the discussion with a Pull Request.

Switching from a custom cache step where we had build cache store to this one, increased build time by a factor of ~5.

Was excluding build cache a conscious decision? If so would you consider input booleans to configure it?

Changes

Gradle stores build cache it the ~/.gradle/caches/build-cache-*
Adds the folder in the list.

[ofalvai]

@carlonzo the reason we don't add the Gradle build cache to the cache paths by default is that it would lead to an ever-growing cache archive. A cache entry is only invalidated if its key changes, and this only happens when one of the files defined in the checksum function changes. For dependency caching, it's easy to define the few Gradle files in checksum that change whenever a dependency changes. For the build cache, this is not possible, other than checksumming all the source files in the repo, but then it wouldn't be an effective cache key.

Feel free to define your own cache rules with the generic save-cache step as long as it benefits your builds, just keep in mind that it gets slower and slower generally as more files are added to the cache without invalidating the old contents.

[carlonzo]

@ofalvai thanks for the detailed explanation, what you say makes sense. There is a problem of ever-growing cache which we experienced directly very recently: we have been having a save-cache step caching ~/.gradle/cache that caused our cache to be about 12-13GB.

But I believe there is a problem even with the current save-gradle-cache step: the folders that are cached can also be ever-growing. Every new dependency version change within the project forces a new download of an aar/jar. We are currently running a clean step before the save cache step that removes every file in ~/.gradle/cache older than 15 days. Not a perfect solution but helps keeping the cache lean.

Would that be something you would consider for this step? if so, build-cache can come back and re-enable incremental builds in Bitrise

[carlonzo]

@ofalvai anything about this? happy to close if you dont agree with it. just lets keep in mind that the new step increases build time as its not caching incremental builds

[rubixhacker]

I think it would be great if the plugin was expanded to offer more caching if the consumer agrees to the risk of the ever growing cache size.

Maybe check boxes for each build-cache and transforms and a clear warning that it will cause the cache size to increase.

Supporting incremental builds for workflows and Remote Gradle cache would be extremely beneficial

[carlonzo]

@ofalvai any update on this one?

[viktorbenei]

Hi @carlonzo 👋

We just released Bitrise Build Cache, which solves this problem in a significantly better way: https://bitrise.io/platform/caching/gradle

You can find the recent announcement posts here, with more details about how you can get started: https://bitrise.io/blog/post/cache-builds-anywhere-with-our-latest-updates-to-bitrise-build-cache

Bitrise Build Cache implements Gradle's remote build cache interface. This means that build cache objects are uploaded and downloaded only when gradle needs them (data and time efficiency), and data eviction happens on a per object level, which means those objects which aren't used for a while are automatically deleted (no infinitely growing cache).

It's a paid for add-on, but you can start a 30 days no-strings-attached trial on app.bitrise.io , and it provides significant performance improvement over storing the build cache in the classic dependency cache (which is used by the Save Gradle Cache step).