Effective Caching w/ GitHub Actions

[iBotPeaches]

Hi,

Our build time went up 4-6 minutes after installing realm for React Native and I was quite confused, because the package was only 16mb, which is higher but shouldn't explain a jump of 2.1 GB into cache and large minute jump. Doing something like this:

      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn cache dir)"

      - uses: actions/cache@v3
        id: yarn-cache
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: ${{ runner.os }}-yarn-

That then takes a bit of time to restore.

Received 2796655535 of 2796655535 (100.0%), 17.6 MBs/sec
Cache Size: ~2667 MB (2796655535 B)

Which takes anywhere from 4 to 6 minutes, so I don't think GitHub Actions like us much anymore. I ended up digging into it and found some archive files in node_modules/realm/vendor/realm-ios/*.a files that I think appear during compilation.

➜  realm-ios: du -sh  
1.4G    .
➜  realm-ios: du -sh *
232K    doc
4.0K    download-realm.lock
2.3M    include
  0B    lib
 81M    librealm-parser-ios-dbg.a
 24M    librealm-parser-ios.a
 22M    librealm-parser-macosx-dbg.a
3.1M    librealm-parser-macosx.a
 41M    librealm-parser-tvos-dbg.a
 12M    librealm-parser-tvos.a
 78M    librealm-parser-watchos-dbg.a
 34M    librealm-parser-watchos.a
307M    librealm-sync-ios-dbg.a
 70M    librealm-sync-ios.a
 92M    librealm-sync-maccatalyst-dbg.a
 10M    librealm-sync-maccatalyst.a
 95M    librealm-sync-macosx-dbg.a
 10M    librealm-sync-macosx.a
160M    librealm-sync-tvos-dbg.a
 35M    librealm-sync-tvos.a
293M    librealm-sync-watchos-dbg.a
 99M    librealm-sync-watchos.a
4.0K    version.txt
➜  realm-ios

I'm wondering from the source itself what is the recommendation here. Do I win out restoring a cache of large size or do you think the compilation overhead of recompiling wins and I should find a way to ignore these files from the cache created?

[iBotPeaches]

Okay, just leaving an update that this isn't realm truly to blame, a few other packages are heavy. Realm just pushed things over the edge and caused the investigation with the largest single cache size at 412M. Above was pictured node_modules, where I incorrectly assumed that was what GH Actions was catching.

It caches yarn cache dir, which returns a more global area that caches all projects locally. So what happened here over time was as follows.

This matches with our installation as we bounced around a few major versions aiming for something with Hermes compatibility.

• Install Realm x.y.z
• Upgrade to Realm x.y.z.1
• Swap to Realm x.y.z

So our global cache ends up looking like:

 412M    npm-realm-10.16.0-1d33d6513e7a84e699a25095421440d83161b73c-integrity
 64M    npm-realm-6.1.6-e8b45257b14fd865a5bbc8f37e9c8593c05aab42-integrity

A large jump for sure, but I'm going to trade having a cache for 99% of packages and explicitly dropping a file cache for the heavy packages. With something like

yarn cache clean realm

Prior to cache save. So this makes our cache file quite small and restores very quick. We ingest some IO with a network download, but until the performance and ability for the cache system to be smart enough to not cache "older" packages. I'm afraid I'll make that trade.

I am however curious how the global cache for realm6 was 64M and realm10 was 412M.