Deploy and Build Site Job failing

[parkyed]

Hi @tobyhodges , I wonder if you are able to assist me here. My Deploy and Build Site Job is failing on this project. I have been unable to figure out why it no longer works. The course is now complete, and I am hoping to begin a pilot in the next month or so.

Thanks for any advice you are able to provide,
Ed

[tobyhodges]

It looks like I forgot to configure your repository for automatic PRs to update the workflows. Sorry for the oversight. I have fixed that now so you can expect to start receiving pull requests from the @carpentries-bot account occasionally to update your GitHub Actions workflows. (These can be safely merged when they appear.)

For now, you can manually update the workflow files by following the instructions here: https://carpentries.github.io/sandpaper-docs/update.html#deployment

I suspect that updating the workflows will solve the problem, and it is certainly a good place to start.

[parkyed]

Hi @tobyhodges, thanks very much for your assistance with this. I have updated the workflows as you advised, and merged the pull requests I received.

Unfortunately this hasn't resolved this issue and I am still unable to build and deploy my site. Looking at the logs, the specific issue is with the setup package cache job. the logs for the most recently failed job is here - https://github.com/carpentries-incubator/rna-seq-data-for-ml/actions/runs/6470423763/job/17566604303#step:25:665

It appears that the error is caused by a package version conflict. Specifically, the package vctrs is a dependancy of multiple packages installed and the version updated to a previous version somewhere in the installation process.

The specific error message is:

The following package(s) have unsatisfied dependencies:

• purrr requires vctrs (>= 0.6.3), but version 0.6.2 is installed

line 552 shows that this package has had the version changed during the package installation job.

• vctrs [repo: RSPM -> CRAN; ver: 0.6.3 -> 0.6.2]

Is there a way I can force the final package version in my code that will be respected when the job runs? Are you aware of any other solution to this? Sorry to bug you again with this issue.

Thanks,
Ed

[tobyhodges]

Thanks for digging into the build logs and identifying the problem, @parkyed.

Please try the solution described at the end of the opening post of this issue carpentries/workbench#40 (comment):

The solution is to update the cache before running the manage dependencies, which is achieved either through sandpaper::update_cache() or manually running the 03 Maintain: Update Cache workflow as shown here: carpentries/workbench-template-rmd#31

Post back here to let me know if that helped or if we need to keep digging.

[parkyed]

Hi @tobyhodges, thanks again for the quick help here.

Running job 03, Update Cache workflow also failed. The error relates to installing packages that are not available on CRAN. These packages are however available on Bioconductor. The error message is as follows:

  The following package(s) were not installed successfully:
  - [DESeq2]: package 'DESeq2' is not available
  - [SummarizedExperiment]: package 'SummarizedExperiment' is not available
  - [GEOquery]: package 'GEOquery' is not available
  - [Biobase]: package 'Biobase' is not available
  You may need to manually download and install these packages.
  
  Error in renv_snapshot_validate_report(valid, prompt, force) : 
    aborting snapshot due to pre-flight validation failure
  Calls: <Anonymous> ... withVisible -> <Anonymous> -> renv_snapshot_validate_report
  Execution halted
  Error: Process completed with exit code 1.

It appears from the logs that when running the update cache from the job 03: Maintain: Update Cache workflow, that where packages are not available on CRAN, the job doesn't then look elsewhere to find these packages. However, when running job 01: Build and Maintain site, within the Setup Package Cache part of the job, when packages are not found on CRAN, the job appears to then look for them on bioconductor. See logs below and here -https://github.com/carpentries-incubator/rna-seq-data-for-ml/actions/runs/6479705635/job/17593771555#step:25:665 line 387 onwards. In this case these packages are successfully downloaded, however the job overall fails due to the race condition around package versions.

Is there an inconsistency between how job 01 and job 03 behave in relation to packages not found on CRAN that may be causing the race condition around package dependencies?

Output from lines 387 onwards of most recent attempt to build site:

! Attempting to install missing packages assuming bioc
  # Downloading packages -------------------------------------------------------
  - Downloading GEOquery from BioCsoft ...        OK [13.1 Mb in 0.36s]
  - Downloading readr from CRAN ...               OK [291.1 Kb in 0.18s]
  - Downloading clipr from CRAN ...               OK [21.4 Kb in 0.13s]
  - Downloading crayon from CRAN ...              OK [39.6 Kb in 0.13s]
  - Downloading hms from CRAN ...                 OK [42.4 Kb in 0.14s]
  - Downloading vroom from CRAN ...               OK [732.6 Kb in 0.19s]
  - Downloading bit64 from CRAN ...               OK [131.9 Kb in 0.15s]
  - Downloading bit from CRAN ...                 OK [808.3 Kb in 0.2s]
  - Downloading tzdb from CRAN ...                OK [572.6 Kb in 0.18s]
  - Downloading cpp11 from CRAN ...               OK [293.8 Kb in 0.17s]
  - Downloading progress from CRAN ...            OK [28.8 Kb in 0.15s]
  - Downloading prettyunits from CRAN ...         OK [95.2 Kb in 0.15s]
  - Downloading xml2 from carpentries ...         OK [295.6 Kb in 0.25s]
  - Downloading data.table from CRAN ...          OK [5.1 Mb in 0.23s]
  - Downloading tidyr from CRAN ...               OK [804.2 Kb in 0.18s]
  - Downloading purrr from CRAN ...               OK [215.7 Kb in 0.17s]
  - Downloading limma from BioCsoft ...           OK [1.4 Mb in 0.27s]
  - Downloading BiocVersion from BioCsoft ...     OK [971 bytes in 0.11s]
  - Downloading curl from CRAN ...                OK [697.4 Kb in 0.2s]
  - Downloading R.utils from CRAN ...             OK [357.3 Kb in 0.21s]
  - Downloading R.oo from CRAN ...                OK [375.4 Kb in 0.19s]
  - Downloading R.methodsS3 from CRAN ...         OK [23.6 Kb in 0.15s]
  - Downloading Biobase from BioCsoft ...         OK [1.7 Mb in 0.17s]
  - Downloading BiocGenerics from BioCsoft ...    OK [47.1 Kb in 0.25s]
  - Downloading SummarizedExperiment from BioCsoft ... OK [668.5 Kb in 0.14s]
  - Downloading S4Vectors from BioCsoft ...       OK [817 Kb in 0.14s]
  - Downloading IRanges from BioCsoft ...         OK [469.6 Kb in 0.14s]
  - Downloading GenomeInfoDb from BioCsoft ...    OK [3.4 Mb in 0.16s]
  - Downloading RCurl from CRAN ...               OK [714.3 Kb in 0.18s]
  - Downloading bitops from CRAN ...              OK [10.6 Kb in 0.13s]
  - Downloading GenomeInfoDbData from BioCann ... OK [11.5 Mb in 0.2s]
  - Downloading S4Arrays from BioCsoft ...        OK [274.5 Kb in 0.12s]
  - Downloading abind from CRAN ...               OK [21.3 Kb in 0.14s]
  - Downloading DelayedArray from BioCsoft ...    OK [613.5 Kb in 0.13s]
  - Downloading MatrixGenerics from BioCsoft ...  OK [30.8 Kb in 0.13s]
  - Downloading matrixStats from CRAN ...         OK [205.7 Kb in 0.18s]
  - Downloading GenomicRanges from BioCsoft ...   OK [1.1 Mb in 0.14s]
  - Downloading XVector from BioCsoft ...         OK [66.2 Kb in 0.11s]
  - Downloading zlibbioc from BioCsoft ...        OK [159.5 Kb in 0.12s]
  - Downloading DESeq2 from BioCsoft ...          OK [2 Mb in 0.15s]
  - Downloading BiocParallel from BioCsoft ...    OK [973.8 Kb in 0.14s]
  - Downloading futile.logger from CRAN ...       OK [17 Kb in 0.14s]
  - Downloading lambda.r from CRAN ...            OK [25.1 Kb in 0.15s]
  - Downloading formatR from CRAN ...             OK [93.8 Kb in 0.15s]
  - Downloading futile.options from CRAN ...      OK [3.8 Kb in 0.15s]
  - Downloading snow from CRAN ...                OK [20 Kb in 0.13s]
  - Downloading BH from CRAN ...                  OK [13.3 Mb in 0.28s]
  - Downloading locfit from CRAN ...              OK [196.4 Kb in 0.18s]
  - Downloading RcppArmadillo from CRAN ...       OK [1.3 Mb in 0.21s]
  Successfully downloaded 49 packages in 23 seconds.
  

[tobyhodges]

Pinging @froggleston @klbarnes20 @ErinBecker @zkamvar I have reached the limit of my Workbench knowledge. Please could you explore what might need to be done here?

To summarise the issue as I understand it:

1. The lesson cannot be built because of conflicting dependencies (something similar to what was described in R Lessons: document how to run update_cache() when there is a package conflict carpentries/workbench#40, I think)
2. Following Zhian's solution described in that issue leads to errors because of bioconductor packages used in the lesson, which are not available from CRAN. The 03 Maintain: Update Package Cache workflow does not seem to know what to do about packages that are not available from CRAN.

[tobyhodges]

@parkyed perhaps you could try merging #6, and see if that helps at all?

[froggleston]

Could the renv.lock file in the PR be edited to force certain package versions? In my limited knowledge, step 01 has an explicit cache update task, but 03 doesn't? I don't think updating this is correct though as this would affect every lesson built.

In my experience in python with dependency issues, locking versions is sufficient until subsequent packages are updated in repositories?

[ErinBecker]

@froggleston @klbarnes20 and I tested the fix proposed by @tobyhodges above by checking out PR #6. This helped some - it updated all of the packages except for Rcpp. There appears to still be some package dependency issue with Rcpp.

Old error message:

The following required packages are not installed:
- cpp11       [required by BiocParallel]
- crayon      [required by S4Arrays]
- curl        [required by GEOquery]
- data.table  [required by GEOquery]
- Rcpp        [required by DESeq2, plyr, RcppArmadillo, and 1 other]
- readr       [required by GEOquery]
- tidyr       [required by GEOquery]
- xml2        [required by GEOquery]
Consider reinstalling these packages before snapshotting the lockfile.

New error message after checking out PR #6

The following required packages are not installed:
- Rcpp  [required by DESeq2, plyr, RcppArmadillo, and 1 other]
Consider reinstalling these packages before snapshotting the lockfile.

This is strange because PR #6 is updating Rcpp to version 1.0.11, which is in fact on CRAN.

[froggleston]

Interestingly, I decided to delete the renv folder from my local update/packages checked out branch and the build succeeds...

[zkamvar]

I had some time to speak with @froggleston about this and from his summary, the solution is indeed to manually (as in on a local machine) remove the renv/ folder and re-run manage_deps() and update_cache() from sandpaper.

The reason we find ourselves in this situation is because of an unfortunate race condition with package dependencies. This is in part because the original lockfile was never updated since the inception of this package. Ideally, the lockfile should be updated once every month.

Luckily, the solution that @froggleston found is the correct solution because removing the renv/ folder resets the lesson package cache to a state that it would be as if it were freshly created.

[parkyed]

@tobyhodges, @froggleston @klbarnes20 @ErinBecker @zkamvar - thanks very much for your comments and help with this. I merged in #6 and this seemed to solve the issue. For now, I'll hold off deleting my local renv folder - I'm not exactly sure which one to delete. Hopefully this is now solved. Thanks very much again all!

[zkamvar]

@parkyed,

In any workbench lesson, if there is a problem with the renv cache, you can use the following commands to reset it inside the lesson:

rm -r renv/
git restore renv/

[froggleston]

Hi @parkyed - was this resolved for you? If so I can close this issue. If not, please let us know if there's anything else we can help with!

[parkyed]

Hi @froggleston, thanks for following up on this. Yes, this has been resolved, and i'm happy for the issue to be closed. I'll close it with this comment. Thanks!