Unable to use future in package with testthat if package isn't pre-installed

[gowerc]

Apologies in advance I feel this is such a common use case that I must be doing something wrong but I've tried reading as many relevant docs / issues as I can find and am unable to see a solution.

Simply put, if you create & evaluate a future within a package then it will throw an error if the package isn't installed. This mainly comes up with testing a package via testthat particularly in CICD pipelines. As a minimal example:

# R/file1.R
obj <- function() {
    return(1)
}

#' @export
obj2 <- function() {
    future::value(future::future(obj()))
}

# tests/testthat/test-obj.R
test_that("it works", {
    future::plan(future::multisession)
    expect_equal(obj2(), 1)
})

Then when running testthat:

> devtools::test()
ℹ Testing newpkg
✔ | F W  S  OK | Context
✖ | 1        0 | obj [1.6s]                                                                                          
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Error (test-obj.R:5:5): it works
<FutureError/error/FutureCondition/condition>
Error: there is no package called ‘newpkg’
Backtrace:
    ▆
 1. ├─testthat::expect_equal(obj2(), 1) at test-obj.R:5:5
 2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
 3. │   └─rlang::eval_bare(expr, quo_get_env(quo))
 4. └─newpkg::obj2()
 5.   ├─future::value(future::future(obj())) at newpkg/R/file.R:8:5
 6.   └─future:::value.Future(future::future(obj()))
 7.     ├─future::result(future)
 8.     └─future:::result.ClusterFuture(future)
 9.       └─future:::receiveMessageFromWorker(future, ...)

Expected behavior

I'm not really sure what the correct behaviour here should be.. Potentially an option to force futures to grab objects from within the package instead of trying to load the package which can be toggled on during testing environments ?

Session information

Please share your session information after the error has occurred so that we also see which packages and versions are involved;

> sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: x86_64-apple-darwin23.4.0
Running under: macOS Sonoma 14.6.1

Matrix products: default
BLAS:   /usr/local/Cellar/openblas/0.3.28/lib/libopenblasp-r0.3.28.dylib 
LAPACK: /usr/local/Cellar/r/4.4.1/lib/R/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] newpkg_1.2.6.0001 testthat_3.2.1.1 

loaded via a namespace (and not attached):
 [1] utf8_1.2.4          future_1.34.0       stringi_1.8.4      
 [4] listenv_0.9.1       digest_0.6.37       magrittr_2.0.3     
 [7] pkgload_1.4.0       fastmap_1.2.0       rprojroot_2.0.4    
[10] pkgbuild_1.4.4      sessioninfo_1.2.2   brio_1.1.5         
[13] urlchecker_1.0.1    promises_1.3.0      purrr_1.0.2        
[16] fansi_1.0.6         codetools_0.2-20    cli_3.6.3          
[19] shiny_1.9.1         rlang_1.1.4         future.apply_1.11.2
[22] parallelly_1.38.0   ellipsis_0.3.2      remotes_2.5.0      
[25] withr_3.0.1         cachem_1.1.0        devtools_2.4.5     
[28] parallel_4.4.1      tools_4.4.1         memoise_2.0.1      
[31] httpuv_1.6.15       globals_0.16.3      vctrs_0.6.5        
[34] R6_2.5.1            mime_0.12           lifecycle_1.0.4    
[37] stringr_1.5.1       fs_1.6.4            htmlwidgets_1.6.4  
[40] usethis_3.0.0       miniUI_0.1.1.1      waldo_0.5.3        
[43] pkgconfig_2.0.3     desc_1.4.3          pillar_1.9.0       
[46] later_1.3.2         glue_1.7.0          profvis_0.3.8      
[49] Rcpp_1.0.13         tibble_3.2.1        rstudioapi_0.16.0  
[52] xtable_1.8-4        htmltools_0.5.8.1   compiler_4.4.1    



> future::futureSessionInfo()
*** Package versions
future 1.34.0, parallelly 1.38.0, parallel 4.4.1, globals 0.16.3, listenv 0.9.1

*** Allocations
availableCores():
system 
    12 
availableWorkers():
$system
 [1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
 [7] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"


*** Settings
- future.plan=<not set>
- future.fork.multithreading.enable=<not set>
- future.globals.maxSize=<not set>
- future.globals.onReference=<not set>
- future.resolve.recursive=<not set>
- future.rng.onMisuse=<not set>
- future.wait.timeout=<not set>
- future.wait.interval=<not set>
- future.wait.alpha=<not set>
- future.startup.script=<not set>

*** Backends
Number of workers: 12
List of future strategies:
1. multisession:
   - args: function (..., workers = availableCores(), lazy = FALSE, rscript_libs = .libPaths(), envir = parent.frame())
   - tweaked: FALSE
   - call: future::plan(future::multisession)

*** Basic tests
Main R session details:
    pid     r sysname release
1 80761 4.4.1  Darwin  23.6.0
                                                                                            version
1 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
  nodename machine   login    user effective_user
1  host001  x86_64 user001 user001        user001
Worker R session details:
   worker   pid     r sysname release
1       1 80846 4.4.1  Darwin  23.6.0
2       2 80847 4.4.1  Darwin  23.6.0
3       3 80848 4.4.1  Darwin  23.6.0
4       4 80851 4.4.1  Darwin  23.6.0
5       5 80849 4.4.1  Darwin  23.6.0
6       6 80850 4.4.1  Darwin  23.6.0
7       7 80852 4.4.1  Darwin  23.6.0
8       8 80853 4.4.1  Darwin  23.6.0
9       9 80856 4.4.1  Darwin  23.6.0
10     10 80854 4.4.1  Darwin  23.6.0
11     11 80855 4.4.1  Darwin  23.6.0
12     12 80845 4.4.1  Darwin  23.6.0
                                                                                             version
1  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
2  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
3  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
4  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
5  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
6  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
7  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
8  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
9  Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
10 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
11 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
12 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64
   nodename machine   login    user effective_user
1   host001  x86_64 user001 user001        user001
2   host001  x86_64 user001 user001        user001
3   host001  x86_64 user001 user001        user001
4   host001  x86_64 user001 user001        user001
5   host001  x86_64 user001 user001        user001
6   host001  x86_64 user001 user001        user001
7   host001  x86_64 user001 user001        user001
8   host001  x86_64 user001 user001        user001
9   host001  x86_64 user001 user001        user001
10  host001  x86_64 user001 user001        user001
11  host001  x86_64 user001 user001        user001
12  host001  x86_64 user001 user001        user001
Number of unique worker PIDs: 12 (as expected)

[gowerc]

Whilst trying to find a workaround for this I came across some additional weird behaviour (happy to make a new issue if needed)

A viable workaround seems to be just force the execution of pkgload::load_all() within the sub-process during the unit tests e.g.

test_that("it works", {

    future::plan(future::multisession, workers = 2)

    devnull <- future.apply::future_lapply(
        seq_len(future::nbrOfFreeWorkers()),
        \(i) suppressMessages({
            pkgload::load_all()
        })
    )

    expect_equal(obj2(), 1)
})

I was concerned though this might not accurately represent real behaviour though as pkgload::load_all() by default exports all functions. However changing the call to pkgload::load_all(export_all = FALSE) results in another error during testthat:

> devtools::test()
ℹ Testing newpkg
✔ | F W  S  OK | Context
✖ | 1        0 | obj [1.3s]                                                                                      
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Error (test-obj.R:14:5): it works
Error in `obj()`: could not find function "obj"
Backtrace:
    ▆
 1. ├─testthat::expect_equal(obj2(), 1) at test-obj.R:14:5
 2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
 3. │   └─rlang::eval_bare(expr, quo_get_env(quo))
 4. └─newpkg::obj2()
 5.   ├─future::value(future::future(obj())) at newpkg/R/file.R:8:5
 6.   └─future:::value.Future(future::future(obj()))
 7.     └─future:::signalConditions(...)

[gowerc]

Some additional weird behaviour even outside of the testthat setup:

runtest <- function() {
    future::plan(future::multisession, workers = 2)
    devnull <- future.apply::future_lapply(
        seq_len(future::nbrOfFreeWorkers()),
        \(i) suppressMessages({ pkgload::load_all(export_all = FALSE) })
    )
    expect_equal(obj2(), 1)
}


pkgload::load_all(export_all = FALSE)
runtest()   # Runs fine

pkgload::load_all(export_all = TRUE)
runtest()   # Error in obj() : could not find function "obj"

Whats weird is if insert print(globals::globalsOf(quote(obj2()))) into runtest() it returns the same output in both instances (was wondering if it was detecting different objects for some reason)

$obj2
function() {
    future::value(future::future(obj()))
}
<environment: namespace:newpkg>

attr(,"where")
attr(,"where")$obj2
<environment: package:newpkg>
attr(,"name")
[1] "package:newpkg"
attr(,"path")
[1] "/Users/gowerc/Desktop/Work/newpkg"

attr(,"class")
[1] "Globals" "list"