trainer.test() is showing error when run

[Daremitsu1]

Hello,

While running trainer.test getting the above error.

Below is the full error:

---------------------------------------------------------------------------
MisconfigurationException                 Traceback (most recent call last)
Input In [80], in <cell line: 1>()
----> 1 trainer.test()

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\trainer.py:936, in Trainer.test(self, model, dataloaders, ckpt_path, verbose, datamodule)
    910 r"""
    911 Perform one evaluation epoch over the test set.
    912 It's separated from fit to make sure you never run on your test set until you want to.
   (...)
    933     The length of the list corresponds to the number of test dataloaders used.
    934 """
    935 self.strategy.model = model or self.lightning_module
--> 936 return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\trainer.py:721, in Trainer._call_and_handle_interrupt(self, trainer_fn, *args, **kwargs)
    719         return self.strategy.launcher.launch(trainer_fn, *args, trainer=self, **kwargs)
    720     else:
--> 721         return trainer_fn(*args, **kwargs)
    722 # TODO: treat KeyboardInterrupt as BaseException (delete the code below) in v1.7
    723 except KeyboardInterrupt as exception:

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\trainer.py:983, in Trainer._test_impl(self, model, dataloaders, ckpt_path, verbose, datamodule)
    980 self._tested_ckpt_path = self.ckpt_path  # TODO: remove in v1.8
    982 # run test
--> 983 results = self._run(model, ckpt_path=self.ckpt_path)
    985 assert self.state.stopped
    986 self.testing = False

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\trainer.py:1158, in Trainer._run(self, model, ckpt_path)
   1155 self._callback_connector._attach_model_callbacks()
   1156 self._callback_connector._attach_model_logging_functions()
-> 1158 verify_loop_configurations(self)
   1160 # hook
   1161 log.detail(f"{self.__class__.__name__}: preparing data")

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\configuration_validator.py:46, in verify_loop_configurations(trainer)
     44     __verify_eval_loop_configuration(trainer, model, "val")
     45 elif trainer.state.fn == TrainerFn.TESTING:
---> 46     __verify_eval_loop_configuration(trainer, model, "test")
     47 elif trainer.state.fn == TrainerFn.PREDICTING:
     48     __verify_eval_loop_configuration(trainer, model, "predict")

File ~\anaconda3\lib\site-packages\pytorch_lightning\trainer\configuration_validator.py:197, in __verify_eval_loop_configuration(trainer, model, stage)
    193 # -----------------------------------
    194 # verify model has an eval_dataloader
    195 # -----------------------------------
    196 if not has_loader:
--> 197     raise MisconfigurationException(f"No `{loader_name}()` method defined to run `Trainer.{trainer_method}`.")
    199 # predict_step is not required to be overridden
    200 if stage == "predict":

MisconfigurationException: No `test_dataloader()` method defined to run `Trainer.test`.

However, I have already defined a test_dataloader function:

class ARCQADataModule(pl.LightningDataModule):

  def __init__(
      self,
      train_df: pd.DataFrame,
      test_df: pd.DataFrame,
      tokenizer: T5Tokenizer,
      batch_size: int = 8,
      source_max_token_len = 396,
      target_max_token_len = 32
    ):
      super().__init__()
      self.batch_size = batch_size
      self.train_df = train_df
      self.test_df = test_df
      self.tokenizer = tokenizer
      self.source_max_token_len = source_max_token_len
      self.target_max_token_len = target_max_token_len

  def setup(self, stage=None):
    self.train_dataset = ARCQADataset(
        self.train_df,
        self.tokenizer,
        self.source_max_token_len,
        self.target_max_token_len
    )

    self.test_dataset = ARCQADataset(
        self.test_df,
        self.tokenizer,
        self.source_max_token_len,
        self.target_max_token_len
    )

  def train_dataloader(self):
    return DataLoader(
        self.train_dataset,
        batch_size = self.batch_size,
        shuffle = True,
        num_workers = 0
    )

  def val_dataloader(self):
    return DataLoader(
        self.train_dataset,
        batch_size = 1,
        shuffle = True,
        num_workers = 0  
    )

  def test_dataloader(self):
    return DataLoader(
        self.train_dataset,
        batch_size = 1,
        shuffle = True,
        num_workers = 4  
    )

Is there something I'm doing wrong here?

[rohitgr7]

can you reproduce the behavior using BoringModel?

[Daremitsu1]

can you reproduce the behavior using BoringModel?

Solved the issue by changing the num_workers in test_loader to 0. This seems to be a problem in pytorchlightning. If the person is not using GPU but on CPU the num_workers does not work. My system can have 8 num_workers assigned but I am not able to.

The training process will not start.

[rohitgr7]

it shouldn't happen. I just tried and it's working with num_workers=4 on CPU. Can you share a failing example to reproduce the issue?