Failing to connect to schedd

[patrickbryant]

I have been unable to get even simple_example.py to work since last week.

I have tried using older coffea-dask images and tried switching to a new interactive node, all with the same error:

Singularity> python simple_example.py
WARNING: GSI authentication is enabled by your security configuration! GSI is no longer supported.
For details, see https://htcondor.org/news/plan-to-replace-gst-in-htcss/
/opt/conda/lib/python3.8/site-packages/coffea/util.py:154: FutureWarning: In coffea version v0.8.0 (target date: 31 Dec 2022), this will be an error.
(Set coffea.deprecations_as_errors = True to get a stack trace now.)
ImportError: coffea.hist is deprecated
  warnings.warn(message, FutureWarning)
Waiting for at least one worker...
Failed to connect to schedd.
Task exception was never retrieved
future: <Task finished name='Task-46' coro=<_wrap_awaitable() done, defined at /opt/conda/lib/python3.8/asyncio/tasks.py:688> exception=AssertionError()>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/asyncio/tasks.py", line 695, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/opt/conda/lib/python3.8/site-packages/distributed/deploy/spec.py", line 64, in _
    assert self.status == Status.running
AssertionError

[lgray]

This one is really weird as I cannot reproduce it... Does condor_q work for you from within the container?

[patrickbryant]

Hmm, no it does not:

Singularity> condor_q

Neither the environment variable CONDOR_CONFIG,
/etc/condor/, /usr/local/etc/, nor ~condor/ contain a condor_config source.
Either set CONDOR_CONFIG to point to a valid config source,
or put a "condor_config" file in /etc/condor/ /usr/local/etc/ or ~condor/
Exiting.

[patrickbryant]

The env created for me by the shell script has no /etc/condor directory... I gather this directory should exist: #11

[patrickbryant]

I needed to delete the .local dir in my home dir which was replacing the .local in the singularity env for some reason.