Remove Conda.jl workarounds

[visr]

In #1959 one of the issues we ran into was

ERROR: Error building `IJulia`: 
ERROR: LoadError: ArgumentError: Path to conda environment is not valid: /home/runner/.julia/conda/3/x86_64
Stacktrace:
 [1] prefix(path::String)
   @ Conda ~/.julia/packages/Conda/zReqD/src/Conda.jl:52
 [2] top-level scope
   @ ~/.julia/packages/Conda/zReqD/src/Conda.jl:57

It's not quite clear why this started happening now, but it is a known issue with Conda.jl: JuliaPy/Conda.jl#251

We work around it with the proposed workaround from that issue like in 2f8b59c.

Before trying this workaround I tried to set CONDA_JL_HOME to make Conda.jl use our pixi environment in 828ed6f, but I couldn't get that to work for unknown reasons.