PyPI mirror in enclosed environments

[stohrendorf]

Found some issues regarding "private" PyPI mirrors for pypi.org, especially regarding environments where the build environment is restricted such that it can't access anything outside of a corporate network and has to go through some sort of proxy/mirror with a different URL, and where people want to define that mirror globally instead of within pyproject.toml (sadly, can't provide any issues/discussions here right now since I'm in a different environment and discussions about this specific issue seem to be sparse). Most of these issues contained some discussion about authentification, but I doubt that a mirror should require any authentification at all. If you call it a mirror, it should be a mirror. Arguments were made that when you change URLs, it would lead to possibly non-reproducible lockfiles. That is acceptable. However, I think it is possible to globally redirect pypi.org/pythonhosted.org requests to a company mirror. Here's what I'm thinking about.

The assumptions:

• If you call it a mirror, it should be a mirror. As such, if you replace pypi.org or pythonhosted.org with some other domain, behaviour/responses must not change. (I'm not sure how poetry would behave if the mirror would block/remove packages for some reason. If it affects poetry, that's an issue, but let's assume it's a true mirror without package removal.)
• The mirror does not require any authorization.

This leads to the following:

• You could switch the domain (and even path) on request level without any problems.
• As it's a 1:1 mirror, the integrity of poetry.lock is guaranteed inside and outside of a restricted environment.
• If the mirror does, for some reason, modify requests, the behaviour of poetry and the integrity of poetry.lock is undefined. I.e., it might work outside the restriced environment, but it is neither a guaranteed nor defined behaviour.
• As such, you could define a mirror within a restricted environment as this: if a request is made to pythonhosted.org, and a mirror is configured (either within pyproject.toml or globally), the request path from the request to pythonhosted.org is taken and appended to the provided mirror URL. E.g., if poetry would do a request for https://files.pythonhosted.org/packages/c6/85/..., and it would have a configured mirror as https://company.xyz/mirror/, it would result in an effective mirror URL of https://company.xyz/mirror/packages/c6/85/...

Disclaimer:

• I'm not familiar with the domains/URLs involved regarding package search or downloads. Thus, I'm somewhat sure it will be more complicated than my simple idea.
• Looking at the code, I think it would be possible to do such URL substitutions, but I couldn't figure out what problems could arise.

Possible issues:

• Maintainability: This would add another feature requiring maintenance, and it seems to be a possible solution for too few people. As such, the benefits are questionable.
• Implementation: I think it would take quite some effort to even implement a first version of this. Even while it is "simple URL re-writing," it needs to go from the most abstract interface (CLI/global config) down to the deepest level of firing requests.