Skip to content

Commit

Permalink
Consistent python version checks and troubleshooting (apache#42944)
Browse files Browse the repository at this point in the history
Follow up after apache#42766 and apache#42936.

* We do not have to check for minimum Python version for Python 3.9
  any more (as we do not support 3.8 any more)
* While Python 3.12 is not yet fully supported by Apache Beam, we
  should still not allow it for releasing providers, but all other
  commands should support Python 3.12
* When you had pre-commit installed with Python 3.8 before, various
  errors might appear when running pre-commit. Troubleshooting
  was added quoting the errors and explaining what to do.
  • Loading branch information
potiuk authored Oct 11, 2024
1 parent 15df465 commit 92a82eb
Show file tree
Hide file tree
Showing 4 changed files with 42 additions and 9 deletions.
32 changes: 32 additions & 0 deletions dev/breeze/doc/04_troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,38 @@ describe your problem.
stated in `This comment <https://github.com/moby/moby/issues/43361#issuecomment-1227617516>`_ and allows to
run Breeze with no problems.

Cannot import name 'cache' or Python >=3.9 required
---------------------------------------------------

When you see this error:

.. code-block::
ImportError: cannot import name 'cache' from 'functools' (/Users/jarek/Library/Application Support/hatch/pythons/3.8/python/lib/python3.8/functools.py)
or

.. code-block::
ERROR: Package 'blacken-docs' requires a different Python: 3.8.18 not in '>=3.9'
It means that your pre-commit hook is installed with (already End-Of-Life) Python 3.8 and you should reinstall
it and clean pre-commit cache.

This can be done (if you use ``pipx`` to install ``pre-commit``):

.. code-block:: bash
pipx uninstall pre-commit
pipx install pre-commit --python $(which python3.9) --force
pre-commit clean
pre-commit install
If you installed ``pre-commit`` differently, you should remove and reinstall
it (and clean cache) in the way you installed it.


Bad Interpreter Error
---------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -797,7 +797,7 @@ def prepare_provider_packages(
skip_tag_check: bool,
version_suffix_for_pypi: str,
):
check_python_version()
check_python_version(release_provider_packages=True)
perform_environment_checks()
fix_ownership_using_docker()
cleanup_python_generated_files()
Expand Down
12 changes: 5 additions & 7 deletions dev/breeze/src/airflow_breeze/utils/python_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,12 @@ def get_python_version_list(python_versions: str) -> list[str]:
return python_version_list


def check_python_version():
error = False
if not sys.version_info >= (3, 9):
get_console().print("[error]At least Python 3.9 is required to prepare reproducible archives.\n")
error = True
if error:
def check_python_version(release_provider_packages: bool = False):
if not sys.version_info < (3, 12) and release_provider_packages:
get_console().print("[error]Python 3.12 is not supported.\n")
get_console().print(
"[warning]Please reinstall Breeze using Python 3.9 - 3.11 environment.[/]\n\n"
"[warning]Please reinstall Breeze using Python 3.9 - 3.11 environment because not all "
"provider packages support Python 3.12 yet.[/]\n\n"
"For example:\n\n"
"pipx uninstall apache-airflow-breeze\n"
"pipx install --python $(which python3.9) -e ./dev/breeze --force\n"
Expand Down
5 changes: 4 additions & 1 deletion providers/src/airflow/providers/apache/beam/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,10 @@ additional-extras:
- apache-beam[gcp]

# Apache Beam currently does not support Python 3.12
# There is an issue tracking it https://github.com/apache/beam/issues/29149
# There is an issue tracking it https://github.com/apache/beam/issues/29149.
# While the original issue above is closed, Apache Beam still does not support Python 3.12
# because the dill version used by them break our PythonVirtualenvOperator when dill is enabled
# See https://github.com/apache/beam/issues/32617
excluded-python-versions: ['3.12']

integrations:
Expand Down

0 comments on commit 92a82eb

Please sign in to comment.