Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.19.0 Release Plan #367

Closed
jorisroovers opened this issue Nov 21, 2022 · 23 comments
Closed

0.19.0 Release Plan #367

jorisroovers opened this issue Nov 21, 2022 · 23 comments
Assignees
Labels
development Issues that are not user-facing but related to gitlint development
Milestone

Comments

@jorisroovers
Copy link
Owner

This issue will be used to track any overarching discussion and work related to 0.19.0.

Scope

As always, everything discussed in this issue is subject to sudden change at my sole discretion. No promises or hard commitments :-)

All related issues will be tagged with the 0.19.0 Milestone.

We also maintain a Kanban board on Github Projects, items marked as TODO are things I'd like to work on for the next release (but again, subject to change).

For 0.19.0, I want to focus on improving (modernizing) our build and CI/CD workflows, with the end-goal of reducing maintenance overhead and making it easier to release.

Timeline

As always, no specific timeline, but probably in the new year (2023).

@jorisroovers jorisroovers self-assigned this Nov 21, 2022
@jorisroovers jorisroovers added the development Issues that are not user-facing but related to gitlint development label Nov 21, 2022
@jorisroovers jorisroovers added this to the 0.19.0 milestone Nov 21, 2022
@jorisroovers jorisroovers moved this to In Progress in gitlint Nov 21, 2022
@jorisroovers jorisroovers pinned this issue Nov 21, 2022
@webknjaz
Copy link
Contributor

@jorisroovers re:GHA — feel free to borrow some ideas from https://github.com/ansible/pylibssh/blob/devel/.github/workflows/build-test-n-publish.yml / https://github.com/cherrypy/cheroot/blob/main/.github/workflows/ci-cd.yml / https://github.com/sphinx-contrib/sphinxcontrib-towncrier/blob/master/.github/workflows/tox-tests.yaml. They implement some of the things I mentioned in https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ and the action I've made for that, like publishing every commit to the default branch to TestPyPI to ensure the pipeline is healthy. Note that these are quite complex because I haven't yet migrated them to rely on reusable workflows but there's a lot of interesting ideas.

@webknjaz
Copy link
Contributor

@jorisroovers as the first step, I've got you this PR #375 — it'll allow you to enable branch projection for one check job which ensures that all of the other jobs are green and there's no need to list each matrix factor in the repo settings.

@jorisroovers
Copy link
Owner Author

@webknjaz just a quick acknowledgement of all the issues you've commented on today. Really appreciate all the input!

Heads up that it will take me a while to get to all of them - my available time is limited - but will work through them all :-)

@webknjaz
Copy link
Contributor

Yep, I noticed the disclaimers :) I was about to file a UX issue but ended up checking whether anybody suggested the same and seeing a few places where I might leave pointers or send a PR. FYI I'm quite proficient with GHA so feel free to tag me if you're stuck with something.

@jorisroovers jorisroovers moved this from In Progress to 0.19.0 in gitlint Mar 7, 2023
@jorisroovers
Copy link
Owner Author

jorisroovers commented Mar 7, 2023

Just released 0.19.0 🎉
https://github.com/jorisroovers/gitlint/releases/tag/v0.19.0

@harens can you publish for macos? Thanks!

@harens
Copy link

harens commented Mar 7, 2023

@jorisroovers thanks for the update. Just a heads up, this isn't a trivial update due to hatch requiring a git repo to build from (i.e. not just a standard tar/zip). This isn't normally too big of an issue, but it's made a bit more complicated since it's the gitlint-core directory that needs to be built, not the higher level gitlint directory.

Same issue appears on the Homebrew side: Homebrew/homebrew-core#125001. I should be able to fix this for MacPorts, but I'll need some time to think about it.

@harens
Copy link

harens commented Mar 7, 2023

Should be all fixed for MacPorts (macports/macports-ports@00073ee). Thanks again for the update @jorisroovers! 🎉

@sigmavirus24
Copy link
Collaborator

Wait, you can't build a wheel from the sdist when using hatch VCS? That's super awful

@harens
Copy link

harens commented Mar 7, 2023

Wait, you can't build a wheel from the sdist when using hatch VCS? That's super awful

Example logs below on my machine. Yeah it's not ideal...

Building from PyPi sdist:

> cd gitlint_core-0.19.0
> hatch build
[sdist]
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
        ^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
                    ^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
    metadata_hook.update(self.core_raw_metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
    urls[key] = formatter.format(url)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
    result.append(self.format_field(obj, format_spec))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
    return self.__formatters[formatter](value, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
    'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
    return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128.

Building from github tarball:

> cd gitlint-0.19.0/gitlint-core
> hatch build
[sdist]
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/build/__init__.py", line 81, in build_impl
    clean_only=clean_only,
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 170, in core
    self._version = self._get_version(metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 226, in _get_version
    version = self.hatch.version.cached
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 1412, in cached
    raise type(e)(message) from None
LookupError: Error getting the version from source `vcs`: setuptools-scm was unable to detect version for /Users/harensamarasinghe/Downloads/gitlint-0.19.0.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj

@jorisroovers
Copy link
Owner Author

Hmm - I clearly didn't consider this case of rebuilding the wheel directly from the sdist. Although I vaguely recall trying this, perhaps after adopting hatch but before adopting hatch-vcs.

hatch-vcs does indeed need the git repo, and not just a shallow clone, but a full one (or at least a including a single tag, not sure):

fetch-depth: 0 # checkout all history, needed for hatch versioning

I wonder whether this is hatch-vcs specific, or dictated by setuptools-scm - I'd have to dig in.

Perhaps @ofek or @RonnyPfannschmidt have thoughts here?

@ofek
Copy link

ofek commented Mar 7, 2023

The latter, but one solution is to write it to a file

@jorisroovers
Copy link
Owner Author

@ofek can you provide a bit more detail or a pointer, so I can have a look? Thanks!

@RonnyPfannschmidt
Copy link

At least a single tag is needed

Git shallow clones are pretty much sabotage for anything working with Metadata

If hatch sdists don't include a pkginfo it would need a fix

@ofek
Copy link

ofek commented Mar 7, 2023

Actually, are you sure that source distributions aren't working for you correctly? It should work out-of-the-box

@jorisroovers
Copy link
Owner Author

Actually, are you sure that source distributions aren't working for you correctly? It should work out-of-the-box

I'll try it out myself tomorrow morning my time and do it bit more digging. Entirely possible I made a mistake :-)

@ofek
Copy link

ofek commented Mar 7, 2023

@harens If you look at the beginning of your output it's trying to build a source distribution because without any arguments the default behavior is to build both that and a wheel, similar to the tool build. Instead, do hatch build -t wheel or python -m build --wheel

@ofek
Copy link

ofek commented Mar 7, 2023

You would likely want to revert that MacPorts change

@ofek
Copy link

ofek commented Mar 7, 2023

Actually there is a much deeper problem here: the source distribution contains nothing

gitlint/pyproject.toml

Lines 40 to 41 in 014bfae

[tool.hatch.build]
exclude = ["*"]

sdist

@harens
Copy link

harens commented Mar 7, 2023

Instead, do hatch build -t wheel or python -m build --wheel

> wget https://files.pythonhosted.org/packages/29/59/c317a4a39657a4a76cd7dc0c9db4147cca57f13ebe11a469d8c6c0f88715/gitlint_core-0.19.0.tar.gz
> tar -xzvf gitlint_core-0.19.0.tar.gz
> hatch build -t wheel
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
        ^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
                    ^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
    metadata_hook.update(self.core_raw_metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
    urls[key] = formatter.format(url)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
    result.append(self.format_field(obj, format_spec))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
    return self.__formatters[formatter](value, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
    'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
    return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128.

Homebrew logs show the same error, although they don't run this exact command.

Just to note that the MacPorts build actually runs python3.11 -m build --wheel --no-isolation --outdir ..., but it's all the same.

@harens
Copy link

harens commented Mar 7, 2023

Actually there is a much deeper problem here: the source distribution contains nothing

I think it should be gitlint_core, not gitlint.

@ofek
Copy link

ofek commented Mar 7, 2023

I see, if you remove this everything will work

gitlint/pyproject.toml

Lines 48 to 49 in 014bfae

'Source Archive' = "https://github.com/jorisroovers/gitlint/archive/{commit_hash}.zip"
'Source Commit' = "https://github.com/jorisroovers/gitlint/tree/{commit_hash}"

Currently the metadata hook does not read the PKG-INFO file from the source distribution and tries the same Git logic. Can you please open an issue?

@webknjaz
Copy link
Contributor

webknjaz commented Mar 9, 2023

Building from github tarball:

I mentioned how to fix this in #410 (comment).
TL;DR — set up .git_archival.txt properly.

@jorisroovers
Copy link
Owner Author

Since 0.19.1 is out the door with the fix, I'm closing this issue.
Development of 0.20.0 is tracked in #462

Thanks for all the help!

@jorisroovers jorisroovers unpinned this issue Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Issues that are not user-facing but related to gitlint development
Projects
Status: 0.19.0
Development

No branches or pull requests

6 participants