-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
0.19.0 Release Plan #367
Comments
@jorisroovers re:GHA — feel free to borrow some ideas from https://github.com/ansible/pylibssh/blob/devel/.github/workflows/build-test-n-publish.yml / https://github.com/cherrypy/cheroot/blob/main/.github/workflows/ci-cd.yml / https://github.com/sphinx-contrib/sphinxcontrib-towncrier/blob/master/.github/workflows/tox-tests.yaml. They implement some of the things I mentioned in https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ and the action I've made for that, like publishing every commit to the default branch to TestPyPI to ensure the pipeline is healthy. Note that these are quite complex because I haven't yet migrated them to rely on reusable workflows but there's a lot of interesting ideas. |
@jorisroovers as the first step, I've got you this PR #375 — it'll allow you to enable branch projection for one check job which ensures that all of the other jobs are green and there's no need to list each matrix factor in the repo settings. |
@webknjaz just a quick acknowledgement of all the issues you've commented on today. Really appreciate all the input! Heads up that it will take me a while to get to all of them - my available time is limited - but will work through them all :-) |
Yep, I noticed the disclaimers :) I was about to file a UX issue but ended up checking whether anybody suggested the same and seeing a few places where I might leave pointers or send a PR. FYI I'm quite proficient with GHA so feel free to tag me if you're stuck with something. |
Just released 0.19.0 🎉 @harens can you publish for macos? Thanks! |
@jorisroovers thanks for the update. Just a heads up, this isn't a trivial update due to hatch requiring a git repo to build from (i.e. not just a standard tar/zip). This isn't normally too big of an issue, but it's made a bit more complicated since it's the Same issue appears on the Homebrew side: Homebrew/homebrew-core#125001. I should be able to fix this for MacPorts, but I'll need some time to think about it. |
Should be all fixed for MacPorts (macports/macports-ports@00073ee). Thanks again for the update @jorisroovers! 🎉 |
Wait, you can't build a wheel from the sdist when using hatch VCS? That's super awful |
Example logs below on my machine. Yeah it's not ideal... Building from PyPi sdist: > cd gitlint_core-0.19.0
> hatch build
[sdist]
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
sys.exit(hatchling())
^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
command(**kwargs)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
for artifact in builder.build(
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
self.metadata.validate_fields()
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
_ = self.version
^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
self._version = self._get_version()
^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
core_metadata = self.core
^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
metadata_hook.update(self.core_raw_metadata)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
urls[key] = formatter.format(url)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
return self.vformat(format_string, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
result.append(self.format_field(obj, format_spec))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
return self.__formatters[formatter](value, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128. Building from github tarball: > cd gitlint-0.19.0/gitlint-core
> hatch build
[sdist]
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/__main__.py", line 6, in <module>
sys.exit(hatchling())
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
command(**kwargs)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/build/__init__.py", line 81, in build_impl
clean_only=clean_only,
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
self.metadata.validate_fields()
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
_ = self.version
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 128, in version
self._version = self._get_version()
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 222, in _get_version
core_metadata = self.core
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 170, in core
self._version = self._get_version(metadata)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 226, in _get_version
version = self.hatch.version.cached
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 1412, in cached
raise type(e)(message) from None
LookupError: Error getting the version from source `vcs`: setuptools-scm was unable to detect version for /Users/harensamarasinghe/Downloads/gitlint-0.19.0.
Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj |
Hmm - I clearly didn't consider this case of rebuilding the wheel directly from the sdist. Although I vaguely recall trying this, perhaps after adopting hatch but before adopting hatch-vcs. hatch-vcs does indeed need the git repo, and not just a shallow clone, but a full one (or at least a including a single tag, not sure): gitlint/.github/workflows/ci.yml Line 28 in 014bfae
I wonder whether this is Perhaps @ofek or @RonnyPfannschmidt have thoughts here? |
The latter, but one solution is to write it to a file |
@ofek can you provide a bit more detail or a pointer, so I can have a look? Thanks! |
At least a single tag is needed Git shallow clones are pretty much sabotage for anything working with Metadata If hatch sdists don't include a pkginfo it would need a fix |
Actually, are you sure that source distributions aren't working for you correctly? It should work out-of-the-box |
I'll try it out myself tomorrow morning my time and do it bit more digging. Entirely possible I made a mistake :-) |
@harens If you look at the beginning of your output it's trying to build a source distribution because without any arguments the default behavior is to build both that and a wheel, similar to the tool |
You would likely want to revert that MacPorts change |
Actually there is a much deeper problem here: the source distribution contains nothing Lines 40 to 41 in 014bfae
|
> wget https://files.pythonhosted.org/packages/29/59/c317a4a39657a4a76cd7dc0c9db4147cca57f13ebe11a469d8c6c0f88715/gitlint_core-0.19.0.tar.gz
> tar -xzvf gitlint_core-0.19.0.tar.gz
> hatch build -t wheel
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
sys.exit(hatchling())
^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
command(**kwargs)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
for artifact in builder.build(
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
self.metadata.validate_fields()
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
_ = self.version
^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
self._version = self._get_version()
^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
core_metadata = self.core
^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
metadata_hook.update(self.core_raw_metadata)
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
urls[key] = formatter.format(url)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
return self.vformat(format_string, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
result.append(self.format_field(obj, format_spec))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
return self.__formatters[formatter](value, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128. Homebrew logs show the same error, although they don't run this exact command. Just to note that the MacPorts build actually runs |
I think it should be |
I see, if you remove this everything will work Lines 48 to 49 in 014bfae
Currently the metadata hook does not read the |
I mentioned how to fix this in #410 (comment). |
Since 0.19.1 is out the door with the fix, I'm closing this issue. Thanks for all the help! |
This issue will be used to track any overarching discussion and work related to
0.19.0
.Scope
As always, everything discussed in this issue is subject to sudden change at my sole discretion. No promises or hard commitments :-)
All related issues will be tagged with the 0.19.0 Milestone.
We also maintain a Kanban board on Github Projects, items marked as TODO are things I'd like to work on for the next release (but again, subject to change).
For 0.19.0, I want to focus on improving (modernizing) our build and CI/CD workflows, with the end-goal of reducing maintenance overhead and making it easier to release.
Timeline
As always, no specific timeline, but probably in the new year (2023).
The text was updated successfully, but these errors were encountered: