Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(BA-458): Upgrade pants to 2.23.1 and replace the scie plugin with intrinsic pex option #3377

Merged
merged 10 commits into from
Jan 7, 2025

Conversation

achimnol
Copy link
Member

@achimnol achimnol commented Jan 5, 2025

resolves #3385 (BA-458): This PR upgrades Pantsbuild from 2.21 to 2.23 release, and also upgrades PEX from 2.10 to 2.28.1 alongside. Thanks to https://docs.pex-tool.org/scie.html (a-scie/lift#116), we can now use PEX to build scie binaries.

The new BUILD configs preserve the same command format, so we don't have to update the CI and documentation. Just run the same way: pants --tag=scie --tag=+lazy package ::.

The most significant advantage is that now pants tracks the file changes properly and the scie binaries will be rebuilt if you modify the source files in the working copy, which greatly eases repeated testing of the scie binaries in development setups.

refs #3285 (BA-4): Upgrading Pantsbuild also enables Python 3.13 support added in Pants v2.22.

Additionally, this PR does:

  • Strip out unimplemented details from the towncrier plugin to adapt with the pants plugin API changes.
  • Update the build-scies job in the GHA workflow to properly suffix the scie binaries.
    • The latest PEX provides --scie-name-style option to automate this suffix attachment, but it confuses Pantsbuild to pick up the generated output file. Hence, we do the required suffixing by oursevles.
    • Add scripts/get-platform-suffix.py for use in the workflow.
  • Let the build-scies job merge the checksums into a single file (checksum.txt) to reduce the number of GitHub release artifacts.
  • Refactor the BUILD files and move the common scie-specific config options for pex_binary targets to tools/build-macros.py.
  • Add missing cross-import visibility rule for ai.backend.account_manager.

Checklist: (if applicable)

  • Milestone metadata specifying the target backport version
  • Mention to the original issue

@achimnol achimnol added this to the 25Q1 milestone Jan 5, 2025
@achimnol achimnol added area:infrastructure Infrastructure-related issues area:upstream Issues with root cause from upstream 3rd parties labels Jan 5, 2025
@achimnol achimnol self-assigned this Jan 5, 2025
@github-actions github-actions bot added comp:installer Related to Installer size:XL 500~ LoC labels Jan 5, 2025
@achimnol achimnol changed the title chore: Upgrade pants to 2.23.1 and replace the scie plugin with intrinsic pex option chore(BA-436): Upgrade pants to 2.23.1 and replace the scie plugin with intrinsic pex option Jan 6, 2025
@achimnol achimnol changed the title chore(BA-436): Upgrade pants to 2.23.1 and replace the scie plugin with intrinsic pex option chore(BA-458): Upgrade pants to 2.23.1 and replace the scie plugin with intrinsic pex option Jan 6, 2025
@achimnol achimnol force-pushed the chore/upgrade-pants-and-retire-scie-plugin branch from 3462021 to f2ded82 Compare January 7, 2025 08:11
@achimnol
Copy link
Member Author

achimnol commented Jan 7, 2025

Confirmed the updated workflow step works as expected:
image

pants.toml Show resolved Hide resolved
@achimnol achimnol enabled auto-merge January 7, 2025 14:33
@achimnol achimnol added this pull request to the merge queue Jan 7, 2025
@achimnol achimnol added the require:pants-export Dependencies updated. You need to run `pants export --resolve=python-default` to make it working. label Jan 7, 2025
Merged via the queue into main with commit d2bf8bb Jan 7, 2025
21 checks passed
@achimnol achimnol deleted the chore/upgrade-pants-and-retire-scie-plugin branch January 7, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:infrastructure Infrastructure-related issues area:upstream Issues with root cause from upstream 3rd parties comp:installer Related to Installer require:pants-export Dependencies updated. You need to run `pants export --resolve=python-default` to make it working. size:XL 500~ LoC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update pants and replace our custom scie plugin with the intrinsic pex feature
2 participants