Package Management

Instructions for how to build, package and release a new version of BMTK package for Github, PyPI, Anaconda, etc.

Before making a new version make sure all required changes are in the develop branch, and it is passing tests and running the examples.

---

Quick Instructions

Manually

Replace {VER} with the appropiate version number, eg (1.0.7)

1. Bumping the Version

Update the major, minor, and/or patch number, located in the __version__ string in bmtk.__init__.py. This can be done as a PR or through the Github interface. Make sure your local repo is up-to-date with the develop branch.

2. Creating a Github release

• Create a release branch; in the main github page select the "branch" drop down > Select "Find or create a branch..." > create a new branch called release/v{VERSION}.

• In the Releases page select "Draft a new release"

  • Under "Choose a tag" enter new tag v{VER} and select "+Create a new tag"
  • For the "Target" choose the release branch created above.
  • For a title use v{VER}
  • Add description and change-log to descrption.
  • Click "Publish Release"

3. Create and Publish PyPI package:

• Pull the latest version of bmtk develop/release branch to your local machine.
• [optional] Remove bmtk bmtk.egg-info/ and build/ directory if they exists (otherwise code may be static)
• Create source-distribution (*.tar.gz) and wheels-distribution (*.whl) packages in the dist/ directory

    $ python setup.py bdist_wheel sdist

• [optional] test by installing package locally on a separate environments:

    $ conda activate test-env
    $ pip install dist/bmtk-{VER}-*[whl|tar.gz]
    $ pytest ...

• [optional] Check the wheel package

    $ check-wheel-contents dist/bmtk-*.whl

• You will need to set-up a $HOME/.pypirc file in order to upload to pypi with a token (or password if available)

[distutils]
index-server =
    pypi
    testpypi
    bmtk

[pypi]
repository = https://upload.pypi.org/legacy/
username = __token__
password = <TOKEN>

[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = <TOKEN>

[bmtk]
repository = https://upload.pypi.org/legacy
username = __token__
password = <TOKEN>

The <TOKEN> values are generated online: https://pypi.org/manage/account/, or get from project manager

• Upload to PyPI using twine:

    $ python -m twine upload dist/bmtk-{VER}-*

4. Create and Publish Anaconda Packages

• If you haven't already, install conda-build and anaconda-client onto a new or existing conda environment:

    $ conda create --name <bmtk-build-env> conda-build anaconda-client
    $ conda activate <bmtk-build-env>

• Pull the latest version of bmtk develop/release branch to your local machine.
• Go to .conda/ directory
• Build the packages (*.tar.bz2 files) for your local machine architecture (usually osx-64 or linux-64) using the conda-build recipes (in the recipes/ directory) into the conda-bld folder

    $ conda build --output-folder conda-bld recipes/

• Create packages for other architectures

    $ conda convert --platform all conda-bld/linux-64/bmtk-{VER}-py3*.tar.bz2 -o conda-bld/

• Use anaconda to upload package to anaconda cloud:

    $ anaconda login
    $ anaconda upload conda-bld/**/bmtk-{VER}*.bz2

Automatically CI/CD

TODO: Get CD working with Github actions

Issues, Requirements, Caveats, etc.

PyPI (pip) Packages

Wheels package At the moment wheels should be packaging the universal package (eg bmtk-{VER}-.py2.py3-none-any.whl file) rather than any python/architecture specific package. This will mean the same package is used for all pip install bmtk instructions. This is specified in the ```setup.cfg`` file

[bdist_wheel]
universal=1

Or can be manually done when running the bdist_wheel command:

 $ python setup.py bdist_wheel --universal

Reuploading to PyPI

⚠️ Once a package has been uploaded to PyPI there is no way of undoing it

As of October 2022, PyPI does not allow you to re-upload a the same version of a package multiple times. Even if you delete it from PyPI you won't be able to reupload a new copy. Thus if there is a problem with the PyPI build, you will need to bump the patch number (eg 1.0.12 --> 1.0.13) and reupload.

If you are unsure if a package will work, PyPI has a testing repository called test.pypi:

  $ python -m twine upload --repository testpypi dist/*
  $ pip install --index-url https://test.pypi.org/simple/ bmtk

Anaconda