Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[do not merge] Demonstrate registering committed models via GitHub Actions #71

Draft
wants to merge 36 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7e69137
Start on RMV registration workflow
liuverta Apr 26, 2023
4699e35
Add env vars
liuverta Apr 26, 2023
c0182c3
Add script
liuverta Apr 26, 2023
dfb86ec
Trigger on all pushes for testing
liuverta Apr 26, 2023
ad826c9
Set models/ as working directory
liuverta Apr 27, 2023
5a4e71d
Add script to create model files
liuverta Apr 27, 2023
8e70046
Add requirements.txt
liuverta Apr 27, 2023
b22fb58
Use secret dev key
liuverta Apr 27, 2023
cb1e6cc
Only trigger on model files
liuverta Apr 27, 2023
4ff7bd8
Register new models
liuverta Apr 27, 2023
d6efd95
Log model filenames
liuverta Apr 27, 2023
5abb4f5
Add first model
liuverta Apr 27, 2023
44c84c4
Try removing leading ./
liuverta Apr 27, 2023
b7a1487
Remove paths filter
liuverta Apr 27, 2023
4fa57ef
Revert "Remove paths filter"
liuverta Apr 27, 2023
1746585
Export MODEL_FILENAMES
liuverta Apr 27, 2023
0d1078e
Use model filename as RMV name
liuverta Apr 27, 2023
65327d1
Import cloudpickle
liuverta Apr 27, 2023
f770406
Correctly unpickle model class
liuverta Apr 27, 2023
4d73ea5
Don't bother with the RMV name
liuverta Apr 27, 2023
a2e3d5b
Correctly set environment version
liuverta Apr 27, 2023
e79baea
Log INFO
liuverta Apr 27, 2023
39964ce
Clear out existing models
liuverta Apr 27, 2023
097733c
Print instead of logging
liuverta Apr 27, 2023
738704a
Include renamed files
liuverta Apr 27, 2023
06f0ae6
Add model
liuverta Apr 27, 2023
f8cd314
Add model
liuverta Apr 27, 2023
57d7c36
Add model
liuverta Apr 27, 2023
bfc00ae
Add 2 models
liuverta Apr 27, 2023
38b5d7a
Remove leading ./
liuverta May 1, 2023
d3a8827
Add model
liuverta May 1, 2023
9894c52
Add docstrings
liuverta May 1, 2023
8691eb0
Add model
liuverta May 1, 2023
e36b1d8
Reorganize subdirs
liuverta May 1, 2023
b529c58
Use filename as RMV name
liuverta May 1, 2023
bb063ad
Add model
liuverta May 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/register_verta_model.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Register Verta Model Version

on:
push:
paths:
- models/*.pkl
branches:
- workflow1

jobs:
Register-Verta-Model-Version:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./models

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0 # fetch full history, in case push is multiple commits

- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: "3.10"
cache: "pip"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

- name: Register model version
env:
VERTA_HOST: dev.verta.ai
VERTA_EMAIL: [email protected]
VERTA_DEV_KEY: ${{ secrets.TEST_ACCOUNT_DEV_KEY }}
REGISTERED_MODEL_NAME: From GitHub Action
run: |
export MODEL_FILENAMES=$(git diff --diff-filter=AR --relative --name-only ${{ github.event.before }} ${{ github.event.after }} | grep '\.pkl$')
python register_model_version.py
Binary file added models/2023-04-27T15:42:56.315299.pkl
Binary file not shown.
Binary file added models/2023-04-27T15:42:57.456224.pkl
Binary file not shown.
Binary file added models/2023-04-27T15:42:59.015792.pkl
Binary file not shown.
Binary file added models/2023-04-27T15:43:15.173062.pkl
Binary file not shown.
Binary file added models/2023-04-27T15:43:16.495668.pkl
Binary file not shown.
30 changes: 30 additions & 0 deletions models/register_model_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# -*- coding: utf-8 -*-

import os

import cloudpickle
from verta import Client
from verta.environment import Python


REGISTERED_MODEL_NAME = os.environ["REGISTERED_MODEL_NAME"]
MODEL_FILENAMES = os.environ["MODEL_FILENAMES"]


if __name__ == "__main__":
client = Client()
reg_model = client.get_or_create_registered_model(REGISTERED_MODEL_NAME)

for model_filename in MODEL_FILENAMES.splitlines():
with open(model_filename, "rb") as f:
model_cls = cloudpickle.load(f)

requirements = Python.read_pip_file(
os.path.join(os.path.dirname(__file__), "requirements.txt"),
)

print(f'Registering model "{model_filename}"')
model_ver = reg_model.create_standard_model(
model_cls,
environment=Python(requirements),
)
1 change: 1 addition & 0 deletions models/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
verta==0.22.2
23 changes: 23 additions & 0 deletions models/save_model_file.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# -*- coding: utf-8 -*-

from datetime import datetime
import os

import cloudpickle
from verta.registry import VertaModelBase, verify_io


class Model(VertaModelBase):
def __init__(self, artifacts=None):
pass

@verify_io
def predict(self, input):
return input


if __name__ == "__main__":
model_filename = f"{datetime.now().isoformat()}.pkl"
model_filepath = os.path.join(os.path.dirname(__file__), model_filename)
with open(model_filepath, "wb") as f:
cloudpickle.dump(Model, f)