Skip to content

Commit

Permalink
Dev (#74)
Browse files Browse the repository at this point in the history
* add deploy workflow

* fix workflows location

* fix ec2 directory path

* add workflow debug logs

* change env. to vars.

* change all env. to vars.

* add appspec

* update workflow to use codedeploy

* remove whitespace in deploy workflow

* update after_install script

* set correct file permissions in after_install.sh

* add timetstamp to after_install logs

* add log to after_install

* fix permissions settings in after_install

* add elastic IP to allowed hosts

* remove permissions logging from after_install

* update nginx group permissions

* add poetry install to after_install.sh

* remove old code from after_install

* update donation admin to fix n+1 query issue

* use tz aware datetime

* specify start block for debugging

* trigger redeployment

* fix settings.py for local celery

* refactor batch function, edit model

* correct start block

* feat: make donation entries update_or_create

* update start block for debugging

* debug logs for after_install migration check

* migration check debugging

* load env vars in after_install

* remove old code

* display donation id on admin

* fix donation id on admin dashboard

* fix typo in after_install

* fix TokenHistoricalPrice price_usd null issue

* add more debug logs

* change debug to info

* fix typo

* use logger.debug

* revert to logger.info

* fix coingecko api error handling

* add error test

* remove ERROR test

* save correct timestamp on TokenHistoricalPrice record

* get start block from cache

* fix None to decimal conversion bug

* add action_result & type unique_together constraint on Activity

* create or update activity even if donation already exists

* await aupdate_or_create Activity call

* remove manual start block setting

* restart indexer at 105_159_364

* restart indexer at 105,363,093

* restart indexer at 105534694

* restart indexer from current block height

* add celery recurring task for account totals

* update after_install to include celery beat service

* use logger in update_account_statistics

* add jobs logger

* debugging account stats task

* fix typo in account stats job

* remove transaction.atomic in account stats task

* update after_install and celery scripts to handle beat start/stop

* remove excess logs for update_account_statistics job

* add celery-beat to restart in after_install

* remove celery-beat from after_install

* move fetch_usd_prices to Donation model method

* add fetch_usd_prices to celery beat schedule

* add celery-beat stop/restart to after_install

* make fetch_usd_prices task synchronous

* use sync db methods in Donation model methods

* allow None chef_fee on Donation

* add migration to set chef_fee to None for existing donations

* fix int None issue in fetch_usd_prices

* return decimal from Token.format_price

* remove redundant log from fetch_usd_prices

* fix decimal * flot issue & restart indexer from block 105_854_538

* update usd prices job to check for no protocol_fee_usd value

* remove manual start block setting

* fix donations query in fetch_usd_prices

* Backfill compatible (#23)

* feat: allow backfill by update_create,index admin challenge response

* fix verbose name typo

* remove unused imports

---------

Co-authored-by: Lachlan Glen <[email protected]>

* restart at block 106388588

* restart at block 106388588

* restart indexer at 106431915

* restart at 106435734

* add pattern matching for pot subaccount

* add cors headers & localhost support

* fix virtual env usage in after_install.sh

* update after_install.sh

* revert after_install

* update start block to 110046700

* avoid logging WorkerLostErrors to Sentry

* update start block to 111941200

* update start block to 112034320

* update start block to 112158850

* update start block to 112461673

* fix display of admins & wl deployers in PotFactory detail (admin)

* update verbose name plurals

* add Pot custom form

* add admin for PotApplicationReview

* comment out worker_shutdown_handler

* fix tuple error when creating pot, update start block to 112961757 to reindex

* update start block to 112920268

* update start block to 113030298

* update start block to 113042335

* update start block to 113058915

* update start block to 113066329

* update start block to 113155075

* update start block to 113284717

* update start block to 113455493

* update start block to 113613466

* update start block to 113614770

* update start block to 114027103

* update start block to 114037338

* update start block to 114056625

* handle None pot application & update start block

* use coingecko pro api

* choose LakeConfig network based on PL_ENVIRONMENT

* Testnet -> Dev (#26)

* add deploy-testnet.yml

* add testnet deployment workflow & appspec

* testnet workflow debugging

* upload appspec as artifact

* upload zipped codebase to s3 in workflow

* continue working on after_install_testnet

* separate watchtower log groups by django env

* add testnet potlock TLA to indexer handler

* add POTLOCK_TLA to settings

* add dev.potlock.io and test-dev.potlock.io to allowed hosts

* remove admin dashboard edit access

* remove admin edit permission for all models

* add timestamp to deployment zip filename

* add env vars to deploy-testnet

* add after_install_dev script

* add debug logs

* add logs

* fix appspec destination for dev

* clean up destination directory before installing in deployment (only dev)

* add clean_destination scripts

* remove BeforeInstall clean destination step

* Testnet (#27)

* add deploy-testnet.yml

* add testnet deployment workflow & appspec

* testnet workflow debugging

* upload appspec as artifact

* upload zipped codebase to s3 in workflow

* continue working on after_install_testnet

* separate watchtower log groups by django env

* add testnet potlock TLA to indexer handler

* add POTLOCK_TLA to settings

* add dev.potlock.io and test-dev.potlock.io to allowed hosts

* remove admin dashboard edit access

* remove admin edit permission for all models

* add timestamp to deployment zip filename

* add env vars to deploy-testnet

* add after_install_dev script

* add after_install debug logs

* handle testnet potfactory & pot contract patterns

* test throttle rate

* update throttling to 100/minute

* Testnet (#28)

* add deploy-testnet.yml

* add testnet deployment workflow & appspec

* testnet workflow debugging

* upload appspec as artifact

* upload zipped codebase to s3 in workflow

* continue working on after_install_testnet

* separate watchtower log groups by django env

* add testnet potlock TLA to indexer handler

* add POTLOCK_TLA to settings

* add dev.potlock.io and test-dev.potlock.io to allowed hosts

* remove admin dashboard edit access

* remove admin edit permission for all models

* add timestamp to deployment zip filename

* add env vars to deploy-testnet

* add after_install_dev script

* add after_install debug logs

* handle testnet potfactory & pot contract patterns

* test throttle rate

* update throttling to 100/minute

* update readme with API urls

* handle None new pot owner

* fix owner_id in new pot

* fix keyerrors in pots admin

* add swagger (drf-spectacular)

* add api response examples

* add social profile data indexing (#31)

* add social profile data indexing

* remove unused imports

* Testnet (#30)

* add deploy-testnet.yml

* add testnet deployment workflow & appspec

* testnet workflow debugging

* upload appspec as artifact

* upload zipped codebase to s3 in workflow

* continue working on after_install_testnet

* separate watchtower log groups by django env

* add testnet potlock TLA to indexer handler

* add POTLOCK_TLA to settings

* add dev.potlock.io and test-dev.potlock.io to allowed hosts

* remove admin dashboard edit access

* remove admin edit permission for all models

* add timestamp to deployment zip filename

* add env vars to deploy-testnet

* add after_install_dev script

* add after_install debug logs

* handle testnet potfactory & pot contract patterns

* test throttle rate

* update throttling to 100/minute

* update readme with API urls

* add poetry lock --no-update to after_install_testnet

* add poetry lock --no-update to after_install_dev

* add near_social_profile_data to accounts admin

* add fetchsocialprofiledata command

* add fetch_near_social_profile_data_async

* update SIMPLE_ACCOUNT_EXAMPLE in OpenAPI schema to include near_social_profile_data

* fix pot pattern matching

* fix fastnear rpc url

* fix async error in handle_social_profile_update

* allow write admin actions for Accounts & Activities

* add autocomplete field for applicant

* add application applicant id to admin

* add application autocomplete field to PotApplicationReviewAdmin

* update PotApplicationAdmin

* update PotApplication __str__

* fix bad queries in admin

* add reviewer autocomplete

* add autocomplete for activities admin

* add pot__id to Donation admin list filter

* switch order of handle_protocol_fee_callback and donate handlers

* add debug logs for pot donations

* remove use of deprecated create_defaults in new pot donation

* rework donations handling

* fix unsupported operand type error

* fix operand issue again

* update donation handler

* add near social profile data schema

* fix Donation aupdate_or_create query to include pot

* add populatedata script

* add donations_received & donations_sent endpoints

* fix caching

* update readme

* populate lists data

* add pot_applications endpoint for accounts

* fix readme typo

* add related pot record to PotApplicationSerializer

* add cache back in

* add pagination to openAPI schema

* fix api schema pagination

* add payouts for acct endpoint; change ft FK to token

* fix remaining references to donation.ft

* add related records to API responses

* add update_pot_statistics task

* fetch usd prices for payouts

* add amount_paid_usd to payout admin

* add donate_contract_config endpoint

* fix paginated response schemas

* fix related records in OpenAPI schema

* add random_registration endpoint

* fix price_data reference error

* add debug log for payout processing paid_at: None issue

* fix project_id error in new pot application

* add debug logs for new list registrations

* attempt to resolve None error on new Donation

* fix error Failed to create/update donation: 'pot'

* update populatedata command

* add break to null donation result handling

* add spotindexer

* add alpha.potlock.io to allowed hosts

* add alpha.potlock.io to cors allowed origins

* Increase rate limit (#33)

* filter registration by plCategories (#35)

* extend schema (#36)

* add other tlds to cors whitelist (#38)

* add dev.local to cors allowed origins

* fix payout-setting error

* make PotPayout.paid_at nullable

* fix sync db calls

* add awaits to async db methods

* remove debug logs

* add https://dev.local to cors allowed origins

* Paginate by page (#39)

* gunicorn graceful reload

* fix after install script

* add graceful gunicorn restart to testnet after install script

* avoid stopping services before migrations in after_install scripts

* Pagination limit and readme update (#40)

* update readme with pagination query param

* customize pagination

* add https://dev.local back to cors allowed origins

* handle pot config update (#37)

* handle pot config update

* review changes: pot cupdate_or-create

* ensure chef exists, add admins

* New nadabot index (#34)

* feat: index nadabot registry, provider and stamps

* chore: add nadabot admin

* chore: review fixes: print to log, model fixes

* filter registration by plCategories (#35)

* extend schema (#36)

* add other tlds to cors whitelist (#38)

* test groups and rules

* add dev.local to cors allowed origins

* fix payout-setting error

* make PotPayout.paid_at nullable

* fix sync db calls

* add awaits to async db methods

* remove debug logs

* add https://dev.local to cors allowed origins

* Paginate by page (#39)

* gunicorn graceful reload

* fix after install script

* add graceful gunicorn restart to testnet after install script

* avoid stopping services before migrations in after_install scripts

* Pagination limit and readme update (#40)

* update readme with pagination query param

* customize pagination

* add blacklist entry model

* add https://dev.local back to cors allowed origins

* handle pot config update (#37)

* handle pot config update

* review changes: pot cupdate_or-create

* ensure chef exists, add admins

* review changes to model

* feat: index nadabot registry, provider and stamps

* chore: add nadabot admin

* chore: review fixes: print to log, model fixes

* test groups and rules

* add blacklist entry model

* review changes to model

* fix conflict issues

* fix provider onchain_id reference

* uncomment start block

* add comment

* fix group indexing and test

---------

Co-authored-by: Lachlan Glen <[email protected]>

* add chains app & Chain model (#41)

* default to NEAR chain when creating new account

* add debug logging to handler.py for memory usage & timing

* add additional debug logs for streamer processing time

* remove BlockHeight db write in cache_block_height

* set cache key in asyncio task

* remove excess debug logs

* add BlockHeight record update back

* add debug log

* create util function for common price fetching (#43)

* create util function for common price fetching

* resolve reviews

* add chains app & Chain model (#41)

* default to NEAR chain when creating new account

* add debug logging to handler.py for memory usage & timing

* add additional debug logs for streamer processing time

* remove BlockHeight db write in cache_block_height

* set cache key in asyncio task

* remove excess debug logs

* add BlockHeight record update back

* add debug log

* Revert "Merge branch 'dev' into common-price-fetch"

This reverts commit 54d526ece477b0a49957c5f085de7090f2d05542, reversing
changes made to 8b6c363.

* small fixes

---------

Co-authored-by: Lachlan Glen <[email protected]>

* handle zero net amount and migration script (#45)

* handle zero net amount and migration script

* remove unused import

* small net_amount updates

---------

Co-authored-by: Lachlan Glen <[email protected]>

* add coingecko_id to token admin

* make coingecko_id nullable

* add blank=True to nullable Token fields for admin

* fix payouts paid_at not being set

* save pot.all_paid_out

* change account to id (#51)

* change account to id

* add donate contract

* Whitelist deployer index (#54)

* index admin add whitelist deployer functin

* fix payout pot id

* api endpoint for pot factories (#55)

* api endpoint for pot factories

* fix response serializer

* get cooldown end while setting oayout (#58)

* use default PageNumberPagination (#59)

* factory update configs (#60)

* method for factory update config

* index config update functions

* use pot config update method for indexing

* clear deployers before adding

* remove caching of blockheight (just save in db) (#61)

* add asyncio handling of blockheight saving (#62)

* remove caching of blockheight (just save in db)

* add asyncio task scheudling back in for saving blockheight

* add block_timestamp to BlockHeightAdmin

* add balnk field to nullable fields (#63)

* customize pagination size (#65)

* customize pagination size

* remove, max page size

* Update readme

* inherit custom pagination for all endpoint

* use id to assign response admin (#66)

* use id to assign response admin

* fix admin payout search

* pot config update (#64)

* pot config update

* use model methid for pot config update

* get all list registrations for an account (#68)

* get all list registrations for an account

* update readme

* remove category filter

* Fix payout challenge indexing (#67)

* use select_related for queries

* remove comment on cache

* add txhah to challenge model

* fix cooldown update in pot method (#69)

* replace unserializable timestamp (#72)

---------

Co-authored-by: Prometheus <[email protected]>
  • Loading branch information
lachlanglen and Prometheo authored Jul 31, 2024
1 parent 738b618 commit 2297ed5
Show file tree
Hide file tree
Showing 103 changed files with 6,460 additions and 737 deletions.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ media/

# Celery beat schedule file
celerybeat-schedule
dump.rdb

# Flask stuff:
instance/
Expand Down Expand Up @@ -99,7 +100,7 @@ poetry.lock
# dotenv
.env
.env.*

pyrightconfig.json
# virtualenv
.venv/
venv/
Expand Down Expand Up @@ -133,4 +134,4 @@ dmypy.json

# static

/static/
/static/
49 changes: 43 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,26 @@
- [Steps to run:](#steps-to-run)
- [Env vars example](#env-vars-example)
- [API Basics](#api-basics)
- [Base URL](#base-url)
- [Authorization](#authorization)
- [Error Responses](#error-responses)
- [Pagination](#pagination)
- [Base URL](#base-url)
- [Authorization](#authorization)
- [Error Responses](#error-responses)
- [Pagination](#pagination)
- [API Endpoints](#api-endpoints)
- [`Account` endpoints](#account-endpoints)
- [✅ Get all accounts: `GET /accounts` (paginated)](#-get-all-accounts-get-accounts-paginated)
- [✅ Get account by ID (address): `GET /accounts/{ACCOUNT_ID}`](#-get-account-by-id-address-get-accountsaccount_id)
- [✅ Get donations received for account: `GET /accounts/{ACCOUNT_ID}/donations_received` (paginated)](#-get-donations-received-for-account-get-accountsaccount_iddonations_received-paginated)
- [✅ Get donations sent for account: `GET /accounts/{ACCOUNT_ID}/donations_sent` (paginated)](#-get-donations-sent-for-account-get-accountsaccount_iddonations_sent-paginated)
- [✅ Get pots for account: `GET /accounts/{ACCOUNT_ID}/active_pots` (paginated)](#-get-pots-for-account-get-accountsaccount_idactive_pots-paginated)
- [✅ Get applications for account: `GET /accounts/{ACCOUNT_ID}/pot_applications` (paginated)](#-get-applications-for-account-get-accountsaccount_idpot_applications-paginated)
- [✅ Get registrations to lists by account: `GET /accounts/{ACCOUNT_ID}/list-registrations` (paginated)](#-get-registrations-to-lists-by-account-get-accountsaccount_idlist-registrations-paginated)
- [`List` endpoints](#list-endpoints)
- [✅ Get all lists: `GET /lists` (paginated)](#-get-all-lists-get-lists-paginated)
- [✅ Get list by ID: `GET /lists/{LIST_ID}` (paginated)](#-get-list-by-id-get-listslist_id-paginated)
- [✅ Get registrations for list: `GET /lists/{LIST_ID}/registrations` (paginated)](#-get-registrations-for-list-get-listslist_idregistrations-paginated)
- [✅ Get random registration for list: `GET /lists/{LIST_ID}/random_registration`](#-get-random-registration-for-list-get-listslist_idrandom_registration)
- [Donate Contract Config endpoint](#donate-contract-config-endpoint)
- [✅ Get donate contract config: `GET /donate_contract_config`](#-get-donate-contract-config-get-donate_contract_config)
- [`Donors` endpoints](#donors-endpoints)
- [✅ Get all donors: `GET /donors` (paginated)](#-get-all-donors-get-donors-paginated)
- [`Pots` endpoints](#pots-endpoints)
Expand Down Expand Up @@ -85,7 +92,7 @@ export PL_SENTRY_DSN=

This is a public, read-only API and as such does not currently implement authentication or authorization.

Rate limits of 100 requests/min are enforced to ensure service for all users.
Rate limits of 500 requests/min are enforced to ensure service for all users.

#### Error Responses

Expand All @@ -102,7 +109,7 @@ Possible Error Codes:

#### Pagination

Pagination available using `limit` and `offset` query params on endpoints that specify `paginated`. Default `limit` is 30.
Pagination available using `page` and `page_size` as query param on endpoints that specify `paginated`. Default `page_size` is 30.

Endpoints that support pagination will return a success response containing the following:

Expand All @@ -121,10 +128,31 @@ _NB: These endpoints are what is required to integrate with BOS app & replace cu

#### ✅ Get account by ID (address): `GET /accounts/{ACCOUNT_ID}`

#### ✅ Get donations received for account: `GET /accounts/{ACCOUNT_ID}/donations_received` (paginated)

#### ✅ Get donations sent for account: `GET /accounts/{ACCOUNT_ID}/donations_sent` (paginated)

#### ✅ Get pots for account: `GET /accounts/{ACCOUNT_ID}/active_pots` (paginated)

Can specify `status=live` query param to retrieve only pots that are currently active (live matching round)

#### ✅ Get applications for account: `GET /accounts/{ACCOUNT_ID}/pot_applications` (paginated)

Can specify `status={PotApplicationStatus}` query param to retrieve applications with a given status:

```py
enum PotApplicationStatus {
Pending,
Approved,
Rejected,
InReview,
}
```

#### ✅ Get registrations to lists by account: `GET /accounts/{ACCOUNT_ID}/list-registrations` (paginated)

Can specify status to filter by using `status` query param if desired, e.g. `status=Approved`

### `List` endpoints

#### ✅ Get all lists: `GET /lists` (paginated)
Expand All @@ -134,6 +162,15 @@ Can specify `status=live` query param to retrieve only pots that are currently a
#### ✅ Get registrations for list: `GET /lists/{LIST_ID}/registrations` (paginated)

Can specify status to filter by using `status` query param if desired, e.g. `status=Approved`
Can also specify project category to filter by using `category` query param if desired, e.g. `category=Education`

#### ✅ Get random registration for list: `GET /lists/{LIST_ID}/random_registration`

Can specify status to filter by using `status` query param if desired, e.g. `status=Approved`

### Donate Contract Config endpoint

#### ✅ Get donate contract config: `GET /donate_contract_config`

### `Donors` endpoints

Expand Down
13 changes: 7 additions & 6 deletions accounts/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ class AccountAdmin(admin.ModelAdmin):
"total_donations_out_usd",
"total_matching_pool_allocations_usd",
"donors_count",
"near_social_profile_data",
)
search_fields = ("id",) # Allow searching by account address
list_filter = (
Expand Down Expand Up @@ -42,11 +43,11 @@ def total_matching_pool_allocations_usd_display(self, obj):
"Total Matching Pool Allocations (USD)"
)

def has_add_permission(self, request):
return False
# def has_add_permission(self, request):
# return False

def has_change_permission(self, request, obj=None):
return False
# def has_change_permission(self, request, obj=None):
# return False

def has_delete_permission(self, request, obj=None):
return False
# def has_delete_permission(self, request, obj=None):
# return False
Loading

0 comments on commit 2297ed5

Please sign in to comment.