This repository has been archived by the owner on Nov 13, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Docs] Minor documentation updates during walkthrough (#46)
Updated some documentation while doing an end-to-end walkthrough.
- Loading branch information
Showing
18 changed files
with
710 additions
and
245 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,19 @@ | ||
# Pocket RPC Configuration | ||
POKT_RPC_FULL_HOST= | ||
HTTP_SERVER_PORT=8080 | ||
POKT_RPC_TIMEOUT=5s | ||
|
||
# Gateway Deployment Configuration | ||
HTTP_SERVER_PORT=8080 | ||
ENVIRONMENT_STAGE=development | ||
POKT_APPLICATIONS_ENCRYPTION_KEY= | ||
EMIT_SERVICE_URL_PROM_METRICS=false | ||
|
||
# Pocket Business Logic | ||
CHAIN_NETWORK=morse_mainnet | ||
SESSION_CACHE_TTL=75m | ||
DB_CONNECTION_URL=postgres://myuser:mypassword@postgres:5432/postgres?sslmode=disable | ||
API_KEY= | ||
ALTRUIST_REQUEST_TIMEOUT=10s | ||
EMIT_SERVICE_URL_PROM_METRICS=false | ||
CHAIN_NETWORK=morse_mainnet | ||
API_KEY= | ||
|
||
# App Stake Management | ||
DB_CONNECTION_URL=postgres://myuser:mypassword@postgres:5433/postgres?sslmode=disable | ||
POKT_APPLICATIONS_ENCRYPTION_KEY= | ||
POKT_APPLICATION_PRIVATE_KEY= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -53,4 +53,10 @@ | |
docker-compose.yml | ||
|
||
## MAC | ||
.DS_Store | ||
.DS_Store | ||
|
||
## VSCode | ||
.vscode | ||
|
||
## Copmiled binary | ||
main |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
include .env | ||
|
||
######################## | ||
### Makefile Helpers ### | ||
######################## | ||
|
||
.PHONY: prompt_user | ||
# Internal helper target - prompt the user before continuing | ||
prompt_user: | ||
@echo "Are you sure? [y/N] " && read ans && [ $${ans:-N} = y ] | ||
|
||
.PHONY: list | ||
list: ## List all make targets | ||
@${MAKE} -pRrn : -f $(MAKEFILE_LIST) 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | egrep -v -e '^[^[:alnum:]]' -e '^$@$$' | sort | ||
|
||
.PHONY: help | ||
.DEFAULT_GOAL := help | ||
help: ## Prints all the targets in all the Makefiles | ||
@grep -h -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-60s\033[0m %s\n", $$1, $$2}' | ||
|
||
######################## | ||
### Database Helpers ### | ||
######################## | ||
|
||
.PHONY: db_migrate | ||
db_migrate: ## Run database migrations | ||
@echo "Running database migrations..." | ||
./scripts/db_migrate.sh -u | ||
|
||
.PHONY: db_migrate | ||
db_migrate: ## Run database migrations | ||
@echo "Running database migrations..." | ||
./scripts/db_migrate.sh -u\ | ||
|
||
|
||
PG_CMD := INSERT INTO pokt_applications (encrypted_private_key) VALUES (pgp_sym_encrypt('$(POKT_APPLICATION_PRIVATE_KEY)', '$(POKT_APPLICATIONS_ENCRYPTION_KEY)')); | ||
db_insert_app_private_key: ## Insert application private key into database | ||
@echo "Running SQL command..." | ||
@echo "$(PG_CMD)" | ||
@psql "$(DB_CONNECTION_URL)" -c "$(PG_CMD)" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,114 +2,180 @@ | |
<img src="https://github.com/pokt-network/gateway-server/blob/main/docs/resources/gateway-server-logo.jpg" width="500" alt="POKT Gateway Server"> | ||
</div> | ||
|
||
## What is POKT Gateway Server? | ||
# What is POKT Gateway Server? <!-- omit in toc --> | ||
|
||
The POKT Gateway Server is a comprehensive solution designed to simplify the integration of applications with the POKT Network. Our goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators. | ||
_tl;dr Streamline access to POKT Network's decentralized supply network._ | ||
|
||
Learn more about the vision and overall architecture [overview](docs%2Foverview.md) | ||
The POKT Gateway Server is a comprehensive solution designed to simplify the integration of applications with POKT Network. Its goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators. | ||
|
||
## Gateway Operator Quick Getting Started | ||
To onboard the gateway server without having to dig deep, you can follow the [Quick Onboarding Guide](docs%2Fquick-onboarding-guide.md) | ||
Learn more about the vision and overall architecture [overview](./docs/overview.md). | ||
|
||
#### Interested in learning more? We have an abdundance of information in [docs](docs) | ||
1. [Gateway Server Overview](docs%2Foverview.md) | ||
2. [Gateway Server API Endpoints](docs%2Fapi-endpoints.md) | ||
3. [Gateway Server System Architecture](docs%2Fsystem-architecture.md) | ||
4. [Gateway Server Node Selection](docs%2Fnode-selection.md) | ||
5. [POKT Primer](docs%2Fpokt-primer.md) | ||
6. [POKT's Relay Specification](docs%2Fpokt-relay-specification.md) | ||
- [Gateway Operator Quickstart Guide](#gateway-operator-quickstart-guide) | ||
- [Interested in learning more?](#interested-in-learning-more) | ||
- [Docker Image Releases](#docker-image-releases) | ||
- [Docker Compose](#docker-compose) | ||
- [Minimum Hardware Requirements](#minimum-hardware-requirements) | ||
- [Database Migrations](#database-migrations) | ||
- [Creating a DB Migration](#creating-a-db-migration) | ||
- [Applying a DB Migration](#applying-a-db-migration) | ||
- [DB Migration helpers](#db-migration-helpers) | ||
- [Applying Migrations](#applying-migrations) | ||
- [Migrations Rollbacks](#migrations-rollbacks) | ||
- [Unit Testing](#unit-testing) | ||
- [Generating Mocks](#generating-mocks) | ||
- [Running Tests](#running-tests) | ||
- [Generating DB Queries](#generating-db-queries) | ||
- [Contributing Guidelines](#contributing-guidelines) | ||
- [Project Structure](#project-structure) | ||
|
||
## Gateway Operator Quickstart Guide | ||
|
||
To onboard the gateway server without having to dig deep, you can follow the [Quick Onboarding Guide](docs/quick-onboarding-guide.md). | ||
|
||
### Interested in learning more? | ||
|
||
We have an abundance of information in the [docs](docs) section: | ||
|
||
1. [Gateway Server Overview](docs/overview.md) | ||
2. [Gateway Server API Endpoints](docs/api-endpoints.md) | ||
3. [Gateway Server System Architecture](docs/system-architecture.md) | ||
4. [Gateway Server Node Selection](docs/node-selection.md) | ||
5. [POKT Primer](docs/pokt-primer.md) | ||
6. [POKT's Relay Specification](docs/pokt-relay-specification.md) | ||
|
||
## Docker Image Releases | ||
Every release candidate is published to https://github.com/pokt-network/gateway-server/pkgs/container/pocket-gateway-server | ||
|
||
Every release candidate is published to [gateway-server/pkgs/container/pocket-gateway-server](https://github.com/pokt-network/gateway-server/pkgs/container/pocket-gateway-server). | ||
|
||
## Docker Compose | ||
|
||
There is an all-inclusive docker-compose file available for development [docker-compose.yml](docker-compose.yml.sample) | ||
|
||
## Minimum Hardware Requirements to run | ||
## Minimum Hardware Requirements | ||
|
||
To run a Gateway Server, we recommend the following minimum hardware requirements: | ||
|
||
- 1GB of RAM | ||
- 1GB of storage | ||
- 4 vCPUs+ | ||
|
||
In production, we have observed memory usage increase to 4GB+. The memory footprint will be dependent on the number of app stakes/chains staked and total traffic throughput. | ||
|
||
## Creating a DB Migration | ||
## Database Migrations | ||
|
||
<!-- TODO_IMPROVE: Need more details on why we need a DB, how it's used, pre-requisites, etc... --> | ||
|
||
### Creating a DB Migration | ||
|
||
Migrations are like version control for your database, allowing your team to define and share the application's database schema definition. | ||
Before running a migration make sure to install the go lang migration cli on your machine. | ||
https://github.com/golang-migrate/migrate/tree/master/cmd/migrate | ||
|
||
Before running a migration make sure to install the go lang migration cli on your machine. See [golang-migrate/migrate/tree/master/cmd/migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate) for reference. | ||
|
||
```sh | ||
./scripts/migration.sh -n {migration_name} | ||
``` | ||
|
||
This command will generate a up and down migration in `db_migrations` | ||
|
||
## Applying a DB Migration | ||
### Applying a DB Migration | ||
|
||
DB Migrations are applied upon server start, but as well, it can be applied manually through: | ||
|
||
```sh | ||
./scripts/migration.sh {--down or --up} {number_of_times} | ||
./scripts/migration.sh {--down or --up} {number_of_times} | ||
``` | ||
|
||
### Usage | ||
### DB Migration helpers | ||
|
||
#### Apply Migrations | ||
#### Applying Migrations | ||
|
||
- To apply all migrations: | ||
|
||
```sh | ||
./scripts/migration.sh --up | ||
``` | ||
|
||
- To apply a specific number of migrations: | ||
|
||
```sh | ||
./scripts/migration.sh --up 2 | ||
``` | ||
|
||
#### Rollback Migrations | ||
#### Migrations Rollbacks | ||
|
||
Make sure to provide either the number of migrations to rollback or the `--all` flag to rollback all migrations. | ||
|
||
- To roll back a specific number of migrations: | ||
|
||
```sh | ||
./scripts/migration.sh --down 2 | ||
``` | ||
|
||
- To roll back all migrations: | ||
|
||
```sh | ||
./scripts/migration.sh --down --all | ||
``` | ||
|
||
## Running Tests | ||
## Unit Testing | ||
|
||
### Generating Mocks | ||
|
||
Install Mockery with | ||
``` | ||
|
||
```bash | ||
go install github.com/vektra/mockery/[email protected] | ||
``` | ||
|
||
You can generate the mock files through: | ||
|
||
```sh | ||
./scripts/mockgen.sh | ||
``` | ||
|
||
By running this command, it will generate the mock files in `./mocks` folder. | ||
Reference for mocks can be found here https://vektra.github.io/mockery/latest/ | ||
|
||
Reference for mocks can be found [here](https://vektra.github.io/mockery/latest). | ||
|
||
### Running Tests | ||
|
||
Run this command to run tests: | ||
|
||
```sh | ||
go test ./... | ||
go test -v -count=1 ./... | ||
``` | ||
|
||
## Generating DB Queries | ||
Gateway server uses [PGGen]('https://github.com/jschaf/pggen) to create autogenerated type-safe queries. | ||
Queries are added inside [queries.sql](internal%2Fdb_query%2Fqueries.sql) and then regenerated via `./scripts/querygen.sh`. | ||
|
||
Gateway server uses [PGGen](https://github.com/jschaf/pggen) to create autogenerated type-safe queries. | ||
Queries are added inside [queries.sql](./internal/Fdb_query/queries.sql) and re-generated via `./scripts/querygen.sh`. | ||
|
||
## Contributing Guidelines | ||
|
||
1. Create a Github Issue on the feature/issue you're working on. | ||
2. Fork the project | ||
3. Create new branch with `git checkout -b "branch_name"` where branch name describes the feature. | ||
- All branches should be based off `main` | ||
3. Write your code | ||
4. Make sure your code lints with `go fmt ./...` (This will Lint and Prettify) | ||
5. Commit code to your branch and issue a pull request and wait for at least one review. | ||
- Always ensure changes are rebased on top of main branch. | ||
- All branches should be based off `main` | ||
4. Write your code | ||
5. Make sure your code lints with `go fmt ./...` (This will Lint and Prettify) | ||
6. Commit code to your branch and issue a pull request and wait for at least one review. | ||
- Always ensure changes are rebased on top of main branch. | ||
--- | ||
## Project Structure | ||
- **cmd:** Contains the entry point of the binaries | ||
- **gateway_server:** HTTP Server for serving requests | ||
- **internal:** Shared internal folder for all binaries | ||
- **pkg:** Distributable dependencies | ||
- **docs:** Project documentation and specifications | ||
A partial high-level view of the code structure (generated) | ||
```bash | ||
. | ||
├── cmd # Contains the entry point of the binaries | ||
│ └── gateway_server # HTTP Server for serving requests | ||
├── internal # Shared internal folder for all binaries | ||
├── pkg # Distributable dependencies | ||
└── scripts # Contains scripts for development | ||
``` | ||
_Generate via `tree -L 2`_ | ||
--- | ||
Olshansky's stream of thought here: https://www.notion.so/buildwithgrove/Olshansky-PATH-Journal-dfc92813ca5945ac88035cca97ccbff5?pvs=4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,40 @@ | ||
# Altruist (Failover) Request | ||
# Chain & Altruist Configurations <!-- omit in toc --> | ||
|
||
## Altruist (Failover) Request | ||
|
||
- [Altruist (Failover) Request](#altruist-failover-request) | ||
- [Chain Configuration](#chain-configuration) | ||
- [Inserting a custom chain configuration](#inserting-a-custom-chain-configuration) | ||
|
||
In rare situations, a relay cannot be served from POKT Network. Some sample scenarios for when this can happen: | ||
|
||
1. Dispatcher Outages - Gateway server cannot retrieve the necessary information to send a relay | ||
2. Bad overall QoS - Majority of Node operators may have their nodes misconfigured improperly or there is a lack of node operators supporting the chain with respect to load. | ||
3. Chain halts - In extreme conditions, if the chain halts, node operators may stop responding to relay requests | ||
|
||
So the Gateway Server will attempt to route the traffic to a backup chain node. This could be any source, for example: | ||
|
||
1. Other gateway operators chain urls | ||
2. Centralized Nodes | ||
|
||
# Chain Configuration | ||
## Chain Configuration | ||
|
||
Given that every chain has differences and have different sources for fail over, the gateway server allows for optional customization for request timeouts, failover relay, and QoS checks. | ||
The data is stored inside the `chain_configuration` table and is accessed via the [chain_configurations_registry_service.go](..%2Finternal%2Fchain_configurations_registry%2Fchain_configurations_registry_service.go). | ||
The data is stored inside the `chain_configuration` table and is accessed via the [chain_configurations_registry_service.go](../internal/chain_configurations_registry/chain_configurations_registry_service.go). | ||
|
||
_While it is **recommended** that you provide a chain configuration, the gateway server will assume defaults provided from the specified [config_provider.go](../internal/global_config/config_provider.go) and the provided QoS [checks](../internal/node_selector_service/checks)_ if not provided. | ||
|
||
_While it is **recommended** that you provide a chain configuration, the gateway server will assume defaults provided from the specified [config_provider.go](..%2Finternal%2Fglobal_config%2Fconfig_provider.go) and the provided QoS [checks](..%2Finternal%2Fnode_selector_service%2Fchecks)_ if not provided. | ||
## Inserting a custom chain configuration | ||
|
||
# Inserting a custom chain configuration | ||
```sql | ||
-- Insert an example configuration for Ethereum -- | ||
INSERT INTO chain_configurations (chain_id, pocket_request_timeout_duration, altruist_url, altruist_request_timeout_duration, top_bucket_p90latency_duration, height_check_block_tolerance, data_integrity_check_lookback_height) VALUES ('0000', '15s', 'https://example.com', '30s', '150ms', 100, 25); | ||
``` | ||
|
||
- `chain_id` - id of the Pocket Network Chain | ||
- `pocket_request_time` - duration of the maximum amount of time for a network relay to respond | ||
- `altruist_url` - source of the relay in the event that a network request fails | ||
- `altruist_url` - source of the relay in the event that a network request fails | ||
- `altruist_request_timeout_duration` - duration of the maximum amount of time for a backup request to respond | ||
- `top_bucket_p90latency_duration` - maximum amount of latency for nodes to be favored 0 <= x <= `top_bucket_p90latency_duration` | ||
- `top_bucket_p90latency_duration` - maximum amount of latency for nodes to be favored 0 <= x <= `top_bucket_p90latency_duration` | ||
- `height_check_block_tolerance` - number of blocks a node is allowed to be behind (some chains may have node operators moving faster than others) | ||
- `data_integrity_check_lookback_height` - number of blocks data integrity will look behind for source of truth block for other node operators to attest too |
Oops, something went wrong.