Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add QueryAPI pages #1479

Merged
merged 13 commits into from
Sep 7, 2023
76 changes: 76 additions & 0 deletions docs/bos/queryapi/component.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
id: bos-components
title: Components handling Historical data
sidebar_label: Components & Historical data
---

Building components that handle historical blockchain data require dedicated solutions that manage the data and reduce the latency of requests, as it's not possible to scan the whole blockchain when a user makes a request.

A simple solution for developers building on NEAR BOS is using [QueryAPI](intro.md), a fully managed solution to build indexer functions, extract on-chain data, store it in a database, and be able to query it using GraphQL endpoints.

## QueryAPI

:::tip
Learn more about QueryAPI in this [QueryAPI Overview](intro.md) article.
:::

#### Tutorials

For a technical implementation deep-dive, check these QueryAPI tutorials:

- [Posts Indexer tutorial](../tutorial/indexer-tutorials/posts-indexer.md): this indexer creates a new row in a pre-defined database for every new BOS post found on the blockchain.
- [Hype Indexer tutorial](../tutorial/indexer-tutorials/hype-indexer.md): this indexer creates a new row in a pre-defined database for every new BOS post or comment found on the blockchain that contains either `PEPE` or `DOGE` in the contents.
- [BOS Feed Indexer tutorial](../tutorial/indexer-tutorials/feed-indexer.md): this indexer keeps track of new posts, comments, and likes on BOS, so a social feed can be rendered quickly.

## GraphQL queries

Using [QueryAPI's GraphiQL](index-function.md#mutations-in-graphql) tab, you can access the GraphiQL Explorer that provides a user friendly GraphQL playground, where you can view and create queries and mutations based on the DB schema that you defined for the indexer.

![QueryAPI Indexer Dashboard](/docs/assets/QAPIgraphiql.png)

You can easily set some fields and select the returning data
that you want, and the tool will build a query on the mutation panel on the right.
Then you can copy the resulting query, either in your JavaScript code so that you pass actual
data manually, or you pass in the mutation data object as a second parameter.

For example, if you go and add a new mutation, click <kbd>+</kbd>, then you can do a bunch of actions here, such as creating, deleting, or inserting posts into your table.

![Playground](/docs/assets/QAPIScreen.gif)

If you want to test your mutation, using [Debug Mode](index-function.md#local-debug-mode) you can add a specific
block to the list, and then play it to see how it works.
Based on the indexer logic you defined, you'll get a call to the GraphQL mutation with the object
and data passed into it.

:::tip Video Walkthrough

**Tip:** watch the video on how to [create mutations in GraphQL](https://www.youtube.com/watch?v=VwO6spk8D58&t=781s).

:::

## Generate a BOS component using Playground

Creating a BOS component from a GraphQL query is simple when using QueryAPI's GraphQL Playground. Just follow these steps:

- go to the GraphiQL tab
- select the query that you want to use
- click on the <kbd>Show GraphiQL Code Exporter</kbd> button
- get some default code here, copy it,
- go to the BOS sandbox, paste it.


This will set up some boilerplate code to execute the GraphQL query, add the query that you had
in your playground and then call that query, extract the data and render it using the
render data function.

Once you have the BOS component code, you can test it out by going to [the sandbox](https://near.org/sandbox),
pasting the generated code, and then selecting <kbd>Component Preview</kbd>.
Next, you can create a nice UI over this boilerplate code, and publish your new BOS component.

### Component Examples

- [Activity Feed widget](https://near.org/near/widget/ComponentDetailsPage?src=roshaan.near/widget/user-activity-feed&tab=source) running on [near.org](https://near.org)

<!--
- Example of BOS component using BigQuery
-->
45 changes: 45 additions & 0 deletions docs/bos/queryapi/how-works.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
id: how-it-works
title: How QueryAPI works
sidebar_label: How it works
---

QueryApi is a streaming indexer implementation that executes queries written by developers on the NEAR blockchain.
bucanero marked this conversation as resolved.
Show resolved Hide resolved
QueryApi allows hosted execution of complex queries (ones that can’t be answered by a [simple RPC](../../5.api/rpc/introduction.md) or [Enhanced API](https://docs.pagoda.co/api) call) and hosting of the data output of those queries using GraphQL endpoints.
bucanero marked this conversation as resolved.
Show resolved Hide resolved


## Components involved

The QueryApi implementation integrates many different components in a single and streamlined solution.
In a high-level overview, the main components are:

:::info Components
`NEAR Protocol` -> `NEAR Lake` -> `Coordinator` -> `Database` -> `API`
bucanero marked this conversation as resolved.
Show resolved Hide resolved
:::


### Detailed overview

An in-depth, detailed overview of the QueryApi components:

[![QueryAPI](/docs/qapi-components.png)](/docs/qapi-components.png)


### Description

- **Protocol:** the underlying NEAR Layer-1 Blockchain, where data `Blocks` and `Chunks` are produced.
- **NEAR Lake:** an indexer built on top of the [NEAR Lake Framework](../../1.concepts/3.advanced/near-lake-framework.md) to watch the Layer-1 network and store all the events as JSON files on AWS S3. Changes are indexed as new `Blocks` arrive.
bucanero marked this conversation as resolved.
Show resolved Hide resolved
- **Coordinator**: the QueryApi coordinator indexer filters matching data `Blocks`, runs historical processing threads, and executes developer's JS code.
bucanero marked this conversation as resolved.
Show resolved Hide resolved
- **Database**: a Postgres database where filtered and indexed results are stored, using a Logical DB per user, and a Logical schema per indexer function.
bucanero marked this conversation as resolved.
Show resolved Hide resolved
- **API**: a Hasura GraphQL server running on Google Cloud Platform exposes simple web API endpoints so users can access GraphQL queries and subscriptions from anywhere.
bucanero marked this conversation as resolved.
Show resolved Hide resolved

## Who hosts QueryAPI

[Pagoda Inc.](https://pagoda.co) runs and manages all the infrastructure of QueryAPI, including NEAR Lake nodes to store the data in JSON format on AWS S3.
bucanero marked this conversation as resolved.
Show resolved Hide resolved

- NEAR Lake indexing is hosted on AWS S3 buckets.
- Hasura GraphQL API server is hosted on GCP.
bucanero marked this conversation as resolved.
Show resolved Hide resolved

:::caution Pricing
QueryAPI is currently free. Pagoda doesn't charge for storage of your indexer code and data as well as running the indexer, but usage pricing will be introduced once QueryApi is out of beta.
:::
34 changes: 20 additions & 14 deletions website/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -580,20 +580,6 @@
"bos/api/storage"
]
},
{
"Indexing": [
"bos/queryapi/intro",
"bos/queryapi/index-functions",
"bos/community/indexers",
{
"Tutorials": [
"bos/tutorial/indexer-tutorials/posts-indexer",
"bos/tutorial/indexer-tutorials/hype-indexer",
"bos/tutorial/indexer-tutorials/feed-indexer"
]
}
]
},
{
"Examples & Tutorials": [
"bos/tutorial/quickstart",
Expand All @@ -606,6 +592,7 @@
"bos/tutorial/lido"
]
},
"bos/queryapi/bos-components",
{
"type": "html",
"value": "<hr/>"
Expand Down Expand Up @@ -684,6 +671,25 @@
"type": "html",
"value": "<hr/>"
},
{
"type": "html",
"value": "<span class='menu__link'><b><small> QueryAPI </small></b></span>"
},
"bos/queryapi/intro",
"bos/queryapi/how-it-works",
"bos/queryapi/index-functions",
"bos/community/indexers",
{
"Tutorials": [
"bos/tutorial/indexer-tutorials/posts-indexer",
"bos/tutorial/indexer-tutorials/hype-indexer",
"bos/tutorial/indexer-tutorials/feed-indexer"
]
},
{
"type": "html",
"value": "<hr/>"
},
{
"type": "html",
"value": "<span class='menu__link'><b><small> NEAR Lake Framework </small></b></span>"
Expand Down
Binary file added website/static/docs/qapi-components.png
bucanero marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading