Skip to content

REST API

Jump to bottom
Piotr Kosecki edited this page Jul 15, 2021 · 3 revisions

REST API Overview & Examples

If you're already running a Conseil instance, there is an OpenAPI route at <conseil-host>/docs.

Metadata Discovery

Conseil provides a dynamic query API. This means that newly exposed datasets can be automatically served through the interface. The metadata routes have a prefix of /v2/metadata/. All results are in JSON format.

Platforms

To get a list of platforms, make the following call.

curl --request GET --header 'apiKey: <API key>' --url '<conseil-host>/v2/metadata/platforms'

The result may look like the following.

[{
    "name": "tezos",
    "displayName": "Tezos"
}]

Networks

To get a list of networks, take one of the platform name attributes from the call above to construct the following URL.

curl --request GET --header 'apiKey: <API key>' --url '<conseil-host>/v2/metadata/tezos/networks'

On a development server the result might be this:

[{
    "name": "carthagenet",
    "displayName": "Carthagenet",
    "platform": "tezos",
    "network": "carnet"
}, {
    "name": "mainnet",
    "displayName": "Mainnet",
    "platform": "tezos",
    "network": "mainnet"
}]

Entities

Taking again the name property from one of the network results, we can list the entities available for that network.

curl --request GET --header 'apiKey: <API key>' --url '<conseil-host>/v2/metadata/tezos/mainnet/entities'

With the following sample result.

[{
    "name": "accounts",
    "displayName": "Accounts",
    "count": 19587
}, {
    "name": "accounts_checkpoint",
    "displayName": "Accounts checkpoint",
    "count": 5
}, {
    "name": "bakers",
    "displayName": "Bakers",
    "count": 9232185
}, {
    "name": "balance_updates",
    "displayName": "Balance updates",
    "count": 8848857
}, {
    "name": "ballots",
    "displayName": "Ballots",
    "count": 0
}, {
    "name": "blocks",
    "displayName": "Blocks",
    "count": 346319
}, {
    "name": "fees",
    "displayName": "Fees",
    "count": 1680
}, {
    "name": "operation_groups",
    "displayName": "Operation groups",
    "count": 2590969
}, {
    "name": "operations",
    "displayName": "Operations",
    "count": 2619443
}, {
    "name": "proposals",
    "displayName": "Proposals",
    "count": 0
}]

The count property above contains the number of records for a particular entity.

Keep in mind that not all of the entities provided in that list map directly to on-chain entities on Tezos mainnet. Conseil may provide custom pre-processed data for convenience. In this example the fees is one such entity that contains moving averages for operation fees in Tezos.

Attributes

Using the results from the entities call, we can get details of their composition.

curl --request GET --header 'apiKey: hooman' --url '<conseil-host>/v2/metadata/tezos/mainnet/accounts/attributes'
[{
    "name": "account_id",
    "displayName": "Account id",
    "dataType": "String",
    "cardinality": 19587,
    "keyType": "UniqueKey",
    "entity": "accounts"
}, {
    "name": "block_id",
    "displayName": "Block id",
    "dataType": "String",
    "cardinality": 4614,
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "manager",
    "displayName": "Manager",
    "dataType": "String",
    "cardinality": 15572,
    "keyType": "UniqueKey",
    "entity": "accounts"
}, {
    "name": "spendable",
    "displayName": "Spendable",
    "dataType": "Boolean",
    "cardinality": 2,
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "delegate_setable",
    "displayName": "Delegate setable",
    "dataType": "Boolean",
    "cardinality": 2,
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "delegate_value",
    "displayName": "Delegate value",
    "dataType": "String",
    "cardinality": 143,
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "counter",
    "displayName": "Counter",
    "dataType": "Int",
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "script",
    "displayName": "Script",
    "dataType": "String",
    "cardinality": 1317,
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "balance",
    "displayName": "Balance",
    "dataType": "Decimal",
    "keyType": "NonKey",
    "entity": "accounts"
}, {
    "name": "block_level",
    "displayName": "Block level",
    "dataType": "Decimal",
    "keyType": "UniqueKey",
    "entity": "accounts"
}]

Using the information above it is possible to make intelligent decisions about how to construct user interfaces. For example, Arronax creates drop-down lists for value selection for low-cardinality properties like accounts.spendable. Furthermore the datatype field allows display of the appropriate editor be it numeric or date entry. You'll note also that certain properties do no contain cardinality. Those are considered high cardinality fields for which that metadata is not collected.

Attribute Values

Finally for low-cardinality fields it is possible to get a list of distinct values to populate the drop-down lists mentioned above for example.

curl --request GET --header 'apiKey: <API key>' --url '<conseil-host>/v2/metadata/tezos/mainnet/operations/kind'

These are the operation types this particular Conseil instance has seen since start.

["seed_nonce_revelation", "delegation", "transaction", "activate_account", "origination", "reveal", "double_endorsement_evidence", "double_baking_evidence", "endorsement"]

Attribute Values with Prefix

It is possible to get values for high-cardinality properties with a prefix. In this example we get all accounts.manager values starting with "KT1".

curl --request GET --header 'apiKey: <API key>' --url 'https://conseil-dev.cryptonomic-infra.tech:443/v2/metadata/tezos/mainnet/accounts/manager/KT1'
...

Extended metadata

As mentioned in the Datasource configuration section, it's possible to provide additional details for the items Conseil manages.

Tezos Chain Data Query

Data requests in Conseil v2 API are sent via POST command and have a prefix of /v2/data/<platform>/<network>/<entity>. The items in angle braces match up to the name property of the related metadata results. There additional header to be set in the request: Content-Type: application/json.

The most basic query has the following JSON structure.

{
    "fields": [],
    "predicates": [],
    "limit": 5
}

This will return five items without any filtering for some entity. Note that the query does not specify which entity is being requested. If this is sent to /v2/data/tezos/mainnet/blocks, it will return five rows with all the fields available in the blocks entity.

curl -d '{ "fields": [], "predicates": [], "limit": 5 }' -H 'Content-Type: application/json' -H 'apiKey: <API key>' -X POST '<conseil-host>/v2/data/tezos/mainnet/blocks/'

The query grammar allows for some fine-grained data extraction.

fields

It is possible to narrow down the fields returned in the result set by listing just the ones that are necessary. The fields property of the query is an array of string, these values are the names from the /v2/metadata/<platform>/<network>/<entity>/attributes/ metadata response.

predicates

predicates is an array containing objects. The inner object has several properties:

  • field – attribute name from the metadata response.
  • operation – one of: in, between, like, lt, gt, eq, startsWith, endsWith, before, after.
  • set – an array of values to compare against. 'in' requires two or more elements, 'between' must have exactly two, the rest of the operations require a single element. Dates are expressed as epoch milliseconds.
  • inverse – boolean, setting it to true applied a NOT to the operator
  • precision – for numeric field matches this specifies the number of decimal places to round to.
  • group - an optional naming to collect predicates in groups that will be eventually OR-ed amongst each other

limit

Specifies the maximum number of records to return.

orderBy

Sort condition, multiple may be supplied for a single query. Inner object(s) will contain field and direction properties. The former is name from the /v2/metadata/<platform>/<network>/<entity>/attributes/ metadata response, the latter is one of 'asc', 'desc'.

It is possible to sort on aggregated results, in this case the field value would be "function_fieldname", for example sum_balance. See query samples below.

aggregation

It is possible to apply an aggregation function to a field of the result set. The aggregation object contains the following properties:

  • field – name from the /v2/metadata/<platform>/<network>/<entity>/attributes/ metadata response.
  • function – one of: sum, count, max, min, avg.
  • predicate – same definition as the predicate object described above. This gets translated into a HAVING condition in the underlying SQL.

Multiple fields can be aggregated and the same field can be aggregated with different functions in the same request. See examples below.

output

Default result set format is JSON, it is possible however to output csv instead for exporting larger datasets that will then be imported into other tools. There is also possibility to return query which is being ran on the DB with sql output type.

'output': 'csv|json|sql'

Business Intelligence & Analytics

Here's a collection of interesting datasets. Run these examples as

curl -H 'Content-Type: application/json' -H 'apiKey: <API key>' \
-X POST '<conseil-host>/v2/data/<platform>/<network>/<entity>/' \
-d '<example>'

Bakers by block count in April 2019

Send this query to /v2/data/tezos/<network>/blocks Note that orderBy below ends up sorting by the aggregated value.

{
    "fields": ["baker", "level"],
    "predicates": [{ "field": "timestamp", "set": [1554076800000, 1556668799000], "operation": "between", "inverse": false }],
    "orderBy": [{ "field": "count_level", "direction": "desc" }],
    "aggregation": [{ "field": "level", "function": "count" }],
    "limit": 50,
    "output": "csv"
}

When you make this query with sql output

{
    "fields": ["baker", "level"],
    "predicates": [{ "field": "timestamp", "set": [1554076800000, 1556668799000], "operation": "between", "inverse": false }],
    "orderBy": [{ "field": "count_level", "direction": "desc" }],
    "aggregation": [{ "field": "level", "function": "count" }],
    "limit": 50,
    "output": "sql"
}

you'll get following response:

SELECT COUNT(level) as count_level,baker FROM tezos.blocks WHERE true  AND timestamp BETWEEN '2019-04-01 00:00:00.0' AND '2019-04-30 23:59:59.0' AND invalidated_asof ISNULL GROUP BY baker ORDER BY count_level desc LIMIT 50

Top 50 Bakers by delegator balance

Send this query to /v2/data/tezos/<network>/accounts

{
    "fields": ["delegate_value", "balance"],
    "predicates": [{ "field": "delegate_value", "set": [], "operation": "isnull", "inverse": true }],
    "orderBy": [{ "field": "sum_balance", "direction": "desc" }],
    "aggregation": [{ "field": "balance", "function": "sum" }],
    "limit": 50,
    "output": "csv"
}

Top 50 Bakers by delegator count

Send this query to /v2/data/tezos/<network>/accounts

{
    "fields": ["delegate_value", "account_id"],
    "predicates": [{ "field": "delegate_value", "set": [], "operation": "isnull", "inverse": true }],
    "orderBy": [{ "field": "count_account_id", "direction": "desc" }],
    "aggregation": [{ "field": "account_id", "function": "count" }],
    "limit": 50,
    "output": "csv"
}

Top 20 Bakers by roll count

Send this query to /v2/data/tezos/<network>/rolls

{
    "fields": ["pkh", "rolls"],
    "predicates": [],
    "orderBy": [
        { "field": "block_level", "direction": "desc" },
        { "field": "rolls", "direction": "desc" }
    ],
    "limit": 20,
    "output": "csv"
}

All originated accounts which are smart contracts

Send this query to /v2/data/tezos/<network>/accounts

{
    "fields": ["account_id"],
    "predicates": [
        { "field": "account_id", "set": ["KT1"], "operation": "startsWith", "inverse": false },
        { "field": "script", "set": [], "operation": "isnull", "inverse": true }
    ],
    "limit": 10000,
    "output": "csv"
}

Top 10 most active contracts

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["destination", "operation_group_hash"],
    "predicates": [
        { "field": "kind", "set": ["transaction"], "operation": "eq", "inverse": false },
        { "field": "destination", "set": ["KT1"], "operation": "startsWith", "inverse": false },
        { "field": "parameters", "set": [], "operation": "isnull", "inverse": true }
    ],
    "orderBy": [{ "field": "count_operation_group_hash", "direction": "desc" }],
    "aggregation": [{ "field": "operation_group_hash", "function": "count" }],
    "limit": 10,
    "output": "csv"
}

Top 10 contract originators

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["source", "operation_group_hash"],
    "predicates": [
        { "field": "kind", "set": ["origination"], "operation": "eq", "inverse": false },
        { "field": "script", "set": [], "operation": "isnull", "inverse": true }
    ],
    "orderBy": [{ "field": "count_operation_group_hash", "direction": "desc" }],
    "aggregation": [{ "field": "operation_group_hash", "function": "count" }],
    "limit": 10,
    "output": "csv"
}

Top 100 transactions in 2019

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["source", "amount", "fee"],
    "predicates": [
        { "field": "kind", "set": ["transaction"], "operation": "eq", "inverse": false },
        { "field": "timestamp", "set": [1546300800000, 1577836799000], "operation": "between", "inverse": false }
    ],
    "orderBy": [{ "field": "sum_amount", "direction": "desc" }],
    "aggregation": [{ "field": "amount", "function": "sum" }, { "field": "fee", "function": "avg" }
    ],
    "limit": 50,
    "output": "csv"
}

Fees by block level for transaction operations in April 2019

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["block_level", "kind", "fee"],
    "predicates": [
        { "field": "timestamp", "set": [1554076800000, 1556668799000], "operation": "between", "inverse": false },
        { "field": "fee", "set": [0], "operation": "gt", "inverse": false }
    ],
    "orderBy": [{ "field": "sum_fee", "direction": "desc" }],
    "aggregation": [{ "field": "fee", "function": "sum" }, { "field": "fee", "function": "avg" }],
    "limit": 100000,
    "output": "csv"
}

Number of transactions by type in April 2019

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["kind", "operation_group_hash"],
    "predicates": [
        { "field": "timestamp", "set": [1554076800000, 1556668799000], "operation": "between", "inverse": false }
    ],
    "orderBy": [{ "field": "count_operation_group_hash", "direction": "desc" }],
    "aggregation": [{ "field": "operation_group_hash", "function": "count" }],
    "limit": 20,
    "output": "csv"
}

Blocks by end-user transaction count in April 2019

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["block_level", "operation_group_hash"],
    "predicates": [
        { "field": "timestamp", "set": [1554076800000, 1556668799000], "operation": "between", "inverse": false },
        { "field": "kind", "set": ["transaction", "origination", "delegation", "activation", "reveal"], "operation": "in", "inverse": false }
    ],
    "orderBy": [{ "field": "count_operation_group_hash", "direction": "desc" }],
    "aggregation": [{ "field": "operation_group_hash", "function": "count" }],
    "limit": 100,
    "output": "csv"
}

Top 20 account controllers

Send this query to /v2/data/tezos/<network>/accounts

{
    "fields": ["manager", "account_id"],
    "predicates": [
        { "field": "script", "set": [], "operation": "isnull", "inverse": false },
        { "field": "balance", "set": [0], "operation": "gt", "inverse": false }
    ],
    "orderBy": [{ "field": "count_account_id", "direction": "desc" }],
    "aggregation": [{ "field": "account_id", "function": "count" }],
    "limit": 20,
    "output": "csv"
}

Top 20 account controllers by balance

Send this query to /v2/data/tezos/<network>/accounts

{
    "fields": ["manager", "balance"],
    "predicates": [
        { "field": "script", "set": [], "operation": "isnull", "inverse": false }
    ],
    "orderBy": [{ "field": "sum_balance", "direction": "desc" }],
    "aggregation": [{ "field": "balance", "function": "sum" }],
    "limit": 20,
    "output": "csv"
}

Blocks by end-user-transactions outside April 2019

Send this query to /v2/data/tezos/<network>/operations

{
    "fields": ["block_level", "operation_group_hash"],
    "predicates": [
        { "field": "timestamp", "set": [1554076800000], "operation": "before", "inverse": false, "group": "before" },
        { "field": "kind", "set": ["transaction"], "operation": "in", "inverse": false, "group": "before" },
        { "field": "timestamp", "set": [1556668799000], "operation": "after", "inverse": false, "group": "after" },
        { "field": "kind", "set": ["transaction"], "operation": "in", "inverse": false, "group": "after" }
    ],
    "orderBy": [{ "field": "count_operation_group_hash", "direction": "desc" }],
    "aggregation": [{ "field": "operation_group_hash", "function": "count" }],
    "limit": 100,
    "output": "csv"
}

A note on predicate conjunction/disjunction

  • By default all predicates with no specified group-name belong to the same un-named group
  • predicates within the same group are joined via the "AND" operator
  • predicates between separate groups are joining via the "OR" operator

Query on the temporal table

Send this query to /v2/data/tezos/<network>/accounts_history

Example temporal table queries:

Get the balance of an account at a specific timestamp.

{
  "predicates": [
    {
      "field": "account_id",
      "operation": "eq",
      "set": ["KT18x4B5BkyGMR5DCr7esP8pC5biQc1M6CGr"]
    }
  ],
  "snapshot": {
    "field": "asof",
    "value": 11530382887000
  },
  "aggregation": [],
  "fields": [
    "account_id", "balance", "block_level", "asof"
  ],
  "orderBy": [
  ],
  "output": "json",
  "limit": 100
}

Get the balance of an account at a specific block level

{
  "predicates": [
    {
      "field": "account_id",
      "operation": "eq",
      "set": ["KT18x4B5BkyGMR5DCr7esP8pC5biQc1M6CGr"]
    }
  ],
  "snapshot": {
    "field": "block_level",
    "value": 2700
  },
  "aggregation": [],
  "fields": [
    "account_id", "balance", "block_level", "asof"
  ],
  "orderBy": [
  ],
  "output": "json",
  "limit": 100
}

Get all records for an account over a range of blocks

{
  "predicates": [
    {
      "field": "account_id",
      "operation": "eq",
      "set": ["KT18x4B5BkyGMR5DCr7esP8pC5biQc1M6CGr"]
    },
    {
      "field": "block_level",
      "operation": "between",
      "set": [1500, 3000]
    }
  ],
  "aggregation": [],
  "fields": [
    "account_id", "balance", "block_level", "asof"
  ],
  "orderBy": [
  ],
  "output": "json",
  "limit": 100
}

Note on temporal tables:

  • Temporal tables contains asof and block_level columns which have information about the timestamp and block level
  • besides that they contain the same columns as regular tables and can be queried as them

Preprocessed Data

Not all entities are necessarily items from the chain. One such example is /v2/data/tezos/<network>/fees. Conseil continuously buckets and averages network fees to give users an idea of what values they should submit when sending operations. Fees are aggregated by operation type.

{
    "fields": [],
    "predicates": [{ "field": "kind", "operation": "eq", "set": [ "transaction" ] }],
    "orderBy": [{"field": "timestamp", "direction": "desc"}],
    "limit": 1
}
Clone this wiki locally