TheGraph POC Testing
AlfredoG87 opened this issue · 3 comments
Problem
Before proceeding with implementing an Hedera TheGraph Node we need to do a POC and answer some questions first.
Solution
In order to assess the viability of setting up a Graph-Node hosted by SL for the community we need to assess first the following items:
- Long standing example sub-graph, indexing HTS Token Transactions.
- Verify that the server is receiving the Header for
auth
when sending the request using thegraph-cli
- Document Metrics and Usage footprint for Task 1 (subgraph). (storage size needed)
- Work on a second sub-graph for Saucer Token from genesis
Alternatives
No response
For Task # 1, There were some block inconsistencies that were needed to be indexed on the initial indexation since block:
I opened up a ticket to keep track of the issue and gather all information here.
Otherwise once the sub-graph indexing catched up to the Head of the chain, it has continued to index correctly for a couple days now.
Request on Port 8030
indexing status:
{
indexingStatusesForSubgraphName(
subgraphName: "subgraph-example-hts"
) {
synced
health
nonFatalErrors {
message
block {
number
hash
}
handler
deterministic
}
fatalError {
message
block {
number
hash
}
handler
}
chains {
chainHeadBlock {
number
}
latestBlock {
number
}
}
}
Response:
{
"data": {
"indexingStatusesForSubgraphName": [
{
"synced": true,
"health": "healthy",
"nonFatalErrors": [],
"fatalError": null,
"chains": [
{
"chainHeadBlock": {
"number": "51532212"
},
"latestBlock": {
"number": "51532212"
}
}
]
}
]
}
}
For Task # 2:
I was able to verify that the graph
(cli) tool is sending the following request to the graph-node
POST / HTTP/1.1
Content-Length: 214
Content-Type: application/json; charset=utf-8
Accept: application/json
User-Agent: jayson-4.0.0
Authorization: Bearer 1234567890
Host: localhost:3000
Connection: close
{"method":"subgraph_deploy","jsonrpc":"2.0","params":{"name":"subgraph-example-hts","ipfs_hash":"QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put","version_label":"0.0.4"},"id":"c771f120-fed4-4567-9c57-61d470a4b09c"}
As observed we can realize that there is the Authorization: Bearer 1234567890
Header that we can use for auth purposes.
Task #3:
Size of database with up-to-date simple HTS Subgraph, over 46K transfers.
Metrics from the graph-node itself:
# HELP deployment_block_processing_duration Measures duration of block processing for a subgraph deployment
# TYPE deployment_block_processing_duration histogram
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.05"} 132925
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.2"} 132946
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.7"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="1.5"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="4"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="10"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="60"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="120"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="240"} 132947
deployment_block_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 132947
deployment_block_processing_duration_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 365.7669097510014
deployment_block_processing_duration_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 132947
# HELP deployment_block_trigger_count Measures the number of triggers in each block for a subgraph deployment
# TYPE deployment_block_trigger_count histogram
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="1"} 39019
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="5"} 46747
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="10"} 46792
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="20"} 46793
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="50"} 46793
deployment_block_trigger_count_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 46793
deployment_block_trigger_count_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 55503
deployment_block_trigger_count_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 46793
# HELP deployment_count Counts the number of deployments currently being indexed by the graph-node.
# TYPE deployment_count gauge
deployment_count 1
# HELP deployment_eth_rpc_errors Counts eth rpc request errors for a subgraph deployment
# TYPE deployment_eth_rpc_errors counter
deployment_eth_rpc_errors{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",method="eth_getLogs",provider="mainnet-rpc-0"} 2
# HELP deployment_eth_rpc_request_duration Measures eth rpc request duration for a subgraph deployment
# TYPE deployment_eth_rpc_request_duration gauge
deployment_eth_rpc_request_duration{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",method="eth_getLogs",provider="mainnet-rpc-0"} 0.053104541
# HELP deployment_failed Boolean gauge to indicate whether the deployment has failed (1 == failed)
# TYPE deployment_failed gauge
deployment_failed{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",network="mainnet",shard="primary"} 0
# HELP deployment_handler_execution_time Measures the execution time for handlers
# TYPE deployment_handler_execution_time histogram
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="0.1"} 55261
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="0.5"} 55261
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="1"} 55261
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="10"} 55261
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="100"} 55261
deployment_handler_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer",le="+Inf"} 55261
deployment_handler_execution_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer"} 63.90402884999965
deployment_handler_execution_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",handler="handleTransfer"} 55261
# HELP deployment_head Track the head block number for a deployment
# TYPE deployment_head gauge
deployment_head{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",network="mainnet",shard="primary"} 51541331
# HELP deployment_host_fn_execution_time Measures the execution time for host functions
# TYPE deployment_host_fn_execution_time histogram
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="0.025"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="0.05"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="0.2"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="2"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="8"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="20"} 55261
deployment_host_fn_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get",le="+Inf"} 55261
deployment_host_fn_execution_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get"} 3.23170416
deployment_host_fn_execution_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",host_fn_name="store_get"} 55261
# HELP deployment_reverted_blocks Track the last reverted block for a subgraph deployment
# TYPE deployment_reverted_blocks gauge
deployment_reverted_blocks{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0
# HELP deployment_sync_secs total time spent syncing
# TYPE deployment_sync_secs counter
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="apply_entity_modifications",stage="writer"} 0.8650716760000147
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="apply_entity_modifications_insert",stage="writer"} 0.1729089430000004
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="apply_entity_modifications_update",stage="writer"} 0.2871829390000049
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="as_modifications",stage="process"} 99.75886822700022
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="check_interface_entity_uniqueness",stage="writer"} 1.842718979999987
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="host_export_other",stage="process"} 1.5708403470000105
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="host_export_store_get",stage="process"} 1.1181899439999918
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="host_export_store_set",stage="process"} 3.018785166999994
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="host_export_store_set__proof_of_indexing",stage="process"} 9.021823890999997
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="host_export_store_set__wasm_instance_context_store_set",stage="process"} 0.008189610999999522
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="insert_modification_insert_query",stage="writer"} 51.386125016999245
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="match_and_decode",stage="process"} 4.474970627000009
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="module_init",stage="process"} 17.16108450399986
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="queue_execute",stage="writer"} 414.48528818600334
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="queue_pop",stage="writer"} 1.1603487950000162
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="queue_wait",stage="writer"} 545994.7752053756
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="run_handler",stage="process"} 17.86891587500011
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="scan_blocks",stage="process"} 11395.774540397779
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="send_store_event",stage="writer"} 107.62298617800023
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="store_get_asc_new",stage="process"} 2.1611455529999923
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="transact_block",stage="process"} 32.53981580199996
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="transact_blocks_get_conn",stage="writer"} 87.43974710900254
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="unknown",stage="process"} 362315.9154172598
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="unknown",stage="writer"} 0.4533282410000071
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="update_modification_clamp_range_query",stage="writer"} 107.74126758500093
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="update_modification_insert_query",stage="writer"} 41.95000717299991
deployment_sync_secs{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",section="update_proof_of_indexing",stage="process"} 4.799766333000016
# HELP deployment_transact_block_operations_duration Measures duration of commiting all the entity operations in a block and updating the subgraph pointer
# TYPE deployment_transact_block_operations_duration histogram
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.01"} 132317
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.05"} 132700
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.1"} 132702
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.3"} 132705
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.7"} 132705
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="2"} 132705
deployment_transact_block_operations_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 132705
deployment_transact_block_operations_duration_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 34.33791630999938
deployment_transact_block_operations_duration_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 132705
# HELP deployment_trigger_processing_duration Measures duration of trigger processing for a subgraph deployment
# TYPE deployment_trigger_processing_duration histogram
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.01"} 55105
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.05"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.1"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.5"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="1.5"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="5"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="10"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="30"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="120"} 55261
deployment_trigger_processing_duration_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 55261
deployment_trigger_processing_duration_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 67.83632966999969
deployment_trigger_processing_duration_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 55261
# HELP eth_rpc_errors Counts eth rpc request errors
# TYPE eth_rpc_errors counter
eth_rpc_errors{method="eth_getLogs",provider="mainnet-rpc-0"} 2
# HELP eth_rpc_request_duration Measures eth rpc request duration
# TYPE eth_rpc_request_duration histogram
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="0.05"} 6
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="0.1"} 5933
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="0.2"} 8166
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="0.4"} 11652
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="0.8"} 15485
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="1.6"} 15834
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="3.2"} 15842
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="6.4"} 15842
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="12.8"} 15843
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="25.6"} 15844
eth_rpc_request_duration_bucket{method="eth_getLogs",provider="mainnet-rpc-0",le="+Inf"} 15844
eth_rpc_request_duration_sum{method="eth_getLogs",provider="mainnet-rpc-0"} 4283.206450164012
eth_rpc_request_duration_count{method="eth_getLogs",provider="mainnet-rpc-0"} 15844
# HELP eth_rpc_status Whether the provider has failed (0 = ok, 1 = net_version failed, 2 = get genesis failed, 3 = net_version timeout, 4 = get genesis timeout)
# TYPE eth_rpc_status gauge
eth_rpc_status{provider="mainnet-rpc-0"} 0
# HELP ethereum_chain_head_number Block number of the most recent block synced from Ethereum
# TYPE ethereum_chain_head_number gauge
ethereum_chain_head_number{network="mainnet"} 51541331
# HELP metrics_register_errors Counts Prometheus metrics register errors
# TYPE metrics_register_errors counter
metrics_register_errors 0
# HELP metrics_unregister_errors Counts Prometheus metrics unregister errors
# TYPE metrics_unregister_errors counter
metrics_unregister_errors 0
# HELP notification_queue_recvd Number of messages received through Postgres LISTEN
# TYPE notification_queue_recvd counter
notification_queue_recvd{channel="chain_head_updates",network="mainnet"} 235049
notification_queue_recvd{channel="store_events",network="none"} 46552
# HELP notification_queue_sent Number of messages sent through pg_notify()
# TYPE notification_queue_sent counter
notification_queue_sent{channel="chain_head_updates",network="mainnet"} 235049
notification_queue_sent{channel="store_events",network="none"} 46552
# HELP notification_queue_usage Time series of pg_notification_queue_usage()
# TYPE notification_queue_usage gauge
notification_queue_usage 0
# HELP polling_monitor_errors counts errors returned from the service being polled
# TYPE polling_monitor_errors counter
polling_monitor_errors{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0
# HELP polling_monitor_not_found counts 'not found' responses returned from the service being polled
# TYPE polling_monitor_not_found counter
polling_monitor_not_found{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0
# HELP polling_monitor_queue_depth size of the queue of polling requests
# TYPE polling_monitor_queue_depth gauge
polling_monitor_queue_depth{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0
# HELP polling_monitor_requests counts the total requests made to the service being polled
# TYPE polling_monitor_requests counter
polling_monitor_requests{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 1
# HELP query_cache_status_count Count toplevel GraphQL fields executed and their cache status
# TYPE query_cache_status_count counter
query_cache_status_count{cache_status="hit"} 58
query_cache_status_count{cache_status="insert"} 214
query_cache_status_count{cache_status="miss"} 223
query_cache_status_count{cache_status="shared"} 0
# HELP query_effort_ms Moving average of time spent running queries
# TYPE query_effort_ms gauge
query_effort_ms 0
# HELP query_execution_time Execution time for successful GraphQL queries
# TYPE query_execution_time histogram
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="0.1"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="0.5"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="1"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="10"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="100"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed",le="+Inf"} 1
query_execution_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed"} 0.008871669
query_execution_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="failed"} 1
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="0.1"} 147
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="0.5"} 272
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="1"} 272
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="10"} 272
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="100"} 272
query_execution_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success",le="+Inf"} 272
query_execution_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success"} 22.392210172000006
query_execution_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",status="success"} 272
query_execution_time_bucket{deployment="notfound",status="failed",le="0.1"} 9
query_execution_time_bucket{deployment="notfound",status="failed",le="0.5"} 9
query_execution_time_bucket{deployment="notfound",status="failed",le="1"} 9
query_execution_time_bucket{deployment="notfound",status="failed",le="10"} 9
query_execution_time_bucket{deployment="notfound",status="failed",le="100"} 9
query_execution_time_bucket{deployment="notfound",status="failed",le="+Inf"} 9
query_execution_time_sum{deployment="notfound",status="failed"} 0.021334936999999995
query_execution_time_count{deployment="notfound",status="failed"} 9
query_execution_time_bucket{deployment="unknown",status="failed",le="0.1"} 2
query_execution_time_bucket{deployment="unknown",status="failed",le="0.5"} 2
query_execution_time_bucket{deployment="unknown",status="failed",le="1"} 2
query_execution_time_bucket{deployment="unknown",status="failed",le="10"} 2
query_execution_time_bucket{deployment="unknown",status="failed",le="100"} 2
query_execution_time_bucket{deployment="unknown",status="failed",le="+Inf"} 2
query_execution_time_sum{deployment="unknown",status="failed"} 0.008601302
query_execution_time_count{deployment="unknown",status="failed"} 2
# HELP query_kill_rate The rate at which the load manager kills queries
# TYPE query_kill_rate gauge
query_kill_rate 0
# HELP query_parsing_time Parsing time for GraphQL queries
# TYPE query_parsing_time histogram
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.1"} 273
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.5"} 273
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="1"} 273
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="10"} 273
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="100"} 273
query_parsing_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 273
query_parsing_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0.035454687
query_parsing_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 273
query_parsing_time_bucket{deployment="notfound",le="0.1"} 9
query_parsing_time_bucket{deployment="notfound",le="0.5"} 9
query_parsing_time_bucket{deployment="notfound",le="1"} 9
query_parsing_time_bucket{deployment="notfound",le="10"} 9
query_parsing_time_bucket{deployment="notfound",le="100"} 9
query_parsing_time_bucket{deployment="notfound",le="+Inf"} 9
query_parsing_time_sum{deployment="notfound"} 0.0035537029999999996
query_parsing_time_count{deployment="notfound"} 9
query_parsing_time_bucket{deployment="unknown",le="0.1"} 2
query_parsing_time_bucket{deployment="unknown",le="0.5"} 2
query_parsing_time_bucket{deployment="unknown",le="1"} 2
query_parsing_time_bucket{deployment="unknown",le="10"} 2
query_parsing_time_bucket{deployment="unknown",le="100"} 2
query_parsing_time_bucket{deployment="unknown",le="+Inf"} 2
query_parsing_time_sum{deployment="unknown"} 0.000236469
query_parsing_time_count{deployment="unknown"} 2
# HELP query_result_max the maximum size of a query result (in CacheWeight)
# TYPE query_result_max gauge
query_result_max 967356
# HELP query_result_size the size of the result of successful GraphQL queries (in CacheWeight)
# TYPE query_result_size histogram
query_result_size_bucket{le="1024"} 1
query_result_size_bucket{le="2048"} 69
query_result_size_bucket{le="4096"} 85
query_result_size_bucket{le="8192"} 92
query_result_size_bucket{le="16384"} 112
query_result_size_bucket{le="32768"} 128
query_result_size_bucket{le="65536"} 137
query_result_size_bucket{le="131072"} 161
query_result_size_bucket{le="262144"} 168
query_result_size_bucket{le="524288"} 184
query_result_size_bucket{le="1048576"} 214
query_result_size_bucket{le="2097152"} 214
query_result_size_bucket{le="4194304"} 214
query_result_size_bucket{le="8388608"} 214
query_result_size_bucket{le="16777216"} 214
query_result_size_bucket{le="33554432"} 214
query_result_size_bucket{le="67108864"} 214
query_result_size_bucket{le="134217728"} 214
query_result_size_bucket{le="268435456"} 214
query_result_size_bucket{le="536870912"} 214
query_result_size_bucket{le="1073741824"} 214
query_result_size_bucket{le="2147483648"} 214
query_result_size_bucket{le="+Inf"} 214
query_result_size_sum 38181436
query_result_size_count 214
# HELP query_semaphore_wait_ms Moving average of time spent on waiting for postgres query semaphore
# TYPE query_semaphore_wait_ms gauge
query_semaphore_wait_ms{pool="main",shard="primary"} 0
# HELP query_validation_time Validation time for GraphQL queries
# TYPE query_validation_time histogram
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.1"} 275
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="0.5"} 275
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="1"} 275
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="10"} 275
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="100"} 275
query_validation_time_bucket{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put",le="+Inf"} 275
query_validation_time_sum{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 0.002678153000000001
query_validation_time_count{deployment="QmTsCPPVU6U5aRnawWCvWQa5zNtsrnDF95op1gTCUL8Put"} 275
# HELP registered_metrics Tracks the number of registered metrics on the node
# TYPE registered_metrics gauge
registered_metrics 40
# HELP store_connection_checkout_count The number of Postgres connections currently checked out
# TYPE store_connection_checkout_count gauge
store_connection_checkout_count{pool="main",shard="primary"} 0
# HELP store_connection_error_count The number of Postgres connections errors
# TYPE store_connection_error_count counter
store_connection_error_count{pool="main",shard="primary"} 0
# HELP store_connection_pool_size_count Overall size of the connection pool
# TYPE store_connection_pool_size_count gauge
store_connection_pool_size_count{pool="main",shard="primary"} 11
# HELP store_connection_wait_time_ms Average connection wait time
# TYPE store_connection_wait_time_ms gauge
store_connection_wait_time_ms{pool="main",shard="primary"} 0