field 'epochs' not found in type: 'query_root'
liv-io opened this issue · 3 comments
Summary
Hello,
I am unable to run cardano-graphql-background
in the preview environment. I am running the whole stack as a "Podman Pod" (like Docker Compose) on a single virtual machine.
-
The versions are:
ubuntu 22.04 (virtual machine and containers) podman 3.0.1 node v14.21.3 cardano-node 1.35.5 cardano-pgsql 14.6 cardano-db-sync 13.1.0.0 cardano-ogmios 5.6.0 cardano-graphql-engine 2.19.0 cardano-graphql-background 8.0.0
-
When I start
cardano-graphql-background
for the very first time after purging thecardano-pgsql
database andcardano-db-sync
data, I get:node /usr/local/src/cardano-graphql/packages/api-cardano-db-hasura/dist/background.js {"name":"background","hostname":"pod_cardano","pid":74,"level":30,"module":"Db","msg":"Initializing...","time":"2023-02-27T09:23:18.820Z","v":0} {"name":"background","hostname":"pod_cardano","pid":74,"level":20,"module":"Db","msg":"pgSubscriber: Connected","time":"2023-02-27T09:23:18.840Z","v":0} {"name":"background","hostname":"pod_cardano","pid":74,"level":30,"module":"HasuraBackgroundClient","msg":"Initializing","time":"2023-02-27T09:23:18.841Z","v":0} {"name":"background","hostname":"pod_cardano","pid":74,"level":20,"module":"HasuraBackgroundClient","msg":"graphql-engine setup","time":"2023-02-27T09:23:31.217Z","v":0} {"name":"background","hostname":"pod_cardano","pid":74,"level":20,"msg":"Detecting DB sync state has reached minimum progress: Attempt 1 of 101, retrying...","time":"2023-02-27T09:23:31.261Z","v":0} <...> {"name":"background","hostname":"pod_cardano","pid":74,"level":20,"msg":"Detecting DB sync state has reached minimum progress: Attempt 101 of 101, exiting","time":"2023-02-27T10:07:02.307Z","v":0} {"name":"background","hostname":"pod_cardano","pid":74,"level":50,"msg":"field 'epochs' not found in type: 'query_root': {\"response\":{\"errors\":[{\"extensions\":{\"code\":\"validation-failed\",\"path\":\"$.selectionSet.epochs\"},\"message\":\"field 'epochs' not found in type: 'query_root'\"}],\"status\":200},\"request\":{\"query\":\"query {\\n epochs (limit: 1, order_by: { number: desc }) {\\n number\\n }\\n }\"}}","time":"2023-02-27T10:07:02.307Z","v":0}
-
When I run it a second time, I get:
node /usr/local/src/cardano-graphql/packages/api-cardano-db-hasura/dist/background.js {"name":"background","hostname":"pod_cardano","pid":113,"level":30,"module":"Db","msg":"Initializing...","time":"2023-02-27T10:10:30.456Z","v":0} {"name":"background","hostname":"pod_cardano","pid":113,"level":20,"module":"Db","msg":"pgSubscriber: Connected","time":"2023-02-27T10:10:30.467Z","v":0} {"name":"background","hostname":"pod_cardano","pid":113,"level":30,"module":"HasuraBackgroundClient","msg":"Initializing","time":"2023-02-27T10:10:30.468Z","v":0} {"name":"background","hostname":"pod_cardano","pid":113,"level":20,"msg":"Applying PostgreSQL schema migrations: Attempt 1 of 10, retrying...","time":"2023-02-27T10:10:30.964Z","v":0} <...> {"name":"background","hostname":"pod_cardano","pid":113,"level":20,"msg":"Applying PostgreSQL schema migrations: Attempt 10 of 10, exiting","time":"2023-02-27T10:13:58.759Z","v":0} {"name":"background","hostname":"pod_cardano","pid":113,"level":50,"msg":"Command failed: /usr/local/bin/cli-hasura --skip-update-check --project /usr/local/src/cardano-graphql/packages/api-cardano-db-hasura/hasura/project --endpoint http://localhost:8080 migrate --database-name default apply --down all\n{\"level\":\"warning\",\"msg\":\"[cli: v2.19.0] [server: v12345.0.0] version mismatch: older cli version might not be compatible with latest server apis, please update cli\",\"time\":\"2023-02-27T10:13:58Z\"}\n{\"level\":\"error\",\"msg\":\"skipping applying migrations on database 'default', encountered: \\n{\\n \\\"code\\\": \\\"not-exists\\\",\\n \\\"error\\\": \\\"source with name \\\\\\\"default\\\\\\\" does not exist\\\",\\n \\\"path\\\": \\\"$\\\"\\n}\",\"time\":\"2023-02-27T10:13:58Z\"}\ntime=\"2023-02-27T10:13:58Z\" level=fatal msg=\"applying migrations failed on database(s): default\"\n","time":"2023-02-27T10:13:58.760Z","v":0}
-
Interestingly, the
hasura-graphql-engine
health status changes fromOK
toWARN
after the first attempt:curl -i --silent http://localhost:8080/healthz HTTP/1.1 200 OK OK
curl -i --silent http://localhost:8080/healthz HTTP/1.1 200 OK WARN: inconsistent objects in schema
I would appreciate any inputs and help. Please let me know if you need more details of my setup.
Steps to reproduce the bug
- Run the listed software versions against the
preview
network - Ensure
cardano-node
andcardano-db-sync
are fully synchronized - Ensure
cardano-ogmios
andcardano-graphql-engine
are running - Start
cardano-graphql-background
- Observe output
Actual Result
-
When running
cardano-graphql-background
on a newly and fully synchronized database, I get:field 'epochs' not found in type: 'query_root
-
When running
cardano-graphql-background
a second time I get:Command failed: /usr/local/bin/cli-hasura --skip-update-check --project /usr/local/src/cardano-graphql/packages/api-cardano-db-hasura/hasura/project --endpoint http://localhost:8080 migrate --database-name default apply --down all
Expected Result
I would expect cardano-graphql-background
to run without throwing errors, especially on a new and clean database/db-sync.
Environment
Unfortunately, I fail at installing and running this CLI:
root@pod_cardano:/usr/local/src/cardano-graphql/node_modules# npx @cardano-graphql/cli system-info
sh: 1: cannot create buildcheck.gypi: Permission denied
internal/modules/cjs/loader.js:934
throw err;
^
Error: Cannot find module '/root/.npm/_npx/1537/lib/node_modules/@cardano-graphql/cli/node_modules/ssh2/install.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:931:15)
at Function.Module._load (internal/modules/cjs/loader.js:774:27)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)
at internal/main/run_main_module.js:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! ssh2@1.11.0 install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the ssh2@1.11.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2023-02-27T10_49_13_748Z-debug.log
Install for [ '@cardano-graphql/cli@latest' ] failed with code 1
Platform
- Linux (Ubuntu)
- Linux (Other)
- macOS
- Windows
Platform version
Ubuntu 22.04 LTS
Runtime
- Node.js
- Docker
Runtime version
node: v14.21.3
I thought I additionally share an excerpt of my other containers:
-
cardano-node
podman logs --follow pod_cardano-cardano-node [pod_card:cardano.node.ChainDB:Notice:196] [2023-02-27 09:11:32.26 UTC] Chain extended, new tip: a0a73150fb370d58f684af4675c2b5055bb3f8089b12b3b9d11ca775032f6825 at slot 10833092 [pod_card:cardano.node.ChainDB:Notice:196] [2023-02-27 09:12:20.52 UTC] Chain extended, new tip: c4a80802aaccbbb194c0e0317668df4905b2e3c0f482e2938db87d01d7c05d36 at slot 10833140
-
cardano-pgsql
podman logs --follow pod_cardano-cardano-pgsql` 2023-02-27 09:03:49.497 UTC [9] LOG: starting PostgreSQL 14.6 (Ubuntu 14.6-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, 64-bit 2023-02-27 09:03:49.498 UTC [9] LOG: listening on IPv4 address "0.0.0.0", port 5432 <...> 2023-02-27 09:03:53.353 UTC [9] LOG: database system is ready to accept connections
-
cardano-db-sync
podman logs --follow pod_cardano-cardano-db-sync All good! [db-sync-node:Info:6] [2023-02-27 09:06:35.04 UTC] Version number: 13.1.0.0 <...> [db-sync-node.Subscription:Notice:54] [2023-02-27 09:06:41.17 UTC] Identity Connection Attempt Start, destination LocalAddress "/var/local/cardano-node/db/node.socket" [db-sync-node.Subscription:Notice:54] [2023-02-27 09:06:41.17 UTC] Identity Connection Attempt End, destination LocalAddress "/var/local/cardano-node/db/node.socket" outcome: ConnectSuccessLast <...> [db-sync-node:Info:66] [2023-02-27 09:11:32.45 UTC] Insert Babbage Block: epoch 125, slot 10833092, block 492104, hash a0a73150fb370d58f684af4675c2b5055bb3f8089b12b3b9d11ca775032f6825 [db-sync-node:Info:66] [2023-02-27 09:12:20.74 UTC] Insert Babbage Block: epoch 125, slot 10833140, block 492105, hash c4a80802aaccbbb194c0e0317668df4905b2e3c0f482e2938db87d01d7c05d36
-
cardano-ogmios
podman logs --follow pod_cardano-cardano-ogmios {"severity":"Info","timestamp":"2023-02-27T09:11:32.272465547Z","thread":"17","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2023-02-27T09:06:33.626451503Z","lastKnownTip":{"slot":10833092,"hash":"a0a73150fb370d58f684af4675c2b5055bb3f8089b12b3b9d11ca775032f6825","blockNo":492104},"lastTipUpdate":"2023-02-27T09:11:32.27151958Z","networkSynchronization":1.00000,"currentEra":"Babbage","metrics":{"activeConnections":0,"runtimeStats":{"cpuTime":633202081,"currentHeapSize":391,"gcCpuTime":350322814,"maxHeapSize":394},"sessionDurations":{"max":0,"mean":0,"min":0},"totalConnections":0,"totalMessages":0,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":125,"slotInEpoch":33092}}},"version":"v5.6.0"} {"severity":"Info","timestamp":"2023-02-27T09:06:54.498292305Z","thread":"17","message":{"Health":{"tag":"HealthTick","status":{"startTime":"2023-02-27T09:06:33.626451503Z","lastKnownTip":{"slot":10832814,"hash":"bebb2ffc3a638f4102890ef1294aad03486fa73c5ee2afac8691afdb77bda9ce","blockNo":492091},"lastTipUpdate":"2023-02-27T09:06:54.498116003Z","networkSynchronization":1.00000,"currentEra":"Babbage","metrics":{"activeConnections":0,"runtimeStats":{"cpuTime":232617944,"currentHeapSize":390,"gcCpuTime":106704140,"maxHeapSize":390},"sessionDurations":{"max":0,"mean":0,"min":0},"totalConnections":0,"totalMessages":0,"totalUnrouted":0},"connectionStatus":"connected","currentEpoch":125,"slotInEpoch":32814}}},"version":"v5.6.0"}
curl --silent http://localhost:1337/metrics # TYPE ogmios_active_connections gauge ogmios_active_connections 0.0 # TYPE ogmios_connected gauge ogmios_connected 1.0 <...>
-
cardano-graphql-engine
podman logs --follow pod_cardano-cardano-graphql-engine {"detail":{"info":{"admin_secret_set":false,"auth_hook":null,"auth_hook_mode":null,"console_assets_dir":null,"console_sentry_dsn":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"enable_allowlist":false,"enable_console":true,"enable_maintenance_mode":false,"enable_metadata_query_logging":false,"enable_telemetry":false,"enabled_apis":["config","pgdump","metadata","graphql"],"enabled_log_types":["startup","webhook-log","http-log","query-log","websocket-log"],"events_fetch_batch_size":100,"experimental_features":[],"graceful_shutdown_timeout":60,"infer_function_permissions":true,"jwt_secret":[],"live_query_options":{"batch_size":100,"refetch_delay":1},"log_level":"debug","port":8080,"remote_schema_permissions":false,"server_host":"HostAny","stringify_numeric_types":true,"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","unauth_role":null,"use_prepared_statements":true,"v1-boolean-null-collapse":false,"websocket_compression_options":"NoCompression","websocket_connection_init_timeout":"Refined (Seconds {seconds = 3s})","websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Refined (Seconds {seconds = 5s})}"},"kind":"server_configuration"},"level":"info","timestamp":"2023-02-27T09:03:55.255+0000","type":"startup"} {"detail":{"info":{"database_url":"postgresql://dbsync:...@localhost:5432/dbsync","retries":1},"kind":"postgres_connection"},"level":"info","timestamp":"2023-02-27T09:03:55.255+0000","type":"startup"}
curl --silent http://localhost:8080/healthz OK
Alright, I might have found the solution to my issue. The cardano-graphql-engine
container is supposed to have the following environment variable:
export HASURA_GRAPHQL_DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
It doesn't make much sense to me, as I've cleanly supplied all parameters on the command-line:
/usr/local/bin/graphql-engine --host localhost --port 5432 --user dbsync --password dbsync --dbname dbsync serve
I will report back, once I made further progress.
✔️ Solved
After adding the HASURA_GRAPHQL_DATABASE_URL
environment variable to the entrypoint script of the hasura-graphql-engine
container, the cardano-graphql-background
service started working.