Simple tool in Python to show Snowflake task graphs in static HTML files with generated SVG, and Gantt charts for task graph runs.
Create a profiles_db.conf copy of the profiles_db_template.conf file, and customize it with your own Snowflake connection parameters. Your top [default] profile is the active profile, considered by our tool. Below you may define other personal profiles, that you may override under [default] each time you want to change your active connection.
We connect to Snowflake with the Snowflake Connector for Python. We have code for (a) password-based connection, (b) connecting with a Key Pair, and (c) connecting with SSO. For password-based connection, save your password in a SNOWFLAKE_PASSWORD local environment variable. Never add the password or any other sensitive information to your code or to profile files. All names must be case sensitive, with no quotes.
The database and schema are also mandatory: we always query all tasks from a database schema!
To compile into a CLI executable:
pip install pyinstaller
pyinstaller --onefile task-graph-viewer.py
dist\task-graph-viewer
Connect with no parameters, to show all root task names in the current database schema. We also generate a DOT graph for all task graphs in the current database schema. The name of the generated HTML file - in the output subfolder - with the previous graph embeded as SVG. We show not just task names and their dependents, but inner information, such as status, id, warehouse etc. Example:
python task-graph-viewer.py
Here is an example with two task graphs (T21 and T51 are root task names that you can use later):
Use the --simple option to render the tasks without properties:
python task-graph-viewer.py --simple
Connect with a root task name (case-sensitive, that you can get from the previous call), to show the different task graph runs you had for this task graph in the past few days. We also generate a DOT graph in a HTML file for this specific task graph only. Example:
python task-graph-viewer.py T51
Here is the same graph from Snowflake:
Use the --vertical option to render the graphs top-bottom:
python task-graph-viewer.py T51 --vertical
Connect with a root task name (case-sensitive!), and a run ID for this task (that you can get from the previous call). We'll generate a Gantt Chart (with the free Google Charts JS library) for the execution of all related tasks. Example:
python task-graph-viewer.py T1 1680125626365
Use the --simple option to render the chart with no scheduled time:
python task-graph-viewer.py T1 1680125626365 --simple
Connect as before - with a root task name (case-sensitive!), and a run ID for this task (that you can get from the previous call). But add a --monitor option. We'll generate a Gantt Chart as before, but that will automatically refresh on screen every 3 seconds. The Python code will also refresh its metadata for this run every 3 seconds. Exit with CTRL-C, or wait until all tasks enter the SUCCEEDED state for this run. Example:
python task-graph-viewer.py T1 1680125626365 --monitor
Here is the intermediate view of a task graph which is still running. Compared to the last image here, it will gradually add new executed tasks until completed.
Last scheduled task here started to run, but it is not yet completed and we don't know when it will complete. So we simply show a gray bar, for the duration between scheduling and starting steps:
