Tool to collect performance reports in JSON format from TeamCity, insert into ClickHouse and visualize.
clickhouse-backup
- backup clickhouse data.clickhouse-restore
- restore clickhouse data.report-aggregator
- stats server, provide access to ClickHouse via HTTP API tailored for analytics (inspired by Cube.js query format).tc-collector
- collect performance reports from TeamCity artifacts and insert to ClickHouse.transform
- transform existing data into another form. Raw JSON report is preserved as is, but for performance reasons maybe needed to pre-analyze and extract data into separate columns during collecting. And as data requirements changes, re-analyze the whole data set maybe required.
Directory dashboard
contains Vue.js application built using Element Plus Desktop UI Library.
pnpm i
to install dependencies. pnpm is recommended, do not use Yarn or NPM.- Run
serve
Run Configuration (don't forget to restart in case you have new data in Database) pnpm run dev
to start a dev server with hot module replacement.- Change Server to
http://localhost:9044
To change dashboard, edit your dashboard page *Dashboard.vue
, for example IntelliJDashboard.vue
or GenericMetricDashboard.vue
.
LineChartCard
or BarChartCard
supports measures
property.
Specify desired metric. Multiple metrics are supported, but keep in mind that each metric means chart series and overuse can make chart unreadable.
If metric is extracted from report to field, just use it's field name. Otherwise use:
activityCategory.activityName
to get duration valueactivityCategory.activityName.s
to get start value.activityCategory.activityName.e
to get end value.
For example, if activity launch terminal
reported under category prepareAppInitActivities
, use prepareAppInitActivities.launch terminal
as metric name. Or prepareAppInitActivities.first render.s
to get start value of first render
.
cd ~/Documents/report-aggregator
clickhouse client -h 127.0.0.1
to use clickhouse client to perform SQL queries.
- Set env
DB_NAME
to desired database name:export DB_NAME=
- Create directory for your database in
db-schema
and copyreport.sql
from another database. - Execute SQL:
clickhouse client -h 127.0.0.1 --query="create database $DB_NAME" clickhouse client -h 127.0.0.1 -d $DB_NAME --multiquery < ./db-schema/common/installer.sql clickhouse client -h 127.0.0.1 -d $DB_NAME --multiquery < ./db-schema/common/collector_state.sql clickhouse client -h 127.0.0.1 -d $DB_NAME --multiquery < "./db-schema/$DB_NAME/report.sql"