octue/octue-sdk-python

Stream service events into BigQuery

cortadocodes opened this issue · 0 comments

Feature request

IMPORTANT: We need to test this before merging the breaking services communication change in case we need to change the format of event attributes (see #603).

Use Case

This will allow advanced querying of events at a lower storage cost and database server cost (as fewer concurrent connections to the database server will be needed e.g. by django-twined's service usage events.

Current state

Events are stored by django-twined in a postgresql database and the database server has to be upgraded to cope with many questions being asked at once.

More info