jamessewell/django-timescaledb

Roadmap Proposal

Opened this issue · 3 comments

daadu commented

Last Updated - 17 Aug 2021

django-timescaledb

django-timescaledb aims to achieve following objectives

  • feature rich integration of TimescaleDB with Django
  • high-quality module - proper testing and documentation
  • keeping the module active with future releases of TimescaleDB and Django
  • clearly communicate - version supports for Django and TimescaleDB - along with LTS, deprecation/upgrade notes

Improvements/Feature Requests

  • #22 Why not allow making "partition key" as primary key? [M.1]
  • Why does the partition key have to be named "time"? [M.1]
  • data retention policy [M.2]
  • Compression policies [M.2]
  • #12 Continuously aggregated views [M.2]
  • additional partitioning key [M.2]

Milestone 1 [Basic usage]

  • Create hypertable (for empty or non-empty tables) with single "time" based partition
  • proper model.Manager that can be used to run queries
  • proper testing framework
  • proper documentation
  • all features in this step should work seamlessly with Django's makemigrations and migrate commands, without user having to run single query himself
  • APIs
    • DB backends (postgres and postgis) - TimescaleSchemaEditor
    • models.Model - TimescaleModel (abstract)
    • models.Manager - TimescaleManager (time_bucket, time_bucket_gapfill, histogram)
    • models.Field - TimescaleDateTimeField
    • models.Func - Interval, TimeBucket, TimeBucketGapFill
    • models.Aggregate - Histogram, Last, First

Milestone 2 [Advance features with manual management]

  • have migrations operations for adding/removing/altering advanced features of TSDB, this has to be dropped in by the user in the migrations files manually
  • support for "AggregateViews" - not sure how to implement it?
  • additional partitioning key - not sure how to implement it?
  • additional functions and/or queryset methods - "Drop chunks", "Percentile approximation", "Time-weighted average functions"
  • new APIs
    • data retention policy - AddRetentionPolicy, AlterRetentionPolicy, RemoveRetentionPolicy
    • compression - AlterTableCompression, AddCompressionPolicy, RemoveCompressionPolicy

Milestone 3 [Advance features with automatic management]

  • Have a parallel "migration system" like Django's
    • Model.TSMeta class - since Model.Meta class in Django is restrictive (cannot add new attributes, etc), need to have a parallel TSMeta class for storing "TSDB related meta" info (policies, etc) about the Model
    • makemigrations_ts command - to generate "migrations" files with "TSDB" migration operations
    • maketsmeta command - that goes through all migrations and outputs all TSMeta class based on "migration operations" added manually - tooling to migrate from M.2 -> M.3
daadu commented

@schlunsen I have been busy with other projects recently.

Now that I am back! I have prepared this. Please give your feedback on it.

This could act as a central issue for planning/discussing new APIs etc.

This document will be updated as we go along.

daadu commented

@schlunsen Please also tag other active contributors for their views/feedback on it.

fabienheureux commented

Hi @daadu, would you say this project is ready for use on production ? Or is it still in a too early stage ?
Thanks !

  • 👀1