GreptimeTeam/greptimedb

Tracking issue for the MetricEngine

waynexia opened this issue · 2 comments

waynexia commented

What type of enhancement is this?

Performance

What does the enhancement do?

RFC metric-engine proposes a new region engine Metric. Here is the tracking issue for implementing it.

Related progress is also tracked in GitHub Project https://github.com/orgs/GreptimeTeam/projects/34

  • RFC document #1925
  • Meta Server
    • Reserve the first 8 bits of region number for region id group #2100
    • Add logical route entry to table route
    • Add special logic to create progress when table engine is metric
    • New alter procedure that acquires the physical table's resource lock before operating the logical table #3098
    • Support grouping CREATE requests
    • Translate table name of physical table to its table id when handle CREATE request of logical table
    • Support alter to logical table
  • Frontend
    • Also support automatically creating the physical table under public schema.
    • Convert requests from prometheus remote write into one request to the physical table
    • Generate DDL for logical table from incoming prom remote write requests and submit to metasrv (for updating metadata of logical table)
    • Separate GreptimeDatabase service into two different services
  • Datanode
    • Define RegionEngine and RegionServer #2160
    • Define MetricEngine on top of MitoEngine
    • Define MetadataRegion to handle operations to metadata #2687
    • DataRegion to handle operations to data #2706
    • Handle Create request #2694
    • Handle Write request #2759
    • Handle Read request #2793
    • Handle Alter request #2726
    • Forward INSERT/SELECT to physical table to mito engine (is implemented together with corresponding logical operation)
    • Be able to recognize physical table (by their table id?)
    • Implement failover/heartbeat/... for MetricEngine
    • Change a platform-independent hash algorithm for TSID
    • Maintains the schema of physical table in metasrv

Implementation challenges

No response

tisonkun commented

Is the TODO list above updated? Perhaps we can create subtasks issue for each task so that we can keep the status synced.

Take https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/about-task-lists as the reference.

killme2008 commented

From now on, we need to keep compatibility with older versions as possible as we can.