Clean up the MappedMetricData data schema
Closed this issue · 2 comments
williewheeler commented
There are several issues:
- The metric definition appears twice
- Detector UUID appears twice.
- Detector UUID appears as detectorUuid one time and detectorUUID the other time.
- The schema includes the
investigationResults
field, which we no longer use in AA - We reference prediction/threshold related fields that don't make sense for many of the detectors
Whoever takes this issue, let's discuss more carefully what the target data schema should be.
Expedia-specific: We will need to roll this out in a way that minimizes client impact. Some potential coordination areas:
- Aquila is posting anomalies, so we will need to update it
- Seyren will be posting anomalies, so we will need to update that
- Vector is reading anomalies
- Armor is reading anomalies
- I don't think Haystack is reading anomalies directly, but double-check
Here's an example:
{
"metricData": {
"metricDefinition": {
"tags": {
"kv": {
"mtype": "gauge",
"unit": "metric",
"what": "[redacted]",
"m_application": "[redacted]",
"org_id": "1",
"name": "[redacted]",
"interval": "60",
"m_account": "[redacted]",
"m_region": "us-west-2"
},
"v": []
},
"meta": {
"kv": {},
"v": []
},
"key": "[redacted]"
},
"value": 0,
"timestamp": 1542295565
},
"detectorUuid": "37ef6be4-dd86-11e8-956a-061bc4b8b7e4",
"detectorType": "constant-detector",
"anomalyResult": {
"detectorUUID": "37ef6be4-dd86-11e8-956a-061bc4b8b7e4",
"metricData": {
"metricDefinition": {
"tags": {
"kv": {
"mtype": "gauge",
"unit": "metric",
"what": "[redacted]",
"m_application": "[redacted]",
"org_id": "1",
"name": "[redacted]",
"interval": "60",
"m_account": "[redacted]",
"m_region": "us-west-2"
},
"v": []
},
"meta": {
"kv": {},
"v": []
},
"key": "[redacted]"
},
"value": 0,
"timestamp": 1542295565
},
"anomalyLevel": "NORMAL",
"predicted": null,
"thresholds": null,
"investigationResults": null
}
}
Also, this issue gives us a good opportunity to document an approach to schema migration.
Updates
- Probably
MappedMetricData
shouldn't have anAnomalyResult
at all. - Probably we should push
AnomalyResult
onto theanomaly
topic instead of pushing aMappedMetricData
onto that topic.
williewheeler commented
I removed the investigationResults
field.
williewheeler commented
Done: 0472915