Question : BroadSoft Dashboards and Discovery toolkit - Data stablization period

Question

Question : BroadSoft Dashboards and Discovery toolkit - Data stablization period

Opened this issue 5 years ago · 4 comments

We have currently deployed the BroadSoft Dashboards and Discovery toolkit using the Radius to ES for Tier3 lookup for problem resolution.

Working on a project that require near time CDR inspection I use the "timestamp" property to pull CDRs that arrive in a given range (gte and lte). I see results that that are troubling

timestamp property is ms epoch of 5 hours + UTC. Yet rest of the data like starttime, answertime etc are UTC
GETs totals from ES for the same range asked repeated every second shows gradual increase and takes about 10 minutes to 15 minute stablize. Our project needs to be within 1 minute of the event.
I am hoping that there is a reason and possible resolution or worksround to this.

Thanks

BG

Answer 1 · 2019-11-12T19:49:36.000Z

The timestamp element in the document is actually the starttime of the CDR. I'm surprised that it it showing different timezone than the starttime element - it perhaps is because in the index template the timestamp is typed as "keyword" instead of a date type - which is probably wrong. I'll try and look at that this week.

There was no great reason we chose the starttime for the timestamp.. Based on your comment, I believe you'd want to search / fetch based on the releasetime.

Answer 2 · 2019-11-12T20:17:08.000Z

Thanks on the timestamp. What about the issue with there being increased hit counts as a request the same timestamp every second over 12-15 minutes and it finally remains teh same. e.g Second 1 I get 17, second 20 212, seconh 700-900 I get 355. and ffinally it remaons 355. Could this be because of the the timestamp is typed as "keyword" instead of a date type? H On Tuesday, November 12, 2019, 02:49:41 PM EST, David Stewart <notifications@github.com> wrote: The timestamp element in the document is actually the starttime of the CDR. I'm surprised that it it showing different timezone than the starttime element - it perhaps is because in the index template the timestamp is typed as "keyword" instead of a date type - which is probably wrong. I'll try and look at that this week. There was no great reason we chose the starttime for the timestamp.. Based on your comment, I believe you'd want to search / fetch based on the releasetime. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Answer 3 · 2019-11-12T20:36:19.000Z

David,

I finally figured out my issue and wanted to see if there was another element in the document that could help me.

My app that requires to get ALL records inserted into ES over 1 second in time, queries ES for a given timespan second, 20 seconds before to be exact assuming the timestamp element is the "insert" datetime. If that is not the case and timestamp is literally the epoch of starttime then ofcourse additional Alive and Stop records will come in later with the same starttime and timestamp.

In order to get all records from ES I will have to glean 3 groups Start = stasrtime, Alive = Answertime and Stop = Releasetime for that second. That way I pull in all teh records

Is that the only way I could get all record that came in for that given second?

BG

Answer 4 · 2019-11-12T21:45:43.000Z

It would be ideal to have a "inserttimestamp". That way we just pickup cdrs as they come in or have a stamp as to when we got the record.