Collecting different metrics from microservices
dimoibiehg opened this issue · 9 comments
Is your feature request related to a problem? Please describe.
Suppose that I am working on the microservices benchmark, called DeathStarBench.
It seems reasonable to have different performance values like CPU usage, Memory usage, and Latency for each microservice in metric logs. However, it seems that the only reported metric is the total latency of the whole application in a specified workload.
Describe the solution you'd like
Having in detail more metrics like CPU usage, Memory usage, and Latency of each microservice besides the total value of the latency can be very helpful in researching on microservices.
Describe alternatives you've considered
Otherwise, I am wondering about the advantage of using VisrtualClient over directly using the application in DeathStarBench.
Additional context
By the way, I am not sure whether what I am saying can be doable in the current version of Virtual Client or not.
Hi @dimoibiehg, what you are describing could be acomplished by our default monitor profile
https://github.com/microsoft/VirtualClient/blob/main/src/VirtualClient/VirtualClient.Main/profiles/MONITORS-DEFAULT.json
In your virtualclient commendline, that profile is included by default, unless you overwrite it with another monitor profile. The data should be in your /logs/metrics.csv next to the virtualclient executable. Please let me know if that is suffice
https://microsoft.github.io/VirtualClient/docs/monitors/0001-atop/
Hi @dimoibiehg, what you are describing could be acomplished by our default monitor profile https://github.com/microsoft/VirtualClient/blob/main/src/VirtualClient/VirtualClient.Main/profiles/MONITORS-DEFAULT.json In your virtualclient commendline, that profile is included by default, unless you overwrite it with another monitor profile. The data should be in your /logs/metrics.csv next to the virtualclient executable. Please let me know if that is suffice
https://microsoft.github.io/VirtualClient/docs/monitors/0001-atop/
Hi @yangpanMS, thanks for mentioning the deafult moirtoring profile. However, when using that profile, I do not get any of those metrics in separation of microservices. For example, when running the default profile for DeathStarBench, I got the attached mterics file.
metrics.csv
What I expected was knowing each of those metrics for each microservice in the social network application. For example, CPU usage, Memory usage, and Latency for Text, User, Media Frontend, Compose Post, Post Storage, Redises, Memcacheds, and so on (see the following microservices in the application, sourced by DeathStarBench Github respository).
Thank you for the response. Sorry that we do not collect the usages for each microservices for this benchmark. For workloads like this, we rely on the workload controller output to get the data, and we parse whatever output that comes out of the workload itself, like this: https://github.com/microsoft/VirtualClient/blob/main/src/VirtualClient/VirtualClient.Actions.UnitTests/Examples/DeathStarBench/DeathStarBenchOutputExample.txt. Do you know if there is a way to include the counters you just mentioned? If so, we will have a way to parse them out.
Thanks for your following up. One way that came to my mind was adding more logs to measure extra metrics for each microservice, probably using Jaeger.
I'm sorry again that our team doesn't not have expertise into this domain. This is open-source software, please feel free to modify and test it out. Meanwhile I'd just like to let you know we are still debugging why the deathstarbench only works for socialnetwork scenarios and not others.
I'm sorry again that our team doesn't not have expertise into this domain. This is open-source software, please feel free to modify and test it out. Meanwhile I'd just like to let you know we are still debugging why the deathstarbench only works for socialnetwork scenarios and not others.
No worries @yangpanMS. Thanks for fixing the issue.
One more thing I have just found. The details of CPU and memory utilization, which are reported according to the default monitoring profile, are not reported by the original workload of DeathStarBench, i.e., the file that you have linked. So, it seems that extra measurements are in the flow of the data collection other than those achieved by parsing the original output of the application. Right?
Did you mean the atop measurements? They should be reported but might be renamed. We are trying to make the name of the perf counters look the same across windows/linux. https://github.com/microsoft/VirtualClient/blob/main/src/VirtualClient/VirtualClient.Monitors/AtopParser.cs
Is there anything in atop that you are looking at?
BTW I noticed there is a timezone difference between us. Let me know if you'd like to schedule a brief call to help you in our mutal wake time.
Hi @dimoibiehg , do you have more questions about this issue? May we close this issue?
Thanks @yangpanMS. In sum, this feature is not directly related to VirtualClient. The primary metrics should have been measured in DeathStarBench and exposed to the output.