open-services-group/metrics

Spike: Select and prioritize Github repos and orgs we are interested in monitoring

Closed this issue ยท 10 comments

The aim of this issue is to select Github repositories and Orgs we want to track

Questions to answer:

  1. Which repository and org can be used as an example for setting up the metric processing pipeline?
  2. Assuming the pipeline is set, which repositories and Orgs should we replicate the process for?

/assign @schwesig

@hemajv can you add some more detail to this issue? What is meant by community entities ? Is this anything other than our repos?

If I understand it right, here my 5cents.
I think, widening the metrics to out-of-OSG communities can be nice, and we should not limit the general architecture to just OSG, but for our first MVP in Q1 we should focus on OSG repos.

@hemajv if opening is the plan, what timeframe do you have in mind?

@MichaelClifford added some description and removed the ambiguous term community entities from the issue.

@schwesig Opening up to other communities seems like a great idea. We can focus on repos by our team in this issue and include other communities in separate issues possibly in subsequent quarters.

Based on our last meeting, it seems BYON repo and the OSG org can be used as a first example. Then we can move on to repos and orgs for data science SIG, community experience SIG, and services SIG or in other words, all the repos and orgs in sigs.yaml
@schwesig, @MichaelClifford, @hemajv if that seems like a good step forward, we can close this issue as both the questions are answered. If you have other ideas about what should be the first example or you think some repos outside the sigs.yaml need to be immediately tracked, we can discuss that here.

@Shreyanand yes I agree ๐Ÿ‘ For Q1, we can focus on implementing the pipeline taking the BYON repo as an example.

@schwesig Once we have a well defined pipeline, we can start extending and reusing it for other orgs/repos and create separate issues to track them.

@Shreyanand agreed, and good change of the title, more focussed, excellent.
my question before we close this issue:
BYON +1
but I don't think that is enough. you said also OSG Org --> what other repo would be needed to also measure the other performances from the OKR,? community? only BYON is imho not enough for Q1. I think at least community to measure the "scale" and "communication part?
what do you all think? Or am I on the wrong track maybe/misunderstood sth?

@schwesig That's a good question. BYON would be repo 1 or the repo that we use to build the pipeline. Then we can replicate it for other repos in the org Open Services Group, i.e., metrics, community, and devsecops. So, the OSG org would be the first example of org wide metrics tracking. The reason for mentioning an org separate from a repo is that in our calculation of metrics, we can aggregate and report metrics for the complete org and answers questions like how many OKRs is this org targeting? What %of them were completely achieved? and so on. All of this should be in scope for Q1.
Thats per my understanding, I could be missing something here.

+1 to all of the above,
But I want to clarify, it is fine to start with BYON notebooks and OSG org, but to achieve our OKR goal, we need to be pulling data and providing metrics for all the repositories mentioned in the sigs.yaml. This extends beyond the OSG org.

I would prioritize getting fewer metrics (1 or 2) from all the repo's as opposed to a complete set of metrics from 1 or 2 repos right now.

Yes agreed! We can start collecting data for all the repos mentioned in sigs.yaml and then start looking into the different metrics mentioned in #3

@MichaelClifford agreed. start with 1 or 2 repos. --> start with metrics on 1 or 2 metrics (even if we can collect data for all repos @hemajv) but be sure to have at least a few metrics (2+) that cover an okr.

To conclude the spike, we are focusing on metrics from the OSG org, and the repositories in the org mentioned in sigs.yaml for the first iteration. Once we have the pipeline in place, we can replicate it for the Operate First and OS Climate Orgs and their repositories.