Implement new collector class that will give pod-container logs

[sophsoph321]

The container logs collector class got removed from the periscope repo in this PR: #97
For arc requirements, we need container logs for the azure-arc namespace as long as the collector is configurable to any namespace, we should be fine.
Implementation details of this new collector class can be worked out in this thread or offline.

[Tatsinnit]

Thank you @sophsoph321 🙏

More context, the docker log was removed once Shweta and Juxian kindly pointed out that files were always empty, because of this: Docker deprecation here : https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ and here https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar

This Issue is the base workitem to implemented new containerd logs collector.

• Thanks @sophsoph321 for opening this, can we discuss the logs type and use of this client please to grab necessary idea from: https://github.com/containerd/containerd#features - Ping to: @arnaud-tincelin for more ideas and any implementation level suggestions in mind. cc: @rzhang628, @qpetraroia + @songjiaxun

• Essentially what logs we will need, etc, feel free to add an internal chat for this as well.

Thanks heaps,

[arnaud-tincelin]

Do you need pod logs or container logs? If you need container logs, would fluentd fit your needs?

[sophsoph321]

As of now, we need container logs. Both fluentd and containerd look like potential options for implementation and we can weigh the pros and cons before deciding which library to use.

Here is an example command we are going for when it comes to generating the logs:
kubectl logs -n azure-arc clusteridentityoperator-578c88fb78-bvqtz -c manager

We can get the containers in the following directory path: var/log/container

[Tatsinnit]

Cool, Thanks for Clarification @sophsoph321, so this means we need Pod-Container logs which makes sense:

As a help for you, I could land up a feature branch for you with client-go implementation of fetch of pods-container-logs and other pod level information, you can improve on it. (If this helps)

• Here is a sample implementation and we can do that using client-go https://github.com/kubernetes/kubernetes/blob/c2e90cd1549dff87db7941544ce15f4c8ad0ba4c/pkg/kubectl/cmd/log.go#L141

[sophsoph321]

Cool, Thanks for Clarification @sophsoph321, so this means we need Pod-Container logs which makes sense:

As a help for you, I could land up a feature branch for you with client-go implementation of fetch of pods-container-logs and other pod level information, you can improve on it. (If this helps)

• Here is a sample implementation and we can do that using client-go https://github.com/kubernetes/kubernetes/blob/c2e90cd1549dff87db7941544ce15f4c8ad0ba4c/pkg/kubectl/cmd/log.go#L141

+1 to this. We can use the sample implementation outlined in the link as a baseline for how to retrieve pod-container logs.