How to use authentication for django-prometheus when deployed to production?
Closed this issue · 7 comments
Every path in our urls go through a LoginMiddleware that requires login authentication and redirects the user to /admin/login if not authenticated. The exceptions are the API enpoints which use a token authentication.
This project is deployed in Kubernetes and when the requests are made to /metrics they are redirected to the login page. Since we don't want to make the /metrics without authentication how can I use an authentication method for django-prometheus in production?
Heya - so here's how we solved it:
Add a little custom middleware that looks like this: https://stackoverflow.com/a/49037059 instread of the the LoginMiddleware.
This is what sets all the URLs in your site to require authentication.
Then add /metrics to the OPEN_URLS list in your settings file.
@eeaston but this will allow any unauthenticated user to still have access to /metrics. I would like to have some sort of authentication to /metrics and provide the credentials in the prometheus deployment
Oh I see, so different auth for the metrics endpoint - sorry I mis-read. In that case, make a new ModelBackend and add it to the first of the back-ends in the settings list at AUTHENTICATION_BACKENDS.
Check the request argument in the authenticate method and only allow auth if the url is /metrics, otherwise return None and it will fall through to the the default auth back-end.
See https://docs.djangoproject.com/en/4.0/topics/auth/customizing/#writing-an-authentication-backend
Ok, I think I see where you're getting at but I've never worked with custom backends. So, because I have a middleware that intercepts all requests if an unauthenticated user tries to fetch http://localhost/metrics, the middleware will make a redirect to http://localhost:8000/admin/login/?next=/metrics.
Then I would write something like this?? (except for the hardcoded password and assuming the user exists):
from django.contrib.auth.backends import BaseBackend
from django.contrib.auth.hashers import check_password
from django.contrib.auth.models import User
class MetricsAuthBackend(BaseBackend):
def authenticate(self, request, username=None, password=None):
"""
Authenticate a user based on the username and password.
"""
if '/metrics' in request.path:
if username == "username":
pwd_valid = check_password(password, "somepassword")
if pwd_valid:
user = User.objects.get(username=username)
return user
return NoneBecause of the middleware, the actual path of the request is /admin/login and doesn't contain the /metrics.
Found a final solution. I added a custom handle for the /metrics in my middleware's process_request() method like this:
def process_request(self, request):
path = request.path_info.lstrip('/')
# Handle '/metrics' endpoint to use a separate authentication backend
# rather than JWT token or admin login
if 'metrics' in path:
if 'Authorization' in request.headers:
auth = request.headers['Authorization']
if auth.startswith('Basic '):
auth = auth.split('Basic ')[1]
auth = base64.b64decode(auth).decode('utf-8')
username, password = auth.split(':')
user = authenticate(request=request, username=username, password=password)
if user:
return self.get_response(request)
return redirect(settings.LOGIN_URL)This will force the request to /metrics to authenticate so the first method attempted will be my custom authentication backend, that needs to be placed as first on the list in settings.py.
The final backends.py looks like this:
from django.conf import settings
from django.contrib.auth.backends import BaseBackend
from django.contrib.auth.hashers import check_password
from django.contrib.auth.models import User
class MetricsAuthBackend(BaseBackend):
"""
Custom authentication backend for the /metrics endpoint only.
This backend will allow prometheus to authenticate with a
username and password in the headers rather than with a
JWT token.
"""
def authenticate(self, request, username=None, password=None):
"""
Authenticate a user based on the username and password.
"""
if '/metrics' in request.path:
if username == settings.PROMETHEUS_USERNAME:
try:
user = User.objects.get(username=username)
if user:
pwd_valid = check_password(password, user.password)
if pwd_valid:
return user
raise Exception('Invalid password')
except User.DoesNotExist:
return None
return None
def get_user(self, user_id):
try:
return User.objects.get(pk=user_id)
except User.DoesNotExist:
return NoneI also came across this issue, looked at various options and in the end my solution is this:
In the ingress, instead of allowing /metrics, I forward it to a "blackhole" nginx service with nothing in it
so /metrics is a 404 via ingress
But prometheus hits the pod directly (and not via ingress), so can parse the metrics correctly
I don't want the world to know I have /metrics, and also wanted to avoid all the extra setup of passwords and so on
My ingress looks like so
...
...
- path: /metrics
pathType: Prefix
backend:
service:
name: nginx-blackhole-service
port:
number: 80
...
...
Where the blackhole service is a basic service with nginx and nothing else (ie: /metrics would be a 404, since it has nothing)
To secure access to a /metrics of our Django application, we build basic authentication middleware. We configured this middleware to verify the authentication information provided by the user before allowing them access to the route. By using this method, we have ensured that only people with the correct credentials can access the page, whether through a web browser or through Prometheus requests.
the middlewire file :
the app setting configuration :
the prometheus config :
`scrape_configs:
- job_name: 'django_metrics'
static_configs:- targets: ['votre_domaine.com:port/metrics']
basic_auth:
username: 'votre_utilisateur'
password: 'votre_mot_de_passe'
`
the connection :
- targets: ['votre_domaine.com:port/metrics']
