request.data or request.json when accessed in middleware fails 400 bad format

Question

request.data or request.json when accessed in middleware fails 400 bad format

Nivraj opened this issue 2 days ago · 3 comments

Bug description

Hello guys,

Im trying add session validator and it fails when i do call request.data or request.json to validate payload requested in post call. this is in middleware and I wanted to validate each request coming to charts api could any one guide me here ? It fails with 400 bad format issue and in dashboard all charts fails

import os
from typing import Dict, Any, Callable
from flask import Flask, Request, Response
class SessionValidator:
    """
    Middleware to validate sessions.
    """
    logger = logging.getLogger(__name__)
    def __init__(self, app: Flask, excluded_paths=None):
        self.app = app
    def __call__(
            self, environ: Dict[str, Any], start_response: Callable[..., Any]
    ) -> Any:
        request = Request(environ)
        
        if "chart" in request.path:
            self.print_request_info(request)
            return self.app(environ, start_response)
        return self.app(environ, start_response)
    def print_request_info(self, request: Request):
        """Print key details from the request."""
        self.logger.debug(f"Request Method: {request.method}")
        self.logger.debug(f"Request Path: {request.path}")
        self.logger.debug(f"Request Headers: {dict(request.headers)}")
        self.logger.debug(f"Request Args: {request.args}")
        self.logger.debug(f"Request Form: {request.form}")
        self.logger.debug(f"Request Cookies: {request.cookies}")
        self.logger.debug(f"Request Data: {request.data}")  ## fails with 400 bad request and dashboard never loads
        self.logger.debug(f"Request JSON: {request.json}")  ## fails with 400 bad request and dashboard never loads
# Example of adding middleware
ADDITIONAL_MIDDLEWARE = [SessionValidator,]```
 

### Screenshots/recordings

i does give me data but fails after that with exception 400 bad gateway

```superset  | 2024-12-20 06:38:19,561:INFO:superset_config:Request Data: b'{"datasource":{"id":2,"type":"table"},"force":false,"queries":[{"filters":[{"col":"updatedat","op":"TEMPORAL_RANGE","val":"No filter"}],"extras":{"having":"","where":""},"applied_time_extras":{},"columns":[],"metrics":["max_time"],"annotation_layers":[],"series_limit":0,"order_desc":true,"url_params":{"depositKey":"S21ISFpDWThDNXVvLnVzMDQ=","native_filters_key":"gTA7HK8g-QLIE3XGU0ZGZY7FnCy78d8XAuOCWMJ1aE37oGcl-B7kAYnfUYHHfRUj","userTimeZone":"Asia/Kolkata"},"custom_params":{},"custom_form_data":{}}],"form_data":{"datasource":"2__table","viz_type":"big_number_total","slice_id":1,"url_params":{"depositKey":"S21ISFpDWThDNXVvLnVzMDQ=","userTimeZone":"Asia/Kolkata"},"metric":"max_time","adhoc_filters":[{"clause":"WHERE","comparator":"No filter","expressionType":"SIMPLE","operator":"TEMPORAL_RANGE","subject":"updatedat"}],"header_font_size":0.6,"subheader_font_size":0.4,"y_axis_format":"SMART_NUMBER","time_format":"%Y-%m-%d %H:%M:%S","force_timestamp_formatting":true,"annotation_layers":[],"dashboards":[1],"extra_form_data":{},"label_colors":{"Active":"#3778FF","Inactive":"#05194E","Never Logged In":"#FF2300","Suspended":"#FFD557","Logins":"#008064"},"shared_label_colors":{},"color_scheme":"supersetColors","extra_filters":[],"dashboardId":1,"force":false,"result_format":"json","result_type":"full"},"result_format":"json","result_type":"full"}'

superset  | 2024-12-20 06:38:19,572:WARNING:superset.views.error_handling:HTTPException
superset  | Traceback (most recent call last):
superset  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/request.py", line 608, in get_json
superset  |     rv = self.json_module.loads(data)
superset  |   File "/usr/local/lib/python3.10/site-packages/flask/json/provider.py", line 188, in loads
superset  |     return json.loads(s, **kwargs)
superset  |   File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
superset  |     return _default_decoder.decode(s)
superset  |   File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
superset  |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
superset  |   File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
superset  |     raise JSONDecodeError("Expecting value", s, err.value) from None
superset  | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
superset  | 
superset  | During handling of the above exception, another exception occurred:
superset  | 
superset  | Traceback (most recent call last):
superset  |   File "/usr/local/lib/python3.10/site-packages/flask/wrappers.py", line 130, in on_json_loading_failed
superset  |     return super().on_json_loading_failed(e)
superset  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/request.py", line 642, in on_json_loading_failed
superset  |     raise BadRequest(f"Failed to decode JSON object: {e}")
superset  | werkzeug.exceptions.BadRequest: 400 Bad Request: Failed to decode JSON object: Expecting value: line 1 column 1 (char 0)
superset  | 
superset  | The above exception was the direct cause of the following exception:
superset  | 
superset  | Traceback (most recent call last):
superset  |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
superset  |     rv = self.dispatch_request()
superset  |   File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
superset  |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
superset  |   File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
superset  |     return f(self, *args, **kwargs)
superset  |   File "/app/superset/views/base_api.py", line 119, in wraps
superset  |     duration, response = time_function(f, self, *args, **kwargs)
superset  |   File "/app/superset/utils/core.py", line 1364, in time_function
superset  |     response = func(*args, **kwargs)
superset  |   File "/app/superset/utils/log.py", line 303, in wrapper
superset  |     value = f(*args, **kwargs)
superset  |   File "/app/superset/charts/data/api.py", line 228, in data
superset  |     json_body = request.json
superset  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/request.py", line 557, in json
superset  |     return self.get_json()
superset  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/request.py", line 617, in get_json
superset  |     rv = self.on_json_loading_failed(e)
superset  |   File "/usr/local/lib/python3.10/site-packages/flask/wrappers.py", line 135, in on_json_loading_failed
superset  |     raise BadRequest() from e
superset  | werkzeug.exceptions.BadRequest: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.```

### Superset version

4.1.1

### Python version

3.9

### Node version

16

### Browser

Chrome

### Additional context

i tried with 3.1.2 and 4.1.1 issue still persists

### Checklist

- [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Answer 1 · 2024-12-20T07:00:09.000Z

Hey, @Nivraj! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this together while we wait for a human maintainer.

The 400 Bad Request error you're encountering when accessing request.data or request.json in your middleware is likely due to issues with the JSON payload or request headers. Here are some common causes and solutions:

Empty or Malformed JSON: The error message indicates a JSONDecodeError, which suggests that the JSON payload might be empty or malformed. Ensure that the request body contains valid JSON data before accessing it.
Content-Type Header: Verify that the Content-Type header of the request is set to application/json. If this header is missing or incorrect, Flask may not parse the request body as JSON.
Request Body Format: Ensure that the JSON payload matches the expected structure for the endpoint being accessed. Each Superset endpoint has specific JSON schema requirements that must be met ^[1].
CSRF Token: If your application uses CSRF protection, ensure that the CSRF token is correctly included in the request headers. Missing or invalid CSRF tokens can lead to 400 errors ^[2].
Middleware Logic: In your SessionValidator middleware, consider checking if the request body is non-empty before attempting to access request.data or request.json. You can add a condition to handle cases where the body might be empty.

By addressing these potential issues, you should be able to resolve the 400 Bad Request error and ensure that your middleware functions correctly.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

Answer 2 · 2024-12-20T07:02:27.000Z

@dosu request body isn't empty i have tried with single chart and it does display me dict properly. right after displaying dict it fails with json decode error

Answer 3 · 2024-12-20T07:03:52.000Z

If the request body isn't empty and you're still encountering a JSON decode error, it might be due to how the request data is being accessed or modified. Here are a few things to consider:

Accessing request.data and request.json: Accessing request.data or request.json multiple times can lead to issues because the request stream can be consumed. Ensure that you're not accessing these properties in a way that consumes the stream before it's fully processed.
Middleware Modifications: Check if any middleware or other parts of your application are modifying the request body before it reaches your SessionValidator. Any changes to the request body could lead to parsing errors.
Logging and Debugging: Add logging to capture the exact state of the request body before and after accessing it. This can help identify if the data is being altered unexpectedly.
Error Handling: Implement error handling around the JSON parsing to catch and log specific errors, which might provide more insight into what's going wrong.

By ensuring that the request body is accessed correctly and not modified unexpectedly, you should be able to prevent the JSON decode error.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}