Perma API may accept URL values that are too long for the database to save
sentry-io opened this issue · 6 comments
It looks like somehow someone tried to submit a capture for a blob of HTML rather than a URL, and then the server errored on the violation of the database constraint. I think I'd have expected the API to be aware of the max char length (or even to have expected the value to be a valid URL) and rejected the request with a 400 rather than a 500.
See the Sentry issue for more details on the request/response.
Sentry Issue: PERMA-1F
StringDataRightTruncation: value too long for type character varying(2100)
File "django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
DataError: value too long for type character varying(2100)
(19 additional frame(s) were not displayed)
...
File "django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
similar #3014
Okay I am finally looking into this, since we are changing how URL validation works.
Experimentally, I changed the serializer's validation max chars to 5, just to see. And... it worked
We have it set to max_length=2100. So.... what's going on here?
As expected the string 'a'*2100 fails validation, though I note not re: length
'a'*2101 though, indeed 500s with this exact stacktrace
Oh. It's because we save invalid URLs in CaptureJob. Doh!
I should have read higher in the trace:
File "/perma/perma_web/api/views.py", line 418, in post
capture_job.save()
et voilà
Fixed in #3462