apache returns 502 (bad gateway) instead of 403 unauthorized
Here's the scenario: a client attempts to upload an event (POST /api/events
) without a proper cert or valid auth. Gunicorn properly returns a 403 Unauthorized
error, but when it gets sent back to apache and then to the client, it gets turned into a 502 Bad Gateway
error. For instance, this happened on CIT early this morning. Here's an example of the gracedb log line with the 502 and the two lines before it.
Jul 18 09:40:12 : DJANGO | 2023-07-18 09:40:12.610 | 9e6074462859 | 10.0.1.42 | performance | INFO | middleware.py, line 58 | create: 403:
Jul 18 09:40:12 : GUNICORN | 131.215.113.168 - - [18/Jul/2023:09:40:12 +0000] "POST /api/events/ HTTP/1.1" 403 58 "-" "gracedb-client/2.10.0"
Jul 18 09:40:12 : APACHE | 10.0.1.35 - - [18/Jul/2023:09:40:12 +0000] "POST /api/events/ HTTP/1.1" 502 315 "-" "gracedb-client/2.10.0"
The DJANGO
performance middleware recognizes it as a 403
, GUNICORN
says it's a 403
, APACHE
says 502
.
What's going to happen in this scenario is, a user will see 502
in their error logs, when the issue isn't with GraceDB, per se, but rather it's returning a catch-all error instead of the proper 403
. Manual intervention by looking in the gracedb error logs is required to get the user the correct information.
I think i remember seeing this before, and the issue was a parameter in apache that controlled the maximum size of a unauthorized request... and since POST
requests to create new events are ~O(1Mb), they exceed this value and so when gunicorn says the request is unauthorized, apache will return the too large bad gateway error.
This isn't a showstopper, but more of a fix for dev sanity. As in, the complaint will be "we got another 502 when creating an event, gracedb is broken", but the real issue was the user wasn't authorized.