Add remote jikan URL and repsonse headers to repsonse objects

[purarue]

Exposing the Jikan URL and headers could be useful if someone wants to save the remote URL to re-request at a later time, or to examine if the context has been modified (by ETag) see here

>>> cowboy_bebop = requests.get("http://api.jikan.moe/v3/anime/1")
>>> requests.get("http://api.jikan.moe/v3/anime/1", headers={"If-None-Match": cowboy_bebop.headers["ETag"]})
<Response [304]>

As discussed on the discord, this would be done by adding the headers dict to the response from jikan, perhaps as 'jikan_url' and 'jikan_headers'

What would be the preferred way to do this? Should I add a function similar to _check_response after check_response, like _add_jikan_metadata, like:

response = _add_jikan_metadata(response)

and have it add the key/vals to the dict?

Let me know and I can work on this feature this week/weekend.

Additional feature for future consideration - to allow users to pass headers or a kwarg that returns None if a 304 is returned.

Something like:

>>> j = jikanpy.Jikan()
>>> cowboy_bebop = j.anime(id=1)
>>> is_modified = j.anime(id=1, check_etag=cowboy_bebop["headers"]["ETag"])
>>> is_modified
None

If the ETag has changed since the request was last issued, is_modified would be a regular jikan response.

Alternatively/in addition, we could let users pass headers which are used, to future proof for anything that may be added. This also allows users to pass an APP_KEY that doesnt match their APP_KEY defined in .env if self-hosting, to bypass cache and re parse the page

[irfan-dahir]

Upcoming Jikan API version (4.x) won't return Request cache/expiry time, etc in JSON as well.

If you check headers on current responses, it's already returning all that in there too. So adding header support be a good step indeed to future proof the wrapper.

Also regarding the APP_KEY, it's a key/value pair in the header. Where the key is auth and the value is APP_KEY.

e.g curl -H "auth:APP_KEY" https://api.selfhostedjikan.moe

[abhinavk99]

Adding a _add_jikan_metadata sounds great, but I think we should take this one step further and create a method along these lines:

def _wrap_response(response):
    _check_response(response)
    return _add_jikan_metadata(response.json())

Then call this method instead of _check_response in the public methods in jikan.py and aiojikan.py

[purarue]

Makes sense to me. I'll work on adding the the headers and jikan_url to the response first, then passing the headers to each request.

[abhinavk99]

After this gets added, we can add support to add headers in general. I don't think we should make a specific kwarg for checking etag but instead make users able to pass in a dict of headers as you mentioned.

I don't think that returning None on a 304 response is good because you lose some visibility, so I would just return the response object and the status (We could also add the response status along with the url and headers to the json response)

Edit: forgot that status is already part of response json

[abhinavk99]

Didn't see your comment above before my comment (love race conditions), and it would be great if you could make a PR for just getting headers before starting work on passing in headers.

[purarue]

Instead of check_etag, if were passing headers, we can let the user do that themselves and add something like this to the documentation:

>>> cowboy_bebop = j.anime(id=1)
>>> has_updated = j.anime(id=1, headers={"If-None-Match": cowboy_bebop["headers"]["ETag"]})
>>> has_updated.status_code
304
>>> has_updated.text
''

[purarue]

I'll work on the jikan_url and returning headers before the PR to add headers, sure.

[purarue]

Actually, the code block above with has_updated would fail, since I believe when a 304 is returned .json (in _check_response) throws a json.decoder.JSONDecodeError since theres no text

>>> x = requests.get("http://api.jikan.moe/v3/anime/1")
>>> x
<Response [200]>
>>> x.headers["ETag"]
'"772396cc4107d6c5b4c6f169c0e8279b"'
>>> y = requests.get("http://api.jikan.moe/v3/anime/1", headers={"If-None-Match": x.headers["ETag"]})
>>> y.json()
...
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Maybe add a case in _check_response to check if response code is 304 and return {} (or None)?

[abhinavk99]

Off the top of my head, I guess we need another if check in _check_response for 304. If it ends up requiring a lot of changes and you don't have much time, I can take a crack at it later.

Edit: once again I sent this before your edit. My solution would be to set the text to {} so it can be parsed as an empty json.

[purarue]

Alright, we can deal with that when we get there, I'll let you know where Im at later this week