Documentation no longer include JSON response body when using with Rack >= 2.1.0
sikachu opened this issue · 16 comments
Hello,
This issue is pretty much for reporting the incompatibility to the gem author, and hopefully it will help anyone who runs into this problem to understand what's going on.
Basically, after we upgrade our dependencies to use Rack 2.1.1, we noticed that our generated documentation no longer show JSON response but instead showing [binary data]
instead.
Digging in further, we found out that in rack/rack@8c62821, especially this change, MockResponse#body
now creates a buffer and use <<
to join the content together. However, on line 195, the author uses String.new
without specifying the encoding, resulting in Ruby creating a new String with ASCII-8BIT
encoding by default.
As it turns out, rspec_api_documentation relies on string encoding to determine if it should include the response body in the documentation or not:
rspec_api_documentation/lib/rspec_api_documentation/client_base.rb
Lines 90 to 95 in 560c3bd
Hence, the change in Rack broke this conditional.
I've reported this issue to Rack in rack/rack#1486, and hopefully we can solve this soon.
The solution right now for us is to lock Rack to ~> 2.0.8
for now.
Thank you very much.
Any other workaround?
I ran into the same issue recently. I don't really like this solution, but I fixed it using mokey patching:
config/initializers/rspec_api_documentation.rb
module RspecApiDocumentation
class RackTestClient < ClientBase
def response_body
last_response.body.encode("utf-8")
end
end
end
Indeed, it seems to be caused by an encoding issue where utf-8 become ascii-8bit
This seems to also be the result after Rails 6 upgrade.
Just created a new rails 6 app and had this problem, Tao's response above fixed it (#456 (comment))
Tao-Galasse's solution worked but I found that I needed to stick it right before the RspecApiDocumentation.configure
block instead of the initializers directory.
There's an additional issue with endpoints that use send_data
:
Failure/Error: last_response.body.encode("utf-8")
Encoding::UndefinedConversionError:
"\xD3" from ASCII-8BIT to UTF-8
# ./config/initializers/rspec_api_documentation.rb:6:in `encode'
# ./config/initializers/rspec_api_documentation.rb:6:in `response_body'
I've tweaked Tao's fix, it ain't perfect but it works for my projects:
module RspecApiDocumentation
class RackTestClient < ClientBase
def response_body
if last_response.headers["Content-Type"].include?("json")
last_response.body.encode("utf-8")
else
"[binary data]"
end
end
end
end
On my side, I fixed it by two mechanisms :
- @skibox's @Tao-Galasse solution remade mine
- Use
response_body_formatter
like it is intended to filter the different kind of output and make it OK forapitome
Resulting in this spec/support/rspec_api_documentation.rb
file :
# frozen_string_literal: true
# Fix a bug that generate erroneous "[binary data]" not yet fixed on rspec_api_documentation:6.1.0
module RspecApiDocumentation
class RackTestClient < ClientBase
def response_body
body = last_response.body
if body.empty? || last_response.headers['Content-Type'].include?('json')
body.encode('utf-8')
else
'"[binary data]"'
end
rescue Encoding::UndefinedConversionError
'"[binary data]"'
end
end
end
# configure rspec_api_documentation
RspecApiDocumentation.configure do |config|
# ..... some unrelated config here
# Change how the response body is formatted by default
# Is proc that will be called with the response_content_type & response_body
# by default response_content_type of `application/json` are pretty formated.
config.response_body_formatter = lambda do |response_content_type, response_body|
if response_content_type.include?('application/json')
JSON.parse(response_body)
return response_body
elsif response_content_type.include?('text') || response_content_type.include?('txt')
# quote it for JSON Parser in documentation reader like APITOME
return "\"#{response_body}\""
else
return '"[binary data]"'
end
rescue JSON::ParseError
'"[binary data]"'
end
# ... Other unrelated config
end
Just to make sure people notice, this issue would be fixed by #458 🙇♂️
For anyone following this, Ive merged #458 please let me know if this resolves the issue for you. cc @artofhuman
Hope this can be fixed. We need to work with Rails 6
Doesn't work with application/vnd.api+json content type. Probably it's better to use include?('json')
?
@incubus Note the change in #458 does not, in itself alter the gem's behavior.
If you use the default response_body_formatter
, then you will experience the same problem as before.
To solve your problem, please:
- Make sure you use the
master
branch (at least until there's a new release). - Define a custom
response_body_formatter
in your configuration, like so:
RspecApiDocumentation.configure do |config|
config.response_body_formatter =
Proc.new do |content_type, response_body|
if content_type =~ /application\/.*json/
JSON.pretty_generate(JSON.parse(response_body))
else
response_body
end
end
end
Note that the above, compared with the new default (see below) introduced in #458 would stop filtering out potentially binary data.
As the test response_body.encoding == Encoding::ASCII_8BIT
has become unreliable to filter binary data, it will be under your responsibility to figure out, in your context, what you can check to determine whether you want to display a given response or not.
Here's a more complex example:
RspecApiDocumentation.configure do |config|
config.response_body_formatter =
Proc.new do |content_type, response_body|
# http://www.libpng.org/pub/png/spec/1.2/PNG-Rationale.html#R.PNG-file-signature
if response_body[0,8] == "\x89PNG\r\n\u001A\n"
"<img src=\"data:image/png;base64,#{Base64.strict_encode64(response_body)}\" />"
elsif content_type =~ /application\/.*json/
JSON.pretty_generate(JSON.parse(response_body))
elsif response_body.encoding == Encoding::ASCII_8BIT # note 1
"[binary?]" # note 2
else
response_body
end
end
end
Notes:
-
This is the old check, still unreliable, but because it happens after checking for JSON or PNG, then you would be able to display those two properly. You might lose the display of very plain text (the
else
part) if Rack returns everything asASCII_8BIT
(same problem as before). -
To alleviate the above, you might want to return a better string than just
[binary?]
if you aren't sure your test is accurate. You could for example return something like this:<details><summary>[binary?]</summary> #{response_body} </details>
This would look like this:
[binary?]
ˇÿˇ‡��JFIF�����`�`��ˇ·�ÄExif��MM�*�����������������������������J�����������R�(����������ái���������Z�������`�������`������†�����������†���������������ˇ· !http://ns.adobe.com/xap/1.0/� �ˇÌ�8Photoshop 3.0�8BIM��������8BIM�%������‘�åŸè�≤�ÈÄ òϯB~ˇ‚�ËICC_PROFILE������ÿappl� ��mntrRGB XYZ �Ÿ����������acspAPPL����appl������������������ˆ÷������”-appl������������������������������������������������desc�������odscm���x���úcprt�������8wtpt���L����rXYZ���`����gXYZ���t����bXYZ���à����rTRC���ú����chad���¨���,bTRC���ú����gTRC���ú����desc��������Generic RGB Profile������������Generic RGB Profile��������������������������������������������������mluc�����������skSK���(���ÑdaDK���.���¨caES���$���⁄viVN���$���˛ptBR���&���"ukUA���*���HfrFU���(���rhuHU���(���özhTW�������¬nbNO���&���ÿcsCZ���"���˛heIL������� itIT���(���>roRO���$���fdeDE���,���äkoKR�������∂svSE���&���ÿzhCN�������ÃjaJP�������‚elGR���"���¸ptPO���&����nlNL���(���DesES���&����thTH���$���ltrTR���"���êfiFI���(���≤hrHR���(���⁄plPL���,����ruRU���"���.arEG���&���PenUS���&���v�V�a�e�o�b�e�c�n�˝� �R�G�B� �p�r�o�f�i�l�G�e�n�e�r�e�l� �R�G�B�-�b�e�s�k�r�i�v�e�l�s�e�P�e�r�f�i�l� �R�G�B� �g�e�n�Ë�r�i�c�C�•�u� �h�Ï�n�h� �R�G�B� �C�h�u�n�g�P�e�r�f�i�l� �R�G�B� �G�e�n�È�r�i�c�o���0�3�0�;�L�=�8�9� �?�@�>�D�0�9�;� �R�G�B�P�r�o�f�i�l� �g�È�n�È�r�i�q�u�e� �R�V�B�¡�l�t�a�l�·�n�o�s� �R�G�B� �p�r�o�f�i�lê�u(� �R�G�B� Çr_icœè�G�e�n�e�r�i�s�k� �R�G�B�-�p�r�o�f�i�l�O�b�e�c�n�˝� �R�G�B� �p�r�o�f�i�l�‰�Ë�’�‰�Ÿ�‹� �R�G�B� �€�‹�‹�Ÿ�P�r�o�f�i�l�o� �R�G�B� �g�e�n�e�r�i�c�o�P�r�o�f�i�l� �R�G�B� �g�e�n�e�r�i�c�A�l�l�g�e�m�e�i�n�e�s� �R�G�B�-�P�r�o�f�i�l«|º�� �R�G�B� ’�∏\”«|fnê�� �R�G�B� cœèeáNˆN�Ç,� �R�G�B� 0◊0Ì0’0°0§0Î�ì�µ�Ω�π�∫�Ã� �¿�¡�ø�∆�Ø�ª� �R�G�B�P�e�r�f�i�l� �R�G�B� �g�e�n�È�r�i�c�o�A�l�g�e�m�e�e�n� �R�G�B�-�p�r�o�f�i�e�l�B���#�D���%�L� �R�G�B� ���1�H�'�D���G�e�n�e�l� �R�G�B� �P�r�o�f�i�l�i�Y�l�e�i�n�e�n� �R�G�B�-�p�r�o�f�i�i�l�i�G�e�n�e�r�i� �k�i� �R�G�B� �p�r�o�f�i�l�U�n�i�w�e�r�s�a�l�n�y� �p�r�o�f�i�l� �R�G�B���1�I�8�9� �?�@�>�D�8�;�L� �R�G�B�E�D�A� �*�9�1�J�A� �R�G�B� �'�D�9�'�E�G�e�n�e�r�i�c� �R�G�B� �P�r�o�f�i�l�etext����Copyright 2007 Apple Inc., all rights reserved.�XYZ ������ÛR�������œXYZ ������tM��=Ó���–XYZ ������Zu��¨s���4XYZ ������(����ü��∏6curv���������Õ��sf32������B���fiˇˇÛ&���í��˝ëˇˇ˚¢ˇˇ˝£���‹��¿lˇ¿���������"�������ˇƒ���������������������������� �ˇƒ�µ����������������}��������!1A��Qa�"q�2Åë°�#B±¡�R—$3brÇ �����%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyzÉÑÖÜáàâäíìîïñóòôö¢£§•¶ß®©™≤≥¥µ∂∑∏π∫¬√ƒ≈∆«»… “”‘’÷◊ÿŸ⁄·‚„‰ÂÊÁËÈÍÒÚÛÙıˆ˜¯˘˙ˇƒ���������������������������� �ˇƒ�µ����������������w�������!1��AQ�aq�"2Å��Bë°±¡ #3R�br— �$4·%Ò����&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyzÇÉÑÖÜáàâäíìîïñóòôö¢£§•¶ß®©™≤≥¥µ∂∑∏π∫¬√ƒ≈∆«»… “”‘’÷◊ÿŸ⁄‚„‰ÂÊÁËÈÍÚÛÙıˆ˜¯˘˙ˇ€�C�������0��0D000D\DDDD\t\\\\\tåttttttåååååååå®®®®®®ƒƒƒƒƒ‹‹‹‹‹‹‹‹‹‹ˇ€�C�"$$848`44`ÊúÄúÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊÊˇ›����ˇ⁄���������?�È(¢ä�ˇŸ@davidstosik it works, thanks!
I ran into the same issue recently. I don't really like this solution, but I fixed it using mokey patching:
config/initializers/rspec_api_documentation.rb
module RspecApiDocumentation class RackTestClient < ClientBase def response_body last_response.body.encode("utf-8") end end end
Indeed, it seems to be caused by an encoding issue where utf-8 become ascii-8bit
It worked for me. Thank you so much
frozen_string_literal: true
Fix a bug that generate erroneous "[binary data]" not yet fixed on rspec_api_documentation:6.1.0
module RspecApiDocumentation
class RackTestClient < ClientBase
def response_body
body = last_response.body
if body.empty? || last_response.headers['Content-Type'].include?('json')
body.encode('utf-8')
else
'"[binary data]"'
end
rescue Encoding::UndefinedConversionError
'"[binary data]"'
end
end
end
I did a slight variation on this one so I could get the responses to show up as pretty printed:
RspecApiDocumentation.configure do |config|
config.response_body_formatter = lambda do |response_content_type, response_body|
if response_content_type.include?('application/json')
return JSON.pretty_generate(JSON.parse(response_body))
elsif response_content_type.include?('text') || response_content_type.include?('txt')
# quote it for JSON Parser in documentation reader like APITOME
return "\"#{response_body}\""
else
return '"[binary data]"'
end
rescue JSON::ParseError
'"[binary data]"'
end
end