Large XHR response objects can exceed the maximum header size.
zachgibb opened this issue · 18 comments
If I respond to route with a very large object, tests waiting for that response will fail with ERR_EMPTY_RESPONSE
cy
.server()
.route("POST", /route/, reallyLargeObject).as("willFail")
.get(".submit").click()
.wait("@willFail")Okay this is not a quick or easy fix.
Node sets a constant maximum total header size at 80kb for all headers. There is no way to get around this. We can't toss the data in the request body either, since that would be altering the entire request.
Probably the only way to do this is transfer the data over websockets while the server begins handling the request.
That means managing all kinds of state, but this would work. There are some other features of the XHR refactor that would also introduce websockets, so I'll hold off on implementing this until it can go in with some new architecture.
In the mean time you'll have to reduce the response sizes to less than 80kb, or just use a fixture and not an alias.
So just move reallyLargeObject to a fixture and update your route
cy.route("POST", /route/, "fixture:reallyLargeObject")Trying to find a workaround for this, e.g. manipulating the request, or the response, or the server object. The available hooks fire after the request is sent, and after the response is delivered to the app, so there's no opportunity to influence it. Also haven't found any way to access the server object, e.g. to override a method. Any suggestions? Using a fixture doesn't help, because it's fixed. ;)
If your fixture exceeds the maximum headers you'll need to modify your app code, such as using a conditional if window.Cypress exists, and if so, to use a property on that (on the XHR response callback)
Then ahead of time in your test code, you can use a fixture and set that property on the Cypress object so your app code can pick it up.
The key takeaway here is that Cypress is just javascript and it has native access to everything. You can share properties and data. You can collaborate between Cypress and your application code.
If your application code is "hardwired" to take a specific path, you can just modify it to open a seam to which you can use Cypress to modify that behavior.
Just realized I could wrap onreadstatechange during onRequest, to flow the data into the app.
cy.route({
url: '/api/bookmarks/bookmark*',
method: 'GET',
onRequest: xhr => {
var orsc = xhr.xhr.onreadystatechange;
xhr.xhr.onreadystatechange = function() {
if (this.readyState === 4) {
Object.defineProperty(this, 'response', {
writable: true
});
this.response = content;
}
orsc.apply(this, arguments);
};
},
response: 'placeholder',
});The Cypress console shows response 'placeholder', but otherwise this functions as a workaround.
Yup that would work.
You could also bind to onload instead of onreadystatechange. Would avoid the readyState conditional
Thank you for your suggestions.
For guys coming from google. Here is our workaround:
put this in e.g. integration/utils/loadLargeFixture.js
export default function loadLargeFixture(url, response, method = 'GET') {
return cy.route({
url,
method,
onRequest: xhr => {
const originalOnLoad = xhr.xhr.onload;
xhr.xhr.onload = function() {
Object.defineProperty(this, 'response', {
writable: true
});
this.response = response;
originalOnLoad.apply(this, xhr);
};
}
});
}Import the json and the loadLargeFixture function in your test:
import largeFixture from '../fixtures/veryLargeJson.json';
import loadLargeFixture from '../utils/loadLargeFixture';
describe('demo', () => {
it('should work with large fixture', () => {
cy.server();
loadLargeFixture('/api/large', largeFixture).as('data');
// do stuff
cy.wait('@data')
// do stuff
});
});I spent some time stuck on this today. 😢 It wasn't very obvious because I made a change to my fixtures that must have JUST put it over size limit for responses. So working yesterday -> not working today.
The ERR_EMPTY_RESPONSE would not display on each failure either, instead if would just say:
CypressError: The network request for this XHR could not be made. Check your console for the reason.
I was able to inspect the network panel and see that the response had a size of 0 bytes, which clued me in on it being this issue.
This only breaks when I generate a bunch of data for an array response (like to check pagination), so would be nice to have fixed.
hah just found a super simple solution:
before I had
cy.fixture(`${endpoint}.json`).as(`fixture${endpoint}`);
cy.route(path, `@fixture${endpoint}.json`)
.as(endpoint);now:
cy.route(path, `fixture:${endpoint}.json`)
.as(endpoint);Cleaner and it works :)
This error mixed with the no support for fetch() ruined my afternoon, and made me think I was crazy. Please add note in docs where it is suggested to use cy.fixture aliases when it seems to not be great for large jsons.
Just not using aliases like @tolicodes suggested fixed everything.
I just ran into this today! @tolicodes method worked perfectly for me with a 1Mb fixture!
@jennifer-shehane i'm getting the same CypressError: The network request for this XHR could not be made. Check your console for the reason.
Looking at the solution from @brian-mann at the top; I'm confused about the solution and why what I'm doing won't work... I'm not using an alias.
options = {
method: 'POST',
status: 200,
url: '/playlist/search'
}
cy.fixture('playlist/search/playlist-search.dto.mock.json').then((data) => {
options['response'] = data;
cy.route(options);
});Cheers
@jennifer-shehane @brian-mann
I'm using an nedb implementation for simulate database inside my tests, so I have many custom functions running in and out all the time.
I've found something interesting, as the response sent through the server is meant to be in the body, why the response is sitting in the headers? Is there any reason to do that ? Because of this, the header gets much, much bigger than it needs
Is this a behavior or something else ?
i've attached a print.
[EDIT]
I've managed to make the header less than 80kb and make it work again, but, still, can we make it work not by sending responses through the header ?
So I unfortunately and traumatically ran into this bug as well.
I was writing some new tests with new fixtures (programmatic fixtures and not static ones) and the cy.route + cy.wait failed with the ERR_EMPTY_RESPONSE error.
Unfortunately, I thought it was an issue on my side so I started to debug it and after rigorous testing found that the issue relates to big response size passed to cy.route.
As I mentioned before - Our fixtures are programmatic, which means that we don't have static json files holding fixtures and rather we have functions that we call and return as customizable fixtures. Therefore, @tolicodes solution doesn't seem to be relevant for us (unless I missed something about how to use cypress fixtures.
Additionally, I've tried using the loadLargeFixture function suggested by @ankri but it failed with XHR AssertionError.
- Is there any way to workaround this issue with my functional fixtures?
- I really think there should be an assertion internal in the
cy.routeimplementation that throws a descriptive error when called with above-threshold size response.
Similarly to @nitzansan none of the workarounds with xhr events worked for my programmatic data.
What did work is a nasty hack of pre-creating a fixture programmatically:
cy.writeFile('cypress/fixtures/my_test_response.json', bigReponseObject);
cy.route('/whatever', 'fixture:my_test_response.json');
Can work as long as unique file names used and the fixtures saved this way are .gitignore'ed.
The code for this is done in cypress-io/cypress#4720, but has yet to be released.
We'll update this issue and reference the changelog when it's released.
The fixture: fix by @tolicodes gets my request to actually load, instead of timeout, however it still takes over a minute and a half for a 22MB response (I realise that's large, but our app requires it and we'll probably have even larger ones), which is frankly too slow when trying to run quite a few tests.
Released in 3.5.0.