Image not found on first load…

Question

Image not found on first load…

jolantis opened this issue 8 years ago · 17 comments

Hi Fabian, I am testing your two plugins, Imagekit and Imageset. What I see (and how it works) is really great so far!

Though I do get a 404 not found in the console — both in latest stable Chrome (see attached image) and Safari on macOS 10.12.3 — on first load and when no thumbs have been generated yet (thumbs folder is empty).

It is only a message, because the images do show up, and the ‘missing’ images can be opened in a new tab (the thumbs URL).

Refreshing the page makes the error messages disappear. And it only happens when I set imagekit.lazy to true.

Any ideas? Script order? I do load my scripts (including imageset.min.js) via Filament Group's enhance script: https://github.com/filamentgroup/enhance…

Answer 1 · 2017-02-22T17:25:03.000Z

Hey @jolantis,

this is kind of strange … in order to hunt your issue, I would recommend the following:

Maybe the file will be generated, but the wrong header is sent to the browser. Have a look at your dev tools’ network tab to figure out, whether an actual image was sent to the browser rather than Kirby’s error page (just look at mime type and download size of the file).
Try to load ImageKit’s JS files without enhance (I never worked with it, but it seems to be possible to load CSS & JS assets directly without too much hassle).

Hope this brings us closer to what’s causing your issue.

Answer 2 · 2017-02-23T09:22:22.000Z

Hi Fabian,

I included imagekit.js directly in the head, instead of bundling it with the rest of my scripts (concatenate with grunt task) and loading it with enhance.

That did solve the 404's error!

That's the good news, but for performance reasons, I still would like to concatenate imagekit.js with the rest of my scripts, and load it with enhance.

I have to look into this how to do this without getting the 404's… but I am not really a JS guy.

For what it's worth: I was using the lazysizes script before I tried your imagekit/imageset, and I could concatenate + load it with enhance without any issue.

Any input is welcome… thanks

Answer 3 · 2017-02-23T09:37:27.000Z

OK, I was too fast with my conclusion…

I still get the 404's, also with the imagekit.js script included directly in the head.

I there are no thumbs generated yet, and I go the the test page with 13 images, and I start scrolling right after loading, I do get the 404's (for all the thumbs imagekit has not generated one?).

So back to where we started.

I have tried the following to isolate the issue (without avail):

I removed my custom thumb driver (it has the same name as the default ImageMagick driver: ‘im’)
I included imagekit.js right in the head

FWIW: I added screenshots of the Chrome's network tab.

It seems like the browser is requesting the images before imagekit is able to generate them (no 404's when imagekit.lazy is set to false)?

Any other ideas?

If it helps I can put a version online and email the link… although I have to manually delete the thumbs before you have a look — when imagekit has generated the thumbs, you won't get the 404's anymore.

[UPDATE] It is not imageset, but just imagekit (and imagekit.lazy). When I remove the imageset plugin folder, I still get the 404's.

Answer 4 · 2017-02-23T17:32:50.000Z

Could you show me the screenshots from the network tab (don’t see them anywhere here)? Did your browser recieve acutal image data (look at the size of the response, it should be easily possible to detect images through response size). Some browsers will not display images, when sent with a 404 code, others do. Maybe PHP throws a notice and thus prevents PHP from both sending valid image data and from sending the correct header.

Answer 5 · 2017-02-23T17:49:19.000Z

Sorry, forgot to attach the screenshots… here they are.

As you can see the browser received actual image data.

Answer 6 · 2017-02-23T17:58:52.000Z

FWIW After reloading the page (no more 404's) some image headers have a provisional headers are shown message, while others show the ‘normal‘ Request Headers information (see attached screenshots).

Answer 7 · 2017-02-24T19:08:32.000Z

Okay, I had a look at your screenshots as well as at my local setup. I have to admit, I only used Apache for testing and don’t have any experience regarding nginx. However, I’ll try my best to help you with your issue. ;-)

I just pushed a little update, which removes the unnecessary charset information from the Content-Type header as well as the X-Powered-By header and sends an explicit status code of 200 on thumb generation. Please try, if the update solves your issue.

Otherwise, the problems occurding on your dev server might be related to nginx configuration, as I did not get similar issue reports from other users of ImageKit so far. If the update does not help., let’s dig into your config file to find out what’s wrong.

5e109dc

Answer 8 · 2017-02-25T16:50:35.000Z

Okay, thanks for pointing me in the right direction: the nginx configuration. And no the little update made no difference.

It took a while, but I isolated the part of the configuration that's causing the 404's in combination with imagekit.

I am using h5bp's server-configs-nginx configuration files. The following media expires lines are causing the 404's:

# Expire rules for static content

# No default expire rule. This config mirrors that of apache as outlined in the
# html5-boilerplate .htaccess file. However, nginx applies rules by location,
# the apache rules are defined by type. A consequence of this difference is that
# if you use no file extension in the url and serve html, with apache you get an
# expire time of 0s, with nginx you'd get an expire header of one month in the
# future (if the default expire rule is 1 month). Therefore, do not use a
# default expire rule with nginx unless your site is completely static

...

# Media: images, icons, video, audio, HTC
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
  expires 1M;
  access_log off;
  add_header Cache-Control "public";
}

...

(source: https://github.com/h5bp/server-configs-nginx/blob/master/h5bp/location/expires.conf)

If I out-comment these media expire lines, I do not get the 404's anymore!

But now the question is, how can I keep expire times for images and use imagekit without getting the 404's?

Answer 9 · 2017-02-26T11:05:29.000Z

Glad to hear, that you could at least figure out what’s wrong so far. I I pointed out before, I don’t have any experience with nginx, so I cannot really help you with that. Maybe you could open a Thread in the Kirby forum, as we have some other nginx users as well. Someone might be able to find a the solution. I’ll leave this issue open for now and you be pleased, if you could post your final solution as a reply, so I can add it to the troubleshooting section of the readme.

FYI: Currently, ImageKit does not send cache headers on the first request (which generates the thumbnail), as it’s hard to determine the caching settings of a server, as we don’t have direct access to server configuration from PHP. I will try to figure out a way of improving this behavior, but this needs some testing with different server configurations.

Answer 10 · 2017-03-05T10:51:05.000Z

Hi Fabian, I did not have the time last week to look into it any further, so my apologies for responding so late!

I did some more trial and error (and Googling) for the nginx expire rules configuration this morning.

This config block below is really the standard way of setting expires rules for static content/media (not only at h5bp, but in many more examples I have found):

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
  expires 1M;
  access_log off;
  add_header Cache-Control "public";
}

As I mentioned before this block causes the 404's. If I completely out-comment (remove) the above block from the config, the 404's are gone, but no caching is done for the images (that's a big performance hit).

If I leave out the jpg|jpeg part from the ‘location’ line, the 404's are gone.
If I leave in the jpg|jpeg part, but out-comment everything within the block (expires 1M to add_header Cache-Control "public") the 404's will just be there. So it looks like the 404's have not so much to do with what is within the expire rules block, but just with the targeting of the j(e)pg's.

While Googling for the issue, I run into this StackExchange answer and linked blog post:

Please, can you have a look at the above links and solution? Maybe it has some leads to solve this issue in the php code generating the thumbnails on first request (that's when the 404's appear, not on subsequent page visits), because for now I do not know how to solve this in the nginx expire rules, except for not caching images at all or ignoring the 404's…

Answer 11 · 2017-03-07T12:39:46.000Z

Hey @jolantis,

I’ve took some time to read through the blog post and they only difference between those and my own solution is, that they are also send a 304 header, depending on the modified date of a file. This is nice, but does not make too much sense in case of ImageKit, because when thumbnails are requested for the first time, they cannot exist at this time and thus sending a 304 would only make sense, if thumbnail cache has been deleted and a user visited the pages afterwards. But in that case, the modified date of a thumb would not match the cached version from browser cache and the 304 status could would thus never be sent.

But back you you problem: Sorry, I have to admit, that I cannot really help you from this point. I’ve never worked with nginx before and even setting up a quick installation in MAMP on my local machine would mean I had to dig into the topic for at least a couple of hours. I really recommend, that you ask for help in either the Kirby forum or an nginx forum for help. Tell them, that the rules in your nginx config are colliding with images sent through PHP, when a request has to a non-existing file gets redirected to PHP to serve the actual thumbnail. As ImageKit is not the only script that redirects requests to non-existing files to PHP, there might be a lot people who can help you with that. It might also be worth to open an issue in the h5bp repo. I’m pretty sure, there also some nginx users around.

I am very sorry that I cannot help you any further at this point, but I hope you’re accepting my honest answer here. Nevertheless, I would really like to know how this issue can be solved, as some users of ImageKit & nginx will profit from ist. If you don’t want to investigate any further into this and already bought a license, you can send me an email to get a refund.

Answer 12 · 2017-03-07T15:22:51.000Z

@jolantis
does disabling the optimization using c::set('imagekit.optimize', false) remove the initial 404s?

Answer 13 · 2017-03-08T07:46:01.000Z

Hi Fabian, thank you for looking at the blog post. I appreciate your help. I understand that setting up a local nginx environment takes too much of your time. No, I did not buy a license yet (though I want to) — just testing it locally for now.

Until now I did not understand exactly what the issue is, it's above my head. That makes it difficult to post a question about the issue on the Kirby forum or in the h5bp repo. But I hope I can just do that now with your explanation/description of the issue at hand!

I will inform you here if and when I find a solution…

Answer 14 · 2017-03-08T08:02:47.000Z

@bnomei thanks for helping!

I aded the c::set('imagekit.optimize', false) line to explicitly disable optimization (I believe false is the default). Unfortunately that did not solve the initial 404's.

Answer 15 · 2017-03-09T07:55:36.000Z

Hey @fabianmichael, I found the solution to the 404's!

Because I understood the nature of the 404's a bit better, I was able to find the solution right at h5bp's nginx docs.

The expire rule for static content (e.g. location ~* \.(?:jpg|jpeg|gif|png|…)$ { … }) also captures any dynamic requests matching that url pattern and does not hand the request over to the php application in the event of an error.

There are 3 strategies to solve this, but the most simple one is this:

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {

    error_page 404 = /index.php;

    expires 1M;
    access_log off;
    add_header Cache-Control "public";
}

Adding the error_page 404 = /index.php; line to the static files expire rules block makes sure nginx will pass requests for files that don't exist to the (php) application.

This fixes the 404's ;)

(See the h5bp common problems docs for all 3 strategies: https://github.com/h5bp/server-configs-nginx/blob/master/doc/common-problems.md#cannot-dynamically-serve--requests)

Answer 16 · 2017-03-09T15:55:08.000Z

Thanks for your research on this issue, as I’m shure you’re not the only one affected by it. I’ll integrate your solution into the readme, so others will benefit from your solution! :-)

Answer 17 · 2017-09-26T16:24:34.000Z

@jolantis I've too been struck by this issue regarding the dynamically generation of thumbs on a server using Nginx. However, your (thus the one from h5bp) solution does nothing to fix it at my end. I have a more thorough configuration built around the Kirby installations on my server. Complete with GZIP etc.

Whenever you reference error_page 404 = /index.php;, you tell Nginx to redirect every 404 URL inside your matched location block to index.php. Unsurprisingly, this is what happens. But that's it. I just get redirected to the homepage with that URL. Am I missing a hook?

When I completely empty my extra Nginx directives, everything starts working but that's, as you said, not the ideal situation.

Have you anything more to add to fix the issue? Thanks in advance.

EDIT: I got it fixed in the end. I'm using nginx as a proxy, and I had to serve PHP through nginx instead of Apache to get it to recognize my updated configuration. @jolantis's last solution works.