spacecatninja/craft-imager-x

Images not writing to S3

Closed this issue · 5 comments

vaughn-taylor commented

Imager-X version: 4.2.4
Craft CMS version: 4.6.0
PHP version: 8.3

Images are transforming and uploading to S3 using the native Craft transform, however, nothing is transferred to S3 from the Imager-X plugin.

This is my imager-x.php:

return [
'suppressExceptions' => false,
'imagerSystemPath' => '@webroot/assets/imager/',
'imagerUrl' => 'https://mybucket.s3.us-east-1.amazonaws.com/imager/',
'removeTransformsOnAssetFileops' => false,
'storages' => ['aws'],
'storageConfig' => [
    'aws'  => [
        'accessKey' => '*********',
        'secretAccessKey' => '*********',
        'region' => 'us-east-1',
        'bucket' => 'mybucket',
        'folder' => 'imager',
        'requestHeaders' => array(),
        'storageType' => 'standard',
        'public' => true,
        'cloudfrontInvalidateEnabled' => true,
        'cloudfrontDistributionId' => '*********',
    ]
],...

This is my imager-x-generate.php:

return [
'volumes' => [
    'assetsHero' => ['transformHero']
]
];

This is my imager-x-transforms.php:

return [
'transformHero' => [
    'transforms' => [
        ['width' => 600],
        ['width' => 1800],
    ],
    'defaults' => [
        'ratio' => 16/9,
        'jpegQuality' => 80
    ],
    'configOverrides' => [
        'fillTransforms' => true,
        'fillInterval' => 300,
    ]
],
];

When I upload an image to the "Assets - Hero" volume (handle: assetsHero) in Craft, I can see the temporary transformations being created in 'web/assets/transforms' directory. These disappear which I assume means that they've been offloaded to the S3 bucket?

When I inspect the folders on S3, I see the original image in a 'hero' folder along with a couple transforms which I believe are thumbnails generated by Craft. However, when I look in the 'imager' folder, there are no transforms.

I enabled debug in Craft, and I don't see anything that looks like an error.

I do believe I was seeing transforms at some point yesterday? But late yesterday, I upgraded Craft and Imager-X to the latest versions, so I'm not sure if either of those updates are responsible? I'm using NYStudio's S3 configuration which is working well for all the native Craft transforms.

I looked at a couple related Issues here on Github, and there was a question about whether the region was set correctly. I assure you that my region - 'us-east-1' is correct. If it weren't correct, I wouldn't see uploads to the hero folder from the S3 plugin.

aelvan commented

Hi, let's see if we can figure it out!

When I upload an image to the "Assets - Hero" volume (handle: assetsHero) in Craft, I can see the temporary transformations being created in 'web/assets/transforms' directory. These disappear which I assume means that they've been offloaded to the S3 bucket?

You write web/assets/transforms, but did you mean web/assets/imager, ie the imagerSystemPath? If so, the transformed images are suppose to be there after it has been uploaded to S3, this will act as the cache that Imager uses to figure out if a transform has been made or not. The external storage is just an additional storage location. If the file is removed, it means that the S3 upload was unsuccessful, and the transformed file is removed so that you are able to correct the error and try again.

From your description ("When I upload an image...") it sounds like you've only tested this with the auto generate functionality? Any errors that occur when doing this, will not end up in the debug toolbar, only in the queue logs, since the transforms are done in a job. For the sake of debugging, create a plain twig template (with valid html and body tags so the debug toolbar works), and do {% set transformedImages = craft.imagerx.transformImage(someImageAsset, 'transformHero') %}. Now check your debug toolbar.

I'm using NYStudio's S3 configuration which is working well for all the native Craft transforms.

That configuration has very restrictive permissions, try removing 'public' => true, from your config.

vaughn-taylor commented

@aelvan yes, I'm sorry I typed that wrong. The path is 'imagerSystemPath' => '@webroot/assets/imager/'

I just uploaded an image and it showed in @webroot/assets/imager/assetshero/100 temporarily then disappeared. I then checked mybucket/imager on S3 and there's nothing there. However, I see an image and transforms in mybucket/hero on S3.

I added a twig template with a valid HTML wrapper and added this to the page:

{% set image = craft.assets().one() %}
{% set transformedImage = craft.imagerx.transformImage(image, { width: 1000 }) %}
<img src="{{ transformedImage.url }}">

For testing, I also tried just a simple output of an image like this:

{% set image = craft.assets().one() %}
<img src="{{ image.getUrl({ width: 1000 }) }}">

The debug toolbar is showing a 403 Forbidden response and AccessDenied for the Imager-X image. I inspected the code and found this:

<img src="https://mybucket.s3.us-east-1.amazonaws.com/imager/assetshero/103/LT-hot-honey-apple-tea_d3867277feb154defec9b24a5714fadb.jpg">
<img src="https://mydistribution.cloudfront.net/hero/transforms/_1000xAUTO_crop_center-center_none/LT-hot-honey-apple-tea.jpg">

The first image could not be found – this is the one generated by Imager-X. The second image rendered properly. But notice that the domain URL for the Imager-X image is not the CloudFront domain. I assume this is the issue?

I tried using the CloudFront domain for the imagerUrl variable, but that didn't work.

I also tried removing 'public' => true from imager-x.php but that didn't work either.

aelvan commented

But notice that the domain URL for the Imager-X image is not the CloudFront domain. I assume this is the issue?

This is due to the imagerUrl being set to point to the s3 bucket, you need to use the cloudfront distributions url. Imager won't modify imagerUrl in any way, it assumes that you have configured it correctly. But this has nothing to do with transforms not being uploaded.

Remove everything except...

{% set image = craft.assets().one() %}
{% set transformedImage = craft.imagerx.transformImage(image, { width: 1000 }) %}

...from you template. The rest is just noise.

Slight error on my part, in your aws configuration set 'public' => false and try again.

If the file still is removed from the imagerSystemPath and not uploaded to S3, check for errors in the "Logs" section of your debug toolbar.

vaughn-taylor commented

@aelvan I managed to get it working with your advice. For the record, I'd like to share the knowledge here.

If you're using the S3 configuration from NYStudio, which is intentionally restrictive and uses CloudFront distribution, there are a couple crucial settings in imager-x.php :

  1. Make sure to set your imagerUrl to the CloudFront URL. This may seem obvious to some, but it's not addressed in the Imager-X documentation. Since your CloudFront distribution is handling the image , the CloudFront URL must be used.

The imagerUrl should look like this:
'imagerUrl' => 'https://d*************.cloudfront.net/imager/'

And not like this:
'imagerUrl' => 'http://s3-eu-west-1.amazonaws.com/imager/'

  1. The public configuration setting in imager-x.php should be set to false. I assume this is because NYStudio's S3 configuration does not make the S3 bucket publicly accessible. Instead, it uses a group policy to handle asset distribution. This is what your settings should look like in imager-x.php:

    'storages' => ['aws'],
    'storageConfig' => [
    'aws' => [
    'accessKey' => 'your-access-key',
    'secretAccessKey' => 'your-secret-access-key',
    'region' => 'your-region-handle',
    'bucket' => 'your-bucket-name',
    'folder' => 'imager',
    'requestHeaders' => array(),
    'storageType' => 'standard',
    'cloudfrontInvalidateEnabled' => true,
    'public' => false,
    'cloudfrontDistributionId' => 'your-distribution-id',
    ]
    ],

aelvan commented

Great, thanks for sharing!

  • 👍1