Extension:AWS - https://www.mediawiki.org/wiki/Extension:AWS

What it does: it stores images in Amazon S3 instead of the local directory.

Why is this needed: when images are in S3, Amazon EC2 instance which runs MediaWiki doesn't contain any important data and can be created/destroyed by Autoscaling.

Installation

Note: This version of Extension:AWS requires MediaWiki 1.35+. For older versions of MediaWiki (1.27-1.34) use the following instructions instead: https://github.com/edwardspec/mediawiki-aws-s3/blob/REL1_34/README.md

1) Download the extension: git clone --depth 1 https://github.com/edwardspec/mediawiki-aws-s3.git AWS

2) Move the AWS directory to the "extensions" directory of your MediaWiki, e.g. /var/www/html/w/extensions (assuming MediaWiki is in /var/www/html/w).

3) Create the file /var/www/html/w/composer.local.json with the following contents:

{
	"extra": {
		"merge-plugin": {
			"include": [
				"extensions/AWS/composer.json"
			]
		}
	}
}

4) Run composer update from /var/www/html/w (to download dependencies). If you don't have Composer installed, see https://www.mediawiki.org/wiki/Composer for how to install it.

5) Create an S3 bucket for images, e.g. wonderfulbali234. Note: this name will be seen in URL of images.

6a) If your EC2 instance has an IAM instance profile (recommended), copy everything from "Needed IAM permissions" (see below) to inline policy of the IAM role. See https://console.aws.amazon.com/iam/home#/roles

6b) If your EC2 instance doesn't have an IAM profile, obtain key/secret for AWS API. You'll need to write it in LocalSettings.php (see below).

7) Modify LocalSettings.php (see below).

Configuration in LocalSettings.php

wfLoadExtension( 'AWS' );

// Configure AWS credentials.
// THIS IS NOT NEEDED if your EC2 instance has an IAM instance profile.
$wgAWSCredentials = [
	'key' => '<something>',
	'secret' => '<something>',
	'token' => false
];

$wgAWSRegion = 'us-east-1'; # Northern Virginia

// Replace <something> with the name of your S3 bucket, e.g. wonderfulbali234.
$wgAWSBucketName = "<something>";

// if your images are stored in directory called "some_prefix"
// you can specify an optional prefix
$wgAWSBucketTopSubdirectory="/some_prefix";

If you do not specify credentials via $wgAWSCredentials, they are retrieved using the default credentials chain. This means they are obtained from IAM instance profile (if this EC2 instance has it) or from environmental variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN.

Needed IAM permissions

Replace <something> with the name of your S3 bucket, e.g. wonderfulbali234.

{
        "Effect": "Allow",
        "Action": [
                "s3:*"
        ],
        "Resource": [
                "arn:aws:s3:::<something>/*"
        ]
},
{
        "Effect": "Allow",
        "Action": [
                "s3:Get*",
                "s3:List*"
        ],
        "Resource": [
                "arn:aws:s3:::<something>"
        ]
}

Custom S3 domain

You can use a domain name for images (for example, img.mysite.com). This is needed when you want a CDN (such as CloudFlare) to cache your images. See [https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html Virtual Hosting of Buckets] for details.

1) At your DNS provider, add a CNAME entry. For example, point img.mysite.com to <your-wgAWSBucketName>.s3.amazonaws.com).

2) In LocalSettings.php, set $wgAWSBucketDomain. The following values are supported:

$wgAWSBucketDomain = 'img.mysite.com';

// This will use <bucket-name>.cloudfront.net
$wgAWSBucketDomain = '$1.cloudfront.net';

// Default
$wgAWSBucketDomain = '$1.s3.amazonaws.com';

Migrating images

By default the extension stores all images in the top-level directory of the bucket.

If you are migrating an existing images folder, MediaWiki uses a hashed directory structure. You will need to add this to your LocalSettings.php for the image paths to be generated correctly.

$wgAWSRepoHashLevels = '2'; # Default 0
# 2 means that S3 objects will be named a/ab/Filename.png (same as when MediaWiki stores files in local directories)

$wgAWSRepoDeletedHashLevels = '3'; # Default 0
# 3 for naming a/ab/abc/Filename.png (same as when MediaWiki stores deleted files in local directories)

If your images folder previously was serving multiple wikis split into different subdirectories, you need to set $wgAWSBucketTopSubdirectory. This setting is not recommended for new wikis.

$wgAWSBucketTopSubdirectory = '/something';
# images will be in bucketname.s3.amazonaws.com/something/File.png instead of bucketname.s3.amazonaws.com/File.png.

Troubleshooting

My wiki uses Extension:MultimediaViewer (or shows images as popups), and now they don't work

If you have this issue, attach a CORS policy to your S3 bucket with images. This will allow JavaScript (in this case, popup-showing script of Extension:MultimediaViewer) from the domain where your Wiki is hosted to download the images from Amazon S3 URL. For example, if the domain of your wiki is www.example.com, you can use the following policy:

<CORSConfiguration>
 <CORSRule>
   <AllowedOrigin>http://www.example.com</AllowedOrigin>
   <AllowedMethod>GET</AllowedMethod>
 </CORSRule>
</CORSConfiguration>

Local storage is still used, even though the extension is shown to be installed

This can happen if some settings are missing. Make sure you have at least $wgAWSBucketName and $wgAWSRegion are set.

I'm getting Exception, even though the extension is shown to be installed

This can happen if some settings are missing. Make sure that $wgAWSRegion is set (even if your config doesn't use it, e.g. when using non-Amazon providers).

Non-standard configuration

Using another S3-compatible service (not Amazon S3 itself)

You can use non-Amazon software that supports S3 API (such as Apache CloudStack, Digital Ocean, etc.) instead of Amazon S3 itself. To enable this, add the following lines to LocalSettings.php:

//The url used for the API (PutObject, etc.)
$wgFileBackends['s3']['endpoint'] = 'https://my-custom-url';
//The url used for showing images. $1 is translated to the bucket name.
$wgAWSBucketDomain = '$1.my-custom-url';

Make sure $wgAWSBucketName and $wgAWSRegion are set as well.

lucas-flis/mediawiki-aws-s3