loftwah/linkarooie

Migrate Avatar and Banner Storage to DigitalOcean Spaces

Closed this issue · 5 comments

loftwah commented

Currently, user avatars and banners are stored locally on the server, which increases storage complexity and reduces scalability. Since we already have a DigitalOcean Spaces bucket configured and operational, migrating these assets to be stored and served from the DigitalOcean Spaces would simplify storage management and improve the performance of the application, especially in production environments.

The task involves:

  • Migrating the storage of avatars and banners from local disk to DigitalOcean Spaces.
  • Ensuring all existing assets are transferred seamlessly.
  • Updating the application to reference assets from the new storage location.
  • Improving performance by using a CDN from DigitalOcean Spaces for faster asset delivery.
loftwah commented

Didn't expect to be spending my night on this.

DigitalOcean Spaces Migration Plan

  1. Prepare DigitalOcean Spaces:

    • Create folders: "avatars", "banners", "defaults"
    • Upload default avatar and banner to the "defaults" folder

  2. Update User Model:

    • Modify avatar_url and banner_url methods to return Spaces URLs
    • Update download_and_store_image method to use Spaces for storage

  3. Create DigitalOcean Spaces Service:

    • Implement methods for uploading, downloading, and deleting files

  4. Update Users::RegistrationsController:

    • Modify create and update actions to use the new Spaces service
    • Allow certain attributes to be updated without password confirmation

  5. Update OpenGraphImageGenerator:

    • Modify to use images from Spaces instead of local storage

  6. Create Migration Rake Task:

    • Implement task to migrate existing avatars and banners to Spaces
    • Handle default images, invalid URLs, and non-image URLs

  7. Update Views:

    • Ensure all views are using the correct methods to display images from Spaces

  8. Test Migration:

    • Run migration task in a staging environment
    • Verify that all images are correctly migrated and displayed

  9. Deploy and Monitor:

    • Deploy changes to production
    • Monitor for any issues with image display or uploads

    Am up to 3.

loftwah commented

Updated Migration Plan:

1. Prepare DigitalOcean Spaces:

  • Status: ✅ Completed.

  • Steps Taken:

    • Two buckets identified: one for backups (sqlite-backup-bucket) and one for content (linkarooie).
    • Created folders for images (avatars, banners, defaults).

2. Update User Model:

  • Status: ✅ Implemented but needs testing.

  • What we did:

    • avatar_url and banner_url methods modified to return URLs from the linkarooie bucket.
    • process_image method handles storing and uploading images to Spaces.

  • Next Steps: Test this functionality with real URLs to ensure images are correctly uploaded and URLs are updated in the database.

3. Create DigitalOcean Spaces Service:

  • Status: ✅ Completed.

  • What we did:

    • Created methods to handle uploading, downloading, and deleting files in DigitalOcean Spaces.
    • Fixed the issue with the wrong environment variable (SPACES_BUCKET_IMAGES to SPACES_BUCKET_CONTENT for content).

4. Update Users::RegistrationsController:

  • Status: Not Started.

  • Next Steps:

    • Modify the create and update actions to ensure image uploads and updates work with Spaces.
    • Allow updating user attributes like avatar and banner without requiring a password.

5. Update OpenGraphImageGenerator:

  • Status: Not Started.
  • Next Steps:
    • Modify it to use URLs from DigitalOcean Spaces for image generation.

6. Create Migration Rake Task:

  • Status: ✅ Done.

  • What we did:

    • Implemented the Rake task to migrate existing avatars and banners to DigitalOcean Spaces.
    • Task handles default images and edge cases like invalid URLs or non-image URLs.

7. Update Views:

  • Status: Needs Verification.
  • Next Steps:
    • Ensure views display images from Spaces correctly by using avatar_url and banner_url.

8. Test Migration:

  • Status: In Progress.
  • Next Steps:
    • Continue testing in the Rails console with valid image URLs (like the Twitter image) to confirm the image upload and URL update process works.

9. Deploy and Monitor:

  • Status: Not Yet.

  • Next Steps:

    • Once everything is verified in the staging or dev environment, deploy to production.
    • Monitor for issues with image display or uploading.

Immediate Next Steps for Testing:

  1. Verify the corrected bucket variable:

    • Ensure the SPACES_BUCKET_CONTENT is now being used in the DigitalOceanSpacesService.

  2. Test image upload with a valid URL:

    • Use the Rails console to trigger the process_image method on a user and confirm the image uploads to Spaces.

  3. Check the updated user avatar URL:

    • After running the upload, verify that the avatar URL in the database points to Spaces correctly.
loftwah commented

Finally! Wow this was a nightmare but at least my test coverage is good.

loftwah commented

Just finished then. Lesson learned I guess.