Tips & Tricks: Using Glide & S3

[tao]

The documentation to get this working is super confusing and doesn't apply to Amazon S3? I struggled with it for ages and after so much difficulty I think the docs need to be improved or a new Tips & Tricks article should be written with some better guidelines. Perhaps the instructions were tested on DO Spaces instead as a few things don't work?

I have a rough draft below with the steps that I remember to get it working:

---

1. Create the bucket glide.statamic.com

   • Enable public access
   • Enable static website hosting

2. Create a CloudFront instance for pretty URLs

   • Use the S3 bucket as the Origin: glide.statamic.com.s3.us-east-1.amazonaws.com
   • Setup CloudFront with alt domain name glide.statamic.com and create Amazon SSL certs.
   • Redirect HTTP to HTTPS
   • Enable GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE (important if we want to write images to the bucket!)

   The cache might need to be removed or use legacy cache settings if you re-generate the images (to refocus them?) as the CloudFront cache is 24 hours by default (I'm not sure how the UUIDs change so it needs more testing)

   Then update your domain and create a CNAME record so glide.statamic.com points to the new CloudFront domain name given xxx.cloudfront.net.

3. Add a filesystem to config/filesystems.php, and make sure to use a brand new s3 driver. Make sure you create a new S3 bucket for glide ONLY as running php please glide:clear will empty the entire bucket!

        's3-glide' => [
            'driver' => 's3',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
            'bucket' => env('AWS_GLIDE_BUCKET'),
            'url' => env('AWS_GLIDE_URL'),
        ],

It seems to work if you don't include AWS_ENDPOINT or AWS_USE_PATH_STYLE_ENDPOINT.

One of the biggest issues I had is that the visibility: public prevents you from uploading images... however that might be resolved if this bug is closed? (https://github.com/thephpleague/flysystem-aws-s3-v3/issues/176)

            // Asset permissions
            // 'visibility' => 'public', # https://github.com/thephpleague/flysystem-aws-s3-v3/issues/176

4. Add your ENV variables

AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=us-east-1
AWS_GLIDE_BUCKET=glide.statamic.com
AWS_GLIDE_URL=https://glide.statamic.com

5. Update the assets.php file:

        'cache' => 's3-glide',
        // 'cache_path' => public_path('img'), # not necessary for AWS S3

6. If you are using SSG you can override Glide*

    'glide' => [
        // Override Glide on SSG as we use an AWS CDN instead
        // See: https://github.com/statamic/ssg/pull/104
        'override' => false,
    ],

7. Setup Cache for images already generated*

To stop the images be re-generated each time you run SSG on Github actions, which will have no knowledge of whats on the S3 bucket as the local cache is in the storage folder, you need a database of images already on S3?

(Does SSG still generate new images that are not in the cache yet if you use override: false or just print a URL without saving the image?)

I'm still working on this but I used an sqlite database and committed it to my repository with Spock, so at-least my Github Action will know what images already exist and not override them each time it runs?

        'glide' => [
            'driver' => 'database',
            'table' => 'cache',
            'connection' => 'sqlite',
        ],

Then in my Github actions I can do the following:

     - name: Verify imagick installed
        run: |
          php -r 'echo "imagick is ".(extension_loaded("imagick")?"":"not ")."installed\n";'
      - name: Generate static site
        env:
          APP_ENV: production
          DB_CONNECTION: sqlite
          DB_DATABASE: database.sqlite
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
          AWS_GLIDE_BUCKET: ${{ secrets.AWS_GLIDE_BUCKET }}
          AWS_GLIDE_URL: ${{ secrets.AWS_GLIDE_URL }}
          STATAMIC_LICENSE_KEY: ${{ secrets.STATAMIC_LICENSE_KEY }}
        run: |
          php please stache:warm
          php please ssg:generate --workers=2

And maybe it could be better if I downloaded and uploaded the database file to another S3 bucket before and after each Github Action to keep it up to date.

9. Test

If you reload the page with assets it should actually show the pretty CloudFront URL and the asset will have been written to the bucket. Something like:

https://glide.statamic.com/containers/aws/articles/covers/Surfing.png/0083becac8719c41540a99fef188f288.png

Instead of:

http://glide.statamic.com.s3.us-east-1.amazonaws.com/containers/aws/articles/covers/Surfing.png/0083becac8719c41540a99fef188f288.png

Any other config that I tried didn't work or just outputted a blank URL...

👀

[imacrayon]

These steps combine with this Glide tag patch made big improvements to my project. I ended up just using a file store for the Glide cache instead of SQLite:

'glide' => [
    'driver' => 'file',
    'path' => resource_path('glide'),
],

Page loads are 2x faster with image-heavy content and static builds sped up by 3x.