Dirt simple caching....

[gagecarto]

This is more of a question than an issue.

First off, I've had great success with this application for number of projects. Nice work!

2nd, I don't really understand how/where to setup caching. Any suggestion for a beginner route to getting my tiles cached etc?

[tobinbradley]

It depends on your use case.

My main Dirt service is behind a Nginx proxy, and Nginx will let you set up a proxy cache. I just set it for a short period of time - i.e. a few minutes - just to knock down repeat traffic to the back end DB. Lots of folks use Varnish for caching (see an example for pg_tileserv). For large projects you can set up a global CDN via Cloudflare or a similar service to cache things at edge endpoints. And if you're planning on caching a tile set for a long time, like a month or more, you're probably better off rendering the data to MBTiles via QGIS or something and then serving via MBTiles Server or something similar.

There's nothing magical about caching vector tiles. You cache them pretty much how you'd cache any kind of web content.

[gagecarto]

Excellent info @tobinbradley - thanks for giving me some homework and things to take a run at. I am using NGINX as well so these methods should work great.

[gagecarto]

Hey @tobinbradley - Not sure this is the best place for this question but figure I'll give it a shot.

Any suggestion for my NGINX server block params below for setting up the caching etc? My tiles are served from the root of my url (e.g. www.myurl.com) on port 3000 via DIRT. I also have something else running on 3500 but I am guessing that does not matter. Everything is SSL through Certbot.

NGINX starts OK with the params below but nothing gets cached. I am verifying this by looking in the cache directory (e.g. /var/lib/nginx/cache ) and through inspecting the response from my tile requests.

Do you see anything major I am missing here to get the caching working?

proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=mvtcache:10m inactive=60m;
server {
  server_name tiles.myurl.com;
 proxy_cache mvtcache;
location /{
    proxy_pass http://localhost:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache mvtcache;
    proxy_cache_valid 200 1d;
  }
location /queries/{
    proxy_pass http://localhost:3500/db/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/tiles.myurl.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/tiles.myurl.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
    if ($host = tiles.myurl.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
        listen 80 ;
        listen [::]:80 ;
        server_name tiles.myurl.com;
        return 404; # managed by Certbot
}

[tobinbradley]

You've got a lot going on there. The example from nginx looks like this:

http {
    proxy_cache_path  /data/nginx/cache  levels=1:2    keys_zone=STATIC:10m
    inactive=24h  max_size=1g;
    server {
        location / {
            proxy_pass             http://1.2.3.4;
            proxy_set_header       Host $host;
            proxy_buffering        on;
            proxy_cache            STATIC;
            proxy_cache_valid      200  1d;
            proxy_cache_use_stale  error timeout invalid_header updating
                                   http_500 http_502 http_503 http_504;
        }
    }
}

I haven't seen any examples where the cache location in the /var/lib folder. Does the cache folder exist, and can nginx write there?

[gagecarto]

Hey.. Thanks for taking a look. I was able to get things working. NGINX did indeed have permissions on the folder. The block below works. I am now going to tease it apart and find which params made the difference. Cheers

location / {
        proxy_cache mvtcache;
        proxy_buffering on;
        proxy_cache_valid 200 10m;
        proxy_ignore_headers Expires;
        proxy_ignore_headers X-Accel-Expires;
        proxy_ignore_headers Cache-Control;
        proxy_ignore_headers Set-Cookie;
        proxy_hide_header X-Accel-Expires;
        proxy_hide_header Expires;
        proxy_hide_header Cache-Control;
        proxy_hide_header Pragma;
        add_header X-Proxy-Cache $upstream_cache_status;
        proxy_pass http://localhost:3000;
    }